A script to identify invalid clusters in Print Holdings MongoDB

[mwarin]

Overview

What is it? A script to identify the clusters in the Print Holdings MongoDB that are invalid. Clusters are the main organizing unit in this database. I would like to run this regularly, perhaps together with the db backup.

Why is it? An invalid cluster doesn't let you save any information on it, until you make it valid. We currently don't have good insight into this, and that could become a problem. It may in fact already be a problem and we just don't know. I'm also going to be doing stuff to the schema, which is one of the ways you can end up with invalid clusters, so that's why now.

How big is it? The diff size may look big, but when you break it down it's:

• 1 file of code: bin/cluster_validator.rb
• 1 file of tests: spec/cluster_validator_spec.rb
• 2 fixture files: spec/fixtures/single_cluster_invalid.json & spec/fixtures/single_cluster_valid.json

The fixture files are identical except for a single field (commitment.phase). The valid file has a 1 (valid phase), the invalid 999 (invalid phase). Why is this? Well, it's hard to make existing clusters invalid by just manipulating them, because the existing validations stop you from setting invalid values. However, data that are loaded directly into the db bypass these validations. The spec file uses the cluster-file loader to get valid/invalid clusters loaded, so it can perform tests on invalid clusters.

Instead of putting this in #286 (which is already large) I broke this out into its own PR.

What kind of feedback am I looking for?

Just a general look at the validator and it's spec. It's all new code, so there isn't any tangle.

Testing

The tests run fine on github but if you want to run them yourself it is:

1. git clone this repo
2. git checkout this branch
3. Spin it up and enter its shell:

bash bin/setup/setup_dev.sh
docker-compose up -d pushgateway
docker-compose run --rm -e MONGOID_ENV=test dev bash

4. $ bundle exec rspec spec/cluster_validator_spec.rb
5. $ exit
6. $ docker-compose down

[coveralls]

coverage: 95.034% (+0.02%) from 95.011% when pulling 16eebf7 on cluster-validator into 65963cc on main.

[aelkiss]

Makes sense; all looks pretty straightforward. Main question is if the buffering is necessary or not.