DEV-1381 use delta computation for dailies with hathifiles-database & report on those changes

[moseshll]

• Rename MonthlyUpdate class to DeltaUpdate
• Change hathifiles_database_full_update to use DeltaUpdate for monthly and update hathifiles
• Add statistics method for evaluating actual work done for a given hathifile
• Each hathifile is now processed in its own tempdir instead of one for all of the files to be processed (may mitigate excessive disk usage on the first of month).
• Using additional comm tricks we can get a count of records added vs updated, at the expense of additional time and storage. Fortunately, these additional operations won't happen if you don't call delta_update_obj.statistics.
• HATHIFILES_MYSQL_CONNECTION replaced with kwargs defaulting to ENV
• All HATHIFILES_MYSQL_* vars migrated to MARIADB_HATHIFILES_RW_*
• New section of README documenting ENV variables.
• Deprecated exe/ files removed (hathifiles_database_full_update is the only survivor).

[moseshll]

I have a reservation about allowing database credentials from ENV to be overridden by the DB::Connection kwargs since those values are not available to the Dumper class: it always uses ENV.

Maybe just add a note to the README that relying on ENV is the preferred way to go.

[aelkiss]

Haven't yet reviewed; based on the description I think before deploying (even in testing) we need to get https://github.com/hathitrust/ht_tanka/tree/database-secrets merged.

[aelkiss]

The database env var / connection changes make sense to me. I need to spend some more time looking at the delta computation.

[aelkiss]

This all looks great to me; tests make sense; I appreciate all the comments/documentation in the delta_update class.

As previously mentioned we will need to make sure that the hathifiles pod has sufficient working space before doing this and also add in the new secrets/env vars (https://github.com/hathitrust/ht_tanka/pull/129).

This should be useful for holdings in terms of looking at possible new/changed items and updating its mapping from ocn to clusters as needed; thinking about the best way to get that data to holdings is future work.

[aelkiss]

Reminders:

• un-pin this gem from https://github.com/hathitrust/holdings-backend after merge

[aelkiss]

FYI @niquerio This is a significant change to how hathifiles-database works on a daily basis. No rush to update to this version right away when merged, but you may want to take a look especially after we test this out on our end and make sure it all works.