Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEV-573: Fix broken phase lookup for Shared Print reporting purposes #286

Merged
merged 1 commit into from
Oct 12, 2023

Conversation

mwarin
Copy link
Contributor

@mwarin mwarin commented Sep 8, 2023

Problem (https://hathitrust.atlassian.net/browse/DEV-573):

We are currently using committed_date to determine which "phase" a commitment belongs to. Meaning, when we want to e.g. look up commitments in phase 3, we have to look up "what's the date associated with phase 3" and query on that committed_date. This, in combination with sloppy date formatting and servers in different timezones, has led to issues with timezone confusion.

Proposed solution:

  • We add an integer field called phase to lib/clusterable/commitment.rb.
  • Keep committed_date, but don't use it as a proxy for phase.
  • Write and run a piece of single-use code that can add correct phase values to commitments, based on date.
  • Then make any code that loads commitments require that the user specifies phase.
  • Add a mongo index for commitments.phase.

Steps toward solution:

  • Enabling bin/inspect_ocn.rb to look up multiple ocns, and output them in a format that bash phctl.sh load cluster-file can load, for testing purposes.
  • Added test suite fixture file for commitments (copied from production using the fix to bin/inspect_ocn.rb).
  • Added phase as a field to Clusterable::Commitment and a corresponding index.
  • Simplified SharedPrint::Finder and updated SharedPrint::Phases.
  • Added a script to get all distinct committed_date from production: lib/shared_print/select_distinct_phases.rb
  • Added a script to set phase on commitments that have a specific committed_date: lib/shared_print/phase_updater.rb

Guide to files changed:

bin/inspect_ocn.rb:
Enable lookup of multiple ocns.
Enable output of multiple clusters.
This makes it easier to copy clusters from production and loading them in dev/test (phctl load cluster-file)

lib/clusterable/commitment.rb:
Added phase field and validation.

lib/shared_print/finder.rb:
Enable lookup by phase.
Simplify some of the stuff that was using committed_date.

lib/shared_print/phase_3_validator.rb
Sets phase and committed_date to appropriate values for phase 3.
Broke apart run into load_all & load_one so I could test loading more directly.

lib/shared_print/phase_updater.rb:
Script to set phase on commitments based on their committed_date.

lib/shared_print/select_distinct_phases.rb:
In order to set phase based on committed_date, we must first have a list of all the committed_date

lib/shared_print/phases.rb:
Added phase constants.

spec/fixtures/shared_print_umich_phase123.json:
3 clusters with commitments, for test purposes.

spec/reports/phase3_oclc_registration_spec.rb:
Using phase instead of committed_date.

spec/shared_print/finder_spec.rb:
Using phase instead of committed_date.

spec/shared_print/phase_3_validator_spec.rb
Added test for load_one, broke out reused vars.

@coveralls
Copy link

coveralls commented Sep 8, 2023

Coverage Status

coverage: 95.011% (-0.05%) from 95.058% when pulling 39ccbda on DEV-573-bugfix into 710e98b on main.

@liseli
Copy link

liseli commented Sep 12, 2023

Martin, the change looks good. You could retrieve the documents of specific dates using the following query:
{"commitments.committed_date": { $gte: ISODate( "2019-02-28T00:00:00.000+0000" ) }}

@mwarin mwarin marked this pull request as ready for review September 14, 2023 21:48
@mwarin mwarin requested review from aelkiss and liseli September 14, 2023 21:48
lib/cluster.rb Show resolved Hide resolved
Copy link
Member

@aelkiss aelkiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all makes sense to me.

@mwarin

This comment was marked as outdated.

@mwarin mwarin changed the title Fix broken phase lookup for shared print DEV-573: Fix broken phase lookup for shared print Oct 10, 2023
@mwarin mwarin changed the title DEV-573: Fix broken phase lookup for shared print DEV-573: Fix broken phase lookup for Shared Print reporting purposes Oct 10, 2023
@mwarin mwarin merged commit 65963cc into main Oct 12, 2023
@aelkiss aelkiss deleted the DEV-573-bugfix branch January 7, 2025 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants