Ensure uniqueness of load table names

[stacimc]

Fixes

Fixes WordPress/openverse#1322 by @stacimc

Description

Some ingestion days for the metropolitan_museum_reingestion_workflow have been failing due to DuplicateTable errors. This is because the load table names are generated with the pattern: provider_data_{media_type}_{provider_name}_{timestamp}_{day_shift} -- but the very long provider name for this DAG causes the table name to be truncated in Postgres, such that the day_shift and part of the timestamp (which are required for uniqueness) are cut off.

This PR changes the order of the component parts of the table name such that provider_name is on the end, and is therefore the part that will get truncated if it is too long. This guarantees that media_type, timestamp, day_shift, and the first part of the provider name will fit.

Functionally, this means that part of 'reingestion` gets cut off:

    # The generated table name on `main`: note that part of the timestamp and the entire dayshift is cut off,
    # meaning that collisions can easily happen
    provider_data_image_metropolitan_museum_reingestion_20230222T20

    # The new table name: note that only the end of the provider name is cut off, and the table name is still
    # guaranteed unique across ingestion days
    provider_data_image_96_20230222T204839_metropolitan_museum_rein

Uniqueness is only an issue if we create two DAGs that have very long provider names, which only differ in the final few characters.

Alternatives considered

1. Give just this particular DAG a shorter, custom DAG id (like met_museum_reingestion_workflow or metropolitan_reingestion_workflow. The problem could still arise in the future.
2. Removing the _reingestion suffix from the load table name. This would work because the 'normal' ingestion flow does not append day_shift, so the two DAGs would still be distinct -- but it is less human readable, and we could still encounter the problem with long provider names in the future.

Testing Instructions

Run the Metropolitan reingestion DAG -- make sure to set a very low INGESTION_LIMIT before you start. Check the logs to make sure the load table names are being generated as expected.

I also ran a few other DAGs, including wikimedia_commons_workflow and wikimedia_reingestion_workflow.

Checklist

• My pull request has a descriptive title (not a vague title like
  Update index.md).
• My pull request targets the default branch of the repository (main) or
  a parent feature branch.
• My commit messages follow best practices.
• My code follows the established code style of the repository.
• I added or updated tests for the changes I made (if applicable).
• I added or updated documentation (if applicable).
• I tried running the project locally and verified that there are no visible
  errors.
• I ran the DAG documentation generator (if applicable).

Developer Certificate of Origin

Developer Certificate of Origin

Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

[krysal]

Makes sense. LGTM!

[AetherUnbound]

Clever thinking!

Do you think it might make sense to move the provider_data_ prefix entirely and favor it for something shorter? I know that'd make this a bigger change, but certainly that feels like a few characters we could easily throw out 😄 If we replaced it with load_, that might give us a few more characters before truncation 🙂 Not necessary though, this is good as-is IMO!

[stacimc]

@AetherUnbound It's funny that you say that -- my original implementation was literally to change provider_data_ to load_ and remove the _reingestion suffix 😄 I changed it to the current approach to keep the readability of the suffix and prevent problems down the line with future long DAG names, but we could definitely keep the shortened prefix part as well. As it happens I've already tested that and it's a tiny change 💯

[AetherUnbound]

As it happens I've already tested that and it's a tiny change 100

Ha, incredible, and you're right! What a nice, small change 💃🏼