Add step to update treasury report when upload is invalidated

[as1729]

Current State

When a new UploadValidation is created the record the treasury report is unaware of these changes and remains in a stale state.

Expected State

When a new UploadValidation is created the record needs to be communicated to the treasury generation process to update the treasury report files if an upload has to be added/removed from the report.

Implementation details

eng sub-tasks needed:

1. De couple send report from report generation
2. Force re-create treasury report when file is invalidated or overwritten Delete treasury report when requested force-regenerate #510

Additional notes:

1. Here is where we are creating the initial UploadValidation record.
2. Add a call to start execution of the treasury-generation step function and add the appropriate arguments to the call based on UploadValidation.passed == true or UploadValidation.passed == false.

If UploadValidation.passed == true then the details of the upload-file must be passed into the uploadsToAdd list here
If Uploadvalidation.passed == false then the details of the upload-file must be passed into the uploadsToRemove list here

projectLambdaPayload = {
  uploadValidation.upload.expenditureCategory.name: {
    organization: uploadValidation.upload.agency.organization
    user: uploadValidation.upload.uploadedBy
    outputTemplateId: uploadValidation.upload.reportingPeriod.outputTemplateId
    ProjectType: uploadValidation.upload.expenditureCategory.name
    uploadsToAdd: { uploadValidation.upload.agencyId: { objectKey: ...., createdAt: uploadValidation.upload.createdAt, filename: uploadValidation.upload.filename } } // this could be empty if passed == false
    uploadsToRemove:{ uploadValidation.upload.agencyId: { objectKey: ...., createdAt: uploadValidation.upload.createdAt, filename: uploadValidation.upload.filename } } // this will be empty if passed == true
  }
}

subrecipientLambdaPayload  = {} // this will be empty

createArchiveLambdaPayload = {
  zip: {
    organization: uploadValidation.upload.agency.organization
  }
}

3. Once the payload information is derived you will mimic the code written at the end of the sendTreasuryReport function here:

     input = {
        '1A': {},
        '1B': {},
        '1C': {},
        Subrecipient: {},
        zip: {},
        email: {},
        ...projectLambdaPayload,
        ...subrecipientLambdaPayload,
        ...createArchiveLambdaPayload,
      }
      await startStepFunctionExecution(
        process.env.TREASURY_STEP_FUNCTION_ARN,
        `Force-kick-off-${uuidv4()}`,
        JSON.stringify(input)
      )

// JS

const crypto = require("crypto")
const encoder = new TextEncoder();
const payload_str = JSON.stringify(payload)
const key = crypto.createHash('sha256').update(encoder.encode(payload_str)).digest("hex")


// Python

import hashlib
payload_str = json.dumps(payload)
key = hashlib.sha256(payload_str).hexdigest()

[as1729]

Three things need to happen here:

1. Add a FIFO queue for upload validation to treasury report generation
2. Add a dynamodb lock for send-report-by-email and report generation
3. De couple send report from report generation
4. Indicate an easy way on the front-end for users to know whether the report is going to be stale based on dynamodb lock.
5. Indicate within the downloaded report what is included. Either some excel metadata or separate CSV for the users.

[ClaireValdivia]

Note - this should address both when a file is invalidated as well as when it is not the most recent valid file.

[ClaireValdivia]

Keeping open for now while we determine if another issue is needed to subtract projects from treasury report if the file is invalidated or overwritten. @as1729 to assess

[ClaireValdivia]

We expect #517 to address the issues with invalidate. Will test once that issue is done and then close this ticket out.