Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: updates the inputs to match the latest lambda payload schema #395

Merged
merged 24 commits into from
Aug 28, 2024

Conversation

as1729
Copy link
Contributor

@as1729 as1729 commented Aug 21, 2024

This PR ensures the following:

  1. The Send Treasury Report by Email button builds the correct input payload for the step function:
image
  1. Updates the lambda function that creates the zip archive to accept a simpler payload structure
  2. Adds helper documentation to run a step function locally
  3. Updates the aws.ts module to use the newer pattern for exporting functions\

Local Test Results

The results here show that when the following input is sent to the step-function locally that it invokes the lambdas with the events as shown.

Input to the State Function where we expect 1A,1B,1C to have an empty object:

{"1A":{},"1B":{},"1C":{},"Subrecipient":{"organization":{"id":1,"preferences":{"current_reporting_period_id":1}},"user":{"email":"[email protected]","id":1},"outputTemplateId":1},"zip":{"organization":{"id":1,"preferences":{"current_reporting_period_id":1}}}}

Output from each of the lambda log-groups

adityasrbookpro:cpf-reporter adityasridhar$ bash clear_group.sh 
awslocal logs get-log-events --log-group-name /aws/lambda/localstack-lambda-zip --log-stream-name 2024/08/27/[$LATEST]40b771854f0759d44ed58a997256c934 --output table
adityasrbookpro:cpf-reporter adityasridhar$ awslocal logs get-log-events --log-group-name /aws/lambda/localstack-lambda-zip --log-stream-name 2024/08/27/[\$LATEST]40b771854f0759d44ed58a997256c934 --output table
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|                                                                                           GetLogEvents                                                                                            |
+------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+
|                                        nextBackwardToken                                       |                                        nextForwardToken                                          |
+------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+
|  b/00000000000000000000000000000000000000000000000000000000                                    |  f/00000000000000000000000000000000000000000000000000000005                                      |
+------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+
||                                                                                             events                                                                                              ||
|+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
|| ingestionTime |                                                                            message                                                                             |   timestamp    ||
|+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
||  1724769416151|  START RequestId: fc233fa1-29de-4da8-8e9d-070868e72a08 Version: $LATEST                                                                                        |  1724769416109 ||
||  1724769416151|  2024-08-27T14:36:56.094Z    fc233fa1-29de-4da8-8e9d-070868e72a08    INFO    Welcome to Zip Lambda                                                                      |  1724769416111 ||
||  1724769416151|  2024-08-27T14:36:56.096Z    fc233fa1-29de-4da8-8e9d-070868e72a08    INFO    zip event {^M  organization: { id: 1, preferences: { current_reporting_period_id: 1 } }^M}   |  1724769416113 ||
||  1724769416151|  2024-08-27T14:36:56.101Z    fc233fa1-29de-4da8-8e9d-070868e72a08    INFO    Ending logs for Zip Lambda                                                                 |  1724769416115 ||
||  1724769416151|  END RequestId: fc233fa1-29de-4da8-8e9d-070868e72a08                                                                                                           |  1724769416118 ||
||  1724769416151|  REPORT RequestId: fc233fa1-29de-4da8-8e9d-070868e72a08      Duration: 10.23 ms      Billed Duration: 11 ms  Memory Size: 128 MB     Max Memory Used: 128 MB                  |  1724769416120 ||
|+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
adityasrbookpro:cpf-reporter adityasridhar$ bash clear_group.sh 
awslocal logs get-log-events --log-group-name /aws/lambda/localstack-lambda-subrecipient --log-stream-name 2024/08/27/[$LATEST]1ada799f230e766f78422070fea90d27 --output table
adityasrbookpro:cpf-reporter adityasridhar$ awslocal logs get-log-events --log-group-name /aws/lambda/localstack-lambda-subrecipient --log-stream-name 2024/08/27/[\$LATEST]1ada799f230e766f78422070fea90d27 --output table
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|                                                                                                                                    GetLogEvents                                                                                                                                    |
+--------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|  nextBackwardToken                                                 |  b/00000000000000000000000000000000000000000000000000000000                                                                                                                                                   |
|  nextForwardToken                                                  |  f/00000000000000000000000000000000000000000000000000000005                                                                                                                                                   |
+--------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
||                                                                                                                                      events                                                                                                                                      ||
|+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
|| ingestionTime |                                                                                                                     message                                                                                                                     |   timestamp    ||
|+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
||  1724769414262|  START RequestId: e802bb51-a3c3-4fc2-9ea6-5f2b86bef389 Version: $LATEST                                                                                                                                                                         |  1724769414228 ||
||  1724769414262|  2024-08-27T14:36:54.194Z    e802bb51-a3c3-4fc2-9ea6-5f2b86bef389    INFO    Welcome to Subrecipient Lambda                                                                                                                                              |  1724769414232 ||
||  1724769414262|  2024-08-27T14:36:54.195Z    e802bb51-a3c3-4fc2-9ea6-5f2b86bef389    INFO    Subrecipient event {^M  organization: { id: 1, preferences: { current_reporting_period_id: 1 } },^M  user: { email: '[email protected]', id: 1 },^M  outputTemplateId: 1^M}   |  1724769414236 ||
||  1724769414262|  2024-08-27T14:36:54.207Z    e802bb51-a3c3-4fc2-9ea6-5f2b86bef389    INFO    Ending logs for Subrecipient Lambda                                                                                                                                         |  1724769414240 ||
||  1724769414262|  END RequestId: e802bb51-a3c3-4fc2-9ea6-5f2b86bef389                                                                                                                                                                                            |  1724769414245 ||
||  1724769414262|  REPORT RequestId: e802bb51-a3c3-4fc2-9ea6-5f2b86bef389      Duration: 18.84 ms      Billed Duration: 19 ms  Memory Size: 128 MB     Max Memory Used: 128 MB                                                                                                   |  1724769414249 ||
|+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
adityasrbookpro:cpf-reporter adityasridhar$ bash clear_group.sh 
awslocal logs get-log-events --log-group-name /aws/lambda/localstack-lambda-project-1A --log-stream-name 2024/08/27/[$LATEST]ffd6af81305c908bb90371aace0cfacf --output table
adityasrbookpro:cpf-reporter adityasridhar$ awslocal logs get-log-events --log-group-name /aws/lambda/localstack-lambda-project-1A --log-stream-name 2024/08/27/[\$LATEST]ffd6af81305c908bb90371aace0cfacf --output table
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|                                                                                    GetLogEvents                                                                                     |
+-----------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
|                                    nextBackwardToken                                    |                                     nextForwardToken                                      |
+-----------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
|  b/00000000000000000000000000000000000000000000000000000000                             |  f/00000000000000000000000000000000000000000000000000000005                               |
+-----------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
||                                                                                      events                                                                                       ||
|+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
|| ingestionTime |                                                                     message                                                                      |   timestamp    ||
|+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
||  1724769414256|  START RequestId: d23f655f-f92a-4ce2-a6b9-fd4ff5a2a82d Version: $LATEST                                                                          |  1724769414223 ||
||  1724769414256|  2024-08-27T14:36:54.201Z    d23f655f-f92a-4ce2-a6b9-fd4ff5a2a82d    INFO    Welcome to Project 1A Lambda                                                 |  1724769414226 ||
||  1724769414256|  2024-08-27T14:36:54.202Z    d23f655f-f92a-4ce2-a6b9-fd4ff5a2a82d    INFO    1A event {}                                                                  |  1724769414230 ||
||  1724769414256|  2024-08-27T14:36:54.209Z    d23f655f-f92a-4ce2-a6b9-fd4ff5a2a82d    INFO    Ending logs for Project 1A Lambda                                            |  1724769414233 ||
||  1724769414256|  END RequestId: d23f655f-f92a-4ce2-a6b9-fd4ff5a2a82d                                                                                             |  1724769414237 ||
||  1724769414256|  REPORT RequestId: d23f655f-f92a-4ce2-a6b9-fd4ff5a2a82d      Duration: 11.22 ms      Billed Duration: 12 ms  Memory Size: 128 MB     Max Memory Used: 128 MB    |  1724769414240 ||
|+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
adityasrbookpro:cpf-reporter adityasridhar$ bash clear_group.sh 
awslocal logs get-log-events --log-group-name /aws/lambda/localstack-lambda-project-1B --log-stream-name 2024/08/27/[$LATEST]3d8bc762cec301c8a8734ec33dc653ec --output table
adityasrbookpro:cpf-reporter adityasridhar$ awslocal logs get-log-events --log-group-name /aws/lambda/localstack-lambda-project-1B --log-stream-name 2024/08/27/[\$LATEST]3d8bc762cec301c8a8734ec33dc653ec --output table
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|                                                                                    GetLogEvents                                                                                     |
+-----------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
|                                    nextBackwardToken                                    |                                     nextForwardToken                                      |
+-----------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
|  b/00000000000000000000000000000000000000000000000000000000                             |  f/00000000000000000000000000000000000000000000000000000005                               |
+-----------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
||                                                                                      events                                                                                       ||
|+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
|| ingestionTime |                                                                     message                                                                      |   timestamp    ||
|+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
||  1724769413743|  START RequestId: fadb59eb-a2bc-470c-9c8c-175ce2b289c3 Version: $LATEST                                                                          |  1724769413517 ||
||  1724769413743|  2024-08-27T14:36:53.487Z    fadb59eb-a2bc-470c-9c8c-175ce2b289c3    INFO    Welcome to Project 1B Lambda                                                 |  1724769413546 ||
||  1724769413743|  2024-08-27T14:36:53.490Z    fadb59eb-a2bc-470c-9c8c-175ce2b289c3    INFO    1B event {}                                                                  |  1724769413576 ||
||  1724769413743|  2024-08-27T14:36:53.497Z    fadb59eb-a2bc-470c-9c8c-175ce2b289c3    INFO    Ending logs for Project 1B Lambda                                            |  1724769413605 ||
||  1724769413743|  END RequestId: fadb59eb-a2bc-470c-9c8c-175ce2b289c3                                                                                             |  1724769413635 ||
||  1724769413743|  REPORT RequestId: fadb59eb-a2bc-470c-9c8c-175ce2b289c3      Duration: 26.79 ms      Billed Duration: 27 ms  Memory Size: 128 MB     Max Memory Used: 128 MB    |  1724769413664 ||
|+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
adityasrbookpro:cpf-reporter adityasridhar$ bash clear_group.sh 
awslocal logs get-log-events --log-group-name /aws/lambda/localstack-lambda-project-1C --log-stream-name 2024/08/27/[$LATEST]8fc5c489b78ebe0088defffbdea326bb --output table
adityasrbookpro:cpf-reporter adityasridhar$ awslocal logs get-log-events --log-group-name /aws/lambda/localstack-lambda-project-1C --log-stream-name 2024/08/27/[\$LATEST]8fc5c489b78ebe0088defffbdea326bb --output table
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|                                                                                    GetLogEvents                                                                                     |
+-----------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
|                                    nextBackwardToken                                    |                                     nextForwardToken                                      |
+-----------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
|  b/00000000000000000000000000000000000000000000000000000000                             |  f/00000000000000000000000000000000000000000000000000000005                               |
+-----------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
||                                                                                      events                                                                                       ||
|+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
|| ingestionTime |                                                                     message                                                                      |   timestamp    ||
|+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
||  1724769414134|  START RequestId: 2d839dff-c770-4396-8cfe-8775a0066dca Version: $LATEST                                                                          |  1724769414061 ||
||  1724769414134|  2024-08-27T14:36:54.043Z    2d839dff-c770-4396-8cfe-8775a0066dca    INFO    Welcome to Project 1C Lambda                                                 |  1724769414065 ||
||  1724769414134|  2024-08-27T14:36:54.046Z    2d839dff-c770-4396-8cfe-8775a0066dca    INFO    1C event {}                                                                  |  1724769414069 ||
||  1724769414134|  2024-08-27T14:36:54.048Z    2d839dff-c770-4396-8cfe-8775a0066dca    INFO    Ending logs for Project 1C Lambda                                            |  1724769414073 ||
||  1724769414134|  END RequestId: 2d839dff-c770-4396-8cfe-8775a0066dca                                                                                             |  1724769414078 ||
||  1724769414134|  REPORT RequestId: 2d839dff-c770-4396-8cfe-8775a0066dca      Duration: 17.74 ms      Billed Duration: 18 ms  Memory Size: 128 MB     Max Memory Used: 128 MB    |  1724769414082 ||
|+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------+----------------+|
adityasrbookpro:cpf-reporter adityasridhar$ 

@github-actions github-actions bot added bug Something isn't working javascript python labels Aug 21, 2024
Copy link

github-actions bot commented Aug 21, 2024

QA Summary

See our documentation for tips on how to resolve failing QA checks.

QA Check Result
🌐 Web Tests
🔗 API Tests
🐍 Python Tests
📏 ESLint
🧼 Ruff
🧹 TFLint

Test Coverage

Coverage report for api suite
St File % Stmts % Branch % Funcs % Lines Uncovered Line #s
🔴 All files 48.13 30.29 55.55 48.56
🔴  src 0 100 0 0
🔴   server.ts 0 100 0 0 6-13
🟢  src/directives/requireAuth 100 100 100 100
🟢   requireAuth.ts 100 100 100 100
🟡  src/directives/skipAuth 50 100 0 50
🟡   skipAuth.ts 50 100 0 50 13
🔴  src/functions 0 100 0 0
🔴   graphql.ts 0 100 0 0 15-27
🔴  src/functions/processValidationJson 42.16 54.54 44.44 41.46
🟢   processValidationJson.scenarios.ts 100 100 100 100
🔴   processValidationJson.ts 41.46 54.54 44.44 40.74 59-98,118-119,165,177-178,193-196,201-242,255-256,270-309,321-324,332
🔴  src/graphql 0 100 100 0
🔴   agencies.sdl.ts 0 100 100 0 1
🔴   expenditureCategories.sdl.ts 0 100 100 0 1
🔴   inputTemplates.sdl.ts 0 100 100 0 1
🔴   organizations.sdl.ts 0 100 100 0 1
🔴   outputTemplates.sdl.ts 0 100 100 0 1
🔴   projects.sdl.ts 0 100 100 0 1
🔴   reportingPeriods.sdl.ts 0 100 100 0 1
🔴   subrecipientUploads.sdl.ts 0 100 100 0 1
🔴   subrecipients.sdl.ts 0 100 100 0 1
🔴   uploadValidations.sdl.ts 0 100 100 0 1
🔴   uploads.sdl.ts 0 100 100 0 1
🔴   users.sdl.ts 0 100 100 0 1
🔴   validationRuleses.sdl.ts 0 100 100 0 1
🔴  src/lib 13 9.28 11.7 13.32
🟡   auth.ts 64.7 48.38 57.14 67.34 70-71,77-78,94-95,117,124,127,132-139,163,167
🔴   aws.ts 25.42 18.75 25 25.42 53-58,74-97,121-123,150-171,186-272
🟢   constants.ts 100 100 100 100
🔴   db.ts 45.45 50 50 45.45 15-35,41,43,50
🔴   ec-codes.ts 0 100 100 0 1
🟢   logger.ts 100 100 100 100
🔴   persist-upload.js 0 0 0 0 16-295
🔴   preconditions.ts 0 0 0 0 2-3
🔴   records.js 0 0 0 0 12-214
🔴   templateRules.ts 0 0 0 0
🔴   tracer.ts 0 100 100 0 5-14
🔴   validate-upload.js 0 0 0 0 18-790
🟢   validation-error.ts 83.33 100 50 83.33 22
🔴   validation-rules.js 0 0 0 0 6-194
🟡  src/services/agencies 67.34 50 80 67.34
🟢   agencies.scenarios.ts 100 100 100 100
🟡   agencies.ts 65.21 50 75 65.21 40-51,60-64,97-98,104,113-121
🟡  src/services/expenditureCategories 78.57 66.66 88.88 78.57
🟢   expenditureCategories.scenarios.ts 100 100 100 100
🟡   expenditureCategories.ts 77.77 66.66 88.88 77.77 30-34,49-52,60,91
🟡  src/services/inputTemplates 77.77 66.66 85.71 77.77
🟢   inputTemplates.scenarios.ts 100 100 100 100
🟡   inputTemplates.ts 76.92 66.66 85.71 76.92 25-29,39-40,50,85
🟡  src/services/organizations 76.81 88.88 55.55 76.81
🟢   organizations.scenarios.ts 100 100 100 100
🟡   organizations.ts 75.75 88.88 50 75.75 36-40,75,147-177,185,203-227
🟢  src/services/outputTemplates 82.85 66.66 85.71 82.85
🟢   outputTemplates.scenarios.ts 100 100 100 100
🟢   outputTemplates.ts 82.35 66.66 85.71 82.35 26-30,40-41,51,114
🟡  src/services/passage 74.07 62.5 100 74.07
🟡   passage.ts 74.07 62.5 100 74.07 18-19,65-76
🟡  src/services/projects 80 100 62.5 80
🟢   projects.scenarios.ts 100 100 100 100
🟡   projects.ts 78.57 100 62.5 78.57 45-51
🟡  src/services/reportingPeriods 69.23 70 54.54 69.23
🟢   reportingPeriods.scenarios.ts 100 100 100 100
🟡   reportingPeriods.ts 68.42 70 54.54 68.42 25-29,39-40,55-58,74,105-121
🟢  src/services/subrecipientUploads 88.88 83.33 85.71 88.88
🟢   subrecipientUploads.scenarios.ts 100 100 100 100
🟢   subrecipientUploads.ts 86.36 83.33 80 86.36 64,94-99
🟢  src/services/subrecipients 92.85 100 88.88 92.85
🟢   subrecipients.scenarios.ts 100 100 100 100
🟢   subrecipients.ts 90 100 81.81 90 53-58
🟡  src/services/uploadValidations 57.14 100 14.28 57.14
🟢   uploadValidations.scenarios.ts 100 100 100 100
🟡   uploadValidations.ts 53.84 100 14.28 53.84 10,16,30,38,45-48
🟢  src/services/uploads 91.34 71.42 88.63 91.34
🟢   uploads.scenarios.ts 100 100 100 100
🟢   uploads.ts 88.31 71.42 75 88.31 37,103,131-145,258-262
🟡  src/services/users 78.78 62.5 92 78.78
🟢   users.scenarios.ts 100 100 100 100
🟡   users.ts 77.41 62.5 88.88 77.41 208,216-225,238-241,263-265,274-278,296-297,311-314,332-334,342-343,348,357-360
🟢  src/services/validationRuleses 85.71 100 71.42 85.71
🟢   validationRuleses.scenarios.ts 100 100 100 100
🟢   validationRuleses.ts 84.61 100 71.42 84.61 43-48
Coverage report for web suite
St File % Stmts % Branch % Funcs % Lines Uncovered Line #s
🔴 All files 10.41 11.62 8.74 9.89
🟢  api/src/lib 100 100 100 100
🟢   constants.ts 100 100 100 100
🔴  web/src 28.57 18.75 66.66 28.57
🔴   App.tsx 0 0 0 0 3-36
🟢   Routes.tsx 100 100 100 100
🟡   auth.ts 50 50 100 50 19-24
🔴   entry.client.tsx 0 0 100 0 10-22
🔴  web/src/auth 7.14 0 4.16 7.14
🔴   localAuth.ts 9.09 0 8.33 9.09 39-68,76-80
🔴   passageAuth.ts 5 0 0 5 22-25,31-60
🔴  web/src/components/Agency/Agencies 0 100 0 0
🔴   Agencies.tsx 0 100 0 0 9-21
🔴  web/src/components/Agency/AgenciesCell 0 100 0 0
🔴   AgenciesCell.tsx 0 100 0 0 8-39
🔴  web/src/components/Agency/Agency 0 0 0 0
🔴   Agency.tsx 0 0 0 0 10-78
🔴  web/src/components/Agency/AgencyCell 0 100 0 0
🔴   AgencyCell.tsx 0 100 0 0 7-27
🔴  web/src/components/Agency/AgencyForm 0 0 0 0
🔴   AgencyForm.tsx 0 0 0 0 25-45
🔴  web/src/components/Agency/EditAgencyCell 0 100 0 0
🔴   EditAgencyCell.tsx 0 100 0 0 10-59
🔴  web/src/components/Agency/NewAgency 0 100 0 0
🔴   NewAgency.tsx 0 100 0 0 9-35
🟢  web/src/components/Navigation 100 60 100 100
🟢   Navigation.tsx 100 60 100 100 24-57
🔴  web/src/components/Organization/EditOrganizationCell 0 100 0 0
🔴   EditOrganizationCell.tsx 0 100 0 0 13-64
🔴  web/src/components/Organization/EditOrganizationForm 0 0 0 0
🔴   EditOrganizationForm.tsx 0 0 0 0 27-41
🔴  web/src/components/Organization/NewOrganization 0 100 0 0
🔴   NewOrganization.tsx 0 100 0 0 9-37
🔴  web/src/components/Organization/NewOrganizationForm 0 0 0 0
🔴   NewOrganizationForm.tsx 0 0 0 0 25-54
🔴  web/src/components/Organization/Organization 0 0 0 0
🔴   Organization.tsx 0 0 0 0 10-70
🔴  web/src/components/Organization/OrganizationCell 0 100 0 0
🔴   OrganizationCell.tsx 0 100 0 0 7-28
🔴  web/src/components/Organization/OrganizationPickListsCell 40 0 27.27 36.36
🟡   OrganizationPickListsCell.mock.ts 50 100 0 100
🔴   OrganizationPickListsCell.stories.tsx 0 0 0 0 6-32
🟡   OrganizationPickListsCell.tsx 64.28 100 50 58.33 14-16,47-70
🔴  web/src/components/Organization/Organizations 0 100 0 0
🔴   Organizations.tsx 0 100 0 0 9-21
🔴  web/src/components/Organization/OrganizationsCell 0 100 0 0
🔴   OrganizationsCell.tsx 0 100 0 0 8-37
🔴  web/src/components/OutputTemplate/EditOutputTemplateCell 0 100 0 0
🔴   EditOutputTemplateCell.tsx 0 100 0 0 18-81
🔴  web/src/components/OutputTemplate/NewOutputTemplate 0 0 0 0
🔴   NewOutputTemplate.tsx 0 0 0 0 17-126
🔴  web/src/components/OutputTemplate/OutputTemplate 0 0 0 0
🔴   OutputTemplate.tsx 0 0 0 0 17-97
🔴  web/src/components/OutputTemplate/OutputTemplateCell 0 100 0 0
🔴   OutputTemplateCell.tsx 0 100 0 0 17-47
🔴  web/src/components/OutputTemplate/OutputTemplateForm 0 0 0 0
🔴   OutputTemplateForm.tsx 0 0 0 0 18-63
🔴  web/src/components/OutputTemplate/OutputTemplates 0 0 0 0
🔴   OutputTemplates.tsx 0 0 0 0 18-94
🔴  web/src/components/OutputTemplate/OutputTemplatesCell 0 100 0 0
🔴   OutputTemplatesCell.tsx 0 100 0 0 18-52
🔴  web/src/components/ReportingPeriod/EditReportingPeriodCell 0 100 0 0
🔴   EditReportingPeriodCell.tsx 0 100 0 0 13-74
🔴  web/src/components/ReportingPeriod/NewReportingPeriod 0 100 0 0
🔴   NewReportingPeriod.tsx 0 100 0 0 9-35
🔴  web/src/components/ReportingPeriod/ReportingPeriod 0 0 0 0
🔴   ReportingPeriod.tsx 0 0 0 0 12-101
🔴  web/src/components/ReportingPeriod/ReportingPeriodCell 0 100 0 0
🔴   ReportingPeriodCell.tsx 0 100 0 0 7-33
🔴  web/src/components/ReportingPeriod/ReportingPeriodForm 0 0 0 0
🔴   ReportingPeriodForm.tsx 0 0 0 0 18-43
🔴  web/src/components/ReportingPeriod/ReportingPeriods 0 0 0 0
🔴   ReportingPeriods.tsx 0 0 0 0 13-96
🔴  web/src/components/ReportingPeriod/ReportingPeriodsCell 0 100 0 0
🔴   ReportingPeriodsCell.tsx 0 100 0 0 8-43
🟡  web/src/components/ReportingPeriodsCell 57.14 0 60 50
🟢   ReportingPeriodsCell.mock.ts 100 100 100 100
🔴   ReportingPeriodsCell.stories.tsx 0 0 0 0 6-32
🟢   ReportingPeriodsCell.tsx 100 100 100 100
🔴  web/src/components/Subrecipient/SubrecipientTableUploadLinksDisplay 0 0 0 0
🔴   SubrecipientTableUploadLinksDisplay.stories.tsx 0 100 100 0 5-82
🔴   SubrecipientTableUploadLinksDisplay.tsx 0 0 0 0 5-77
🔴  web/src/components/Subrecipient/Subrecipients 0 0 0 0
🔴   Subrecipients.tsx 0 100 0 0 5-8
🔴   columns.tsx 0 0 0 0 7-93
🔴  web/src/components/Subrecipient/SubrecipientsCell 0 100 0 0
🔴   SubrecipientsCell.tsx 0 100 0 0 7-62
🔴  web/src/components/TableBuilder 0 0 0 0
🔴   DebouncedInput.tsx 0 0 0 0 13-32
🔴   Filter.tsx 0 0 0 0 6-15
🔴   TableBuilder.tsx 0 0 0 0 22-70
🔴   TableHeader.tsx 0 0 0 0 5-42
🔴   TableRow.tsx 0 100 0 0 3-7
🟡  web/src/components/TemplateUploadReportingPeriodCell 55 0 55.55 47.05
🟢   TemplateUploadReportingPeriodCell.mock.ts 100 100 100 100
🔴   TemplateUploadReportingPeriodCell.stories.tsx 0 0 0 0 11-37
🟢   TemplateUploadReportingPeriodCell.tsx 100 100 100 100
🔴  web/src/components/TreasuryGeneration/DownloadTreasuryFiles 0 100 0 0
🔴   DownloadTreasuryFiles.tsx 0 100 0 0 6-30
🔴  web/src/components/TreasuryGeneration/NewTreasuryGeneration 0 100 0 0
🔴   NewTreasuryGeneration.tsx 0 100 0 0 8-39
🔴  web/src/components/TreasuryGeneration/NewTreasuryGenerationForm 0 0 0 0
🔴   NewTreasuryGenerationForm.tsx 0 0 0 0 20-31
🔴  web/src/components/Upload/EditUploadCell 0 100 0 0
🔴   EditUploadCell.tsx 0 100 0 0 10-66
🔴  web/src/components/Upload/NewUpload 0 100 0 0
🔴   NewUpload.tsx 0 100 0 0 7-35
🔴  web/src/components/Upload/Upload 0 0 0 0
🔴   Upload.stories.tsx 0 100 100 0 5-93
🔴   Upload.tsx 0 0 0 0 11-61
🔴  web/src/components/Upload/UploadCell 0 100 0 0 ...[Comment body truncated]

Copy link

github-actions bot commented Aug 22, 2024

Terraform Summary

Step Result
🖌 Terraform Format & Style
⚙️ Terraform Initialization
🤖 Terraform Validation
📖 Terraform Plan

Hint: If "Terraform Format & Style" failed, run terraform fmt -recursive from the terraform/ directory and commit the results.

Output

Validation Output
Success! The configuration is valid.


Plan Summary
CHANGE RESOURCE
add aws_s3_object.origin_dist_artifact["static/js/app.9ca3946a.js"]
aws_s3_object.origin_dist_artifact["static/js/app.9ca3946a.js.LICENSE.txt"]
update aws_ecs_service.console
aws_s3_object.origin_dist_artifact["200.html"]
aws_s3_object.origin_dist_artifact["build-manifest.json"]
aws_s3_object.origin_dist_artifact["chunk-references.json"]
aws_s3_object.origin_dist_artifact["index.html"]
module.lambda_function-cpfCreateArchive.aws_lambda_function.this[0]
module.lambda_function-cpfValidation.aws_lambda_function.this[0]
module.lambda_function-graphql.aws_lambda_function.this[0]
module.lambda_function-processValidationJson.aws_lambda_function.this[0]
module.lambda_function-subrecipientTreasuryReportGen.aws_lambda_function.this[0]
module.lambda_function-treasuryProjectFileGeneration.aws_lambda_function.this[0]
module.treasury_generation_step_function.aws_sfn_state_machine.this[0]
recreate aws_ecs_task_definition.console
aws_s3_object.lambda_artifact-graphql
aws_s3_object.lambda_artifact-processValidationJson
aws_s3_object.lambda_artifact-python
module.lambda_function-cpfCreateArchive.aws_lambda_permission.current_version_triggers["StepFunctionTrigger"]
module.lambda_function-cpfValidation.aws_lambda_permission.current_version_triggers["S3BucketNotification"]
module.lambda_function-graphql.aws_lambda_permission.current_version_triggers["APIGateway"]
module.lambda_function-processValidationJson.aws_lambda_permission.current_version_triggers["S3BucketNotification"]
module.lambda_function-subrecipientTreasuryReportGen.aws_lambda_permission.current_version_triggers["StepFunctionTrigger"]
module.lambda_function-treasuryProjectFileGeneration.aws_lambda_permission.current_version_triggers["S3BucketNotification"]
module.lambda_function-treasuryProjectFileGeneration.aws_lambda_permission.current_version_triggers["StepFunctionTrigger"]
delete aws_s3_object.origin_dist_artifact["static/js/app.81b5490f.js"]
aws_s3_object.origin_dist_artifact["static/js/app.81b5490f.js.LICENSE.txt"]

Pusher: @as1729, Action: pull_request_target, Workflow: Continuous Integration

@github-actions github-actions bot added the documentation Improvements or additions to documentation label Aug 26, 2024
@as1729 as1729 marked this pull request as ready for review August 26, 2024 16:59
@as1729 as1729 requested a review from a team as a code owner August 26, 2024 16:59
@as1729 as1729 requested a review from laurelkorwin August 26, 2024 17:01
@as1729 as1729 enabled auto-merge (squash) August 26, 2024 17:03
@as1729 as1729 disabled auto-merge August 27, 2024 14:41
@as1729 as1729 enabled auto-merge (squash) August 27, 2024 14:41
// The EC code was never added. This is the time to initialize it.
uploadsByEC[upload.expenditureCategory.code] = {
organization: {
id: organization.id,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit but any reason not to just do organization here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the organization here is of type Organization which contains many other fields like name the usual timestamps, etc so figured I'll pair it down to only the relevant fields to send to the step-function.

Although this brings up a good point- maybe when I query for the organization here I can only select these specific fields that way I don't have to re-define this everywhere.

CreateArchiveLambdaPayload.model_validate({"Payload": data})
organizationObj = {
# "id": "1234", Missing a required field
"preferences": {"current_reporting_period_id": 5678},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

supernit: could we make this reporting period ID a global and use it across test cases since we're using the same one?

Copy link
Contributor Author

@as1729 as1729 Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the ideal I'd like to get to, but for some reason I couldn't quite get it working with how to pass parameters from step-functions to each of the lambda invocations.

Here's the ideal payload to invoke the step-function:

{
     common_metadata: { organization: { id: 99, preferences: { current_reporting_period: 2 } } },
     project1A: { ... details for project 1A only ... }
     project1B: { ... details for project 1B only ... }
     project1C: { ... details for project 1C only ... }
     subrecipient: { ... details for subrecipient only ... }
     zip: { ... details for zip creation only .... }
}

Here's the current payload to invoke the step-function:

{
     
     project1A: { ... details for project 1A only ...common_metadata: { organization: { id: 99, preferences: { current_reporting_period: 2 } } }, }
     project1B: { ... details for project 1B only ...common_metadata: { organization: { id: 99, preferences: { current_reporting_period: 2 } } }, }
     project1C: { ... details for project 1C only ...common_metadata: { organization: { id: 99, preferences: { current_reporting_period: 2 } } }, }
     subrecipient: { ... details for subrecipient only ... common_metadata: { organization: { id: 99, preferences: { current_reporting_period: 2 } } },}
     zip: { ... details for zip creation only .... common_metadata: { organization: { id: 99, preferences: { current_reporting_period: 2 } } },}
}

Currently I filter this in treasury_generation_step_function.tf here as follows:
"Payload.$" : "$$.Execution.Input.['project1A']",

"Payload.$" : "{ "common": "$$.Execution.Input.['common_metadata']", "specific": $$.Execution.Input.['project1A']",

According to https://jsonpath.com/ there is some union operator that can be used but I couldn't quite grok it. Lmk if anything pops out as an obvious solution here. In theory something like this would work:
"Payload.$": "$$.Execution.Input.['project1A'], $$.Execution.Input.['common_metadata']"

@as1729 as1729 disabled auto-merge August 28, 2024 13:10
@as1729 as1729 merged commit b275c51 into main Aug 28, 2024
22 checks passed
@as1729 as1729 deleted the as/update-treasury-gen-inputs branch August 28, 2024 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation Improvements or additions to documentation infra javascript python terraform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants