Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS-SDK: Fixing broken exports #2410

Merged
merged 22 commits into from
Aug 19, 2024
Merged

Conversation

engelhartrueben
Copy link
Collaborator

@engelhartrueben engelhartrueben commented Aug 1, 2024

Fixes # (issue)

Fixes broken exports.

Description

During the upgrade to Node 20, aws-sdk had to be upgraded to V3. This was simple in nature, considering a code-mod existed. However, our specific way of initiating an S3 bucket was not covered by this code-mod, and subsequently failed. Theoretically, our implementation (still in V2) was still supported, but clearly not.

This new implementation uses aws-sdk V3 api calls to create the S3 client and bucket. Additionally, we can set the region for the bucket, which is called from the environment variables.

This PR:

  • Checks if an AWS bucket exists, if not creates one of the specificed name found in AWS_S3_BUCKET_NAME=
  • Brings v3 AWS-SDK update to export logic
  • Only shows UI related to the type of export:
    Local Export:
    image
    AWS Export:
    image
    (NOTE) this UI is fixed with Dynamic Replies Feature - by MoveOn #2423
  • Places a hard check on if an email (SMTP or MailGun) is set up for exports
  • Adjusts the expiration for cached campaignExportUrl to match AWS expiration (1 day)

Tested on:

Mode AWS Bucket and Download Local download
production ✔️ ❌*
development ✔️ ✔️

*Local download is not supported in production.

Checklist:

  • I have manually tested my changes on desktop and mobile
  • The test suite passes locally with my changes
  • If my change is a UI change, I have attached a screenshot to the description section of this pull request
  • My change is 300 lines of code or less, or has a documented reason in the description why it’s longer
  • I have made any necessary changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • My PR is labeled [WIP] if it is in progress

@engelhartrueben engelhartrueben added the S-work in progress (WIP) Status: PR label for work that is not yet ready to be reviewed label Aug 1, 2024
@engelhartrueben engelhartrueben requested a review from mau11 August 1, 2024 16:27
…d links regardless of email set-up. Then, if AWS is not set up, maybe create a direct email stream?
{(this.props.organizationData &&
this.props.organizationData.emailEnabled) ?
" we'll e-mail you when it's done. " :
" check your Spoke directory. "}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried this locally, if we have AWS S3 set up, even if email isn't enabled, the data will save to S3, not in the spoke directory

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disregard my previous comment, the last change to this check fixed the issue I was seeing

});
const bucketName = process.env.AWS_S3_BUCKET_NAME;
const command = new CreateBucketCommand({ Bucket : bucketName });

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we want to create a new bucket each time we export data, it's likely someone will already have bucket created for the data and want to upload directly to it

we can add a check, if the bucket exists, save the exported data, if it doesn't, create a new one and save


console.log(`S3 bucket "${bucketName}" already exists.`);
} catch (error) {
if (error.name === "NotFound") {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (error.code === "NoSuchBucket")

Source

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the bucket is not found, it doesn't look like we get this error code, only the error name

@engelhartrueben engelhartrueben requested a review from mau11 August 15, 2024 17:38
@engelhartrueben engelhartrueben marked this pull request as ready for review August 15, 2024 17:39
@engelhartrueben
Copy link
Collaborator Author

@mau11 this is set up for main, but can rebase to stage-main-14.1 upon request!

Copy link
Member

@mau11 mau11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@mau11 mau11 changed the base branch from main to stage-main-14.0.2 August 18, 2024 14:33
@engelhartrueben engelhartrueben changed the title (WIP) AWS-SDK: Fixing broken exports AWS-SDK: Fixing broken exports Aug 19, 2024
@engelhartrueben engelhartrueben merged commit 01d3bf2 into stage-main-14.0.2 Aug 19, 2024
17 checks passed
@engelhartrueben engelhartrueben deleted the re/fix-exports branch August 19, 2024 14:48
@mau11 mau11 mentioned this pull request Aug 19, 2024
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-work in progress (WIP) Status: PR label for work that is not yet ready to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants