Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Athena engine version to v3 #886

Merged
merged 1 commit into from
Nov 21, 2023
Merged

Conversation

dlpzx
Copy link
Contributor

@dlpzx dlpzx commented Nov 21, 2023

Feature or Bugfix

  • Bugfix

Detail

It is not a bug yet but Athena has already included he following warning at workgroup creation Athena engine version 2 will be deprecated in the near future. You can view this information in your AWS Personal Health Dashboard. Workgroups not upgraded by the deprecation date will be upgraded to version 3 automatically.

To avoid future failures, we are updating the engine version selected for the creation to new workgroups in data.all to version 3.

Here is the documentation of version 3 which include the breaking changes from v2. Engine version 3 was released on October 13, 2022 and as it can be verified in the Athena release notes, it has been constantly improved during this year, surpassing in features athena v2.

Relates

Security

Please answer the questions below briefly where applicable, or write N/A. Based on
OWASP 10. N/A

  • Does this PR introduce or modify any input fields or queries - this includes
    fetching data from storage outside the application (e.g. a database, an S3 bucket)?
    • Is the input sanitized?
    • What precautions are you taking before deserializing the data you consume?
    • Is injection prevented by parametrizing queries?
    • Have you ensured no eval or similar functions are used?
  • Does this PR introduce any functionality or component that requires authorization?
    • How have you ensured it respects the existing AuthN/AuthZ mechanisms?
    • Are you logging failed auth attempts?
  • Are you using or adding any cryptographic features?
    • Do you use a standard proven implementations?
    • Are the used keys controlled by the customer? Where are they stored?
  • Are you introducing any new policies/roles/users?
    • Have you used the least-privilege principle? How?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@dlpzx dlpzx marked this pull request as ready for review November 21, 2023 12:35
@dlpzx
Copy link
Contributor Author

dlpzx commented Nov 21, 2023

  • Tested locally (but triggering the creation of the environment stack in AWS) --> It creates successfully
  • I am also testing what happens when selected_engine_version='Athena engine version 2', as some users have reported errors. The error appears in the creation of the environment stack and fails in the Athena workgroup creation with the logs Invalid request provided: The selected engine version is not valid but I have not been able to reproduce the issue.

@dlpzx
Copy link
Contributor Author

dlpzx commented Nov 21, 2023

It seems that Athena has deprecated engine version 2 in some regions but not all. When running the command aws athena list-engine-versions, in some regions "Athena engine version 2" is still available, in others it is not. When athena engine version 2 is not available then we see the error in the environment stack

Regions that CANNOT use Athena engine v2:

  • eu-central-1

I tested this PR in eu-central-1 and it solves the issue.

Also tested upgrade from engine 2 to 3 and the environment stack updates successfully

@dlpzx dlpzx requested a review from noah-paige November 21, 2023 13:02
@dlpzx dlpzx added type: bug Something isn't working status: in-review This issue has been implemented and is currently in review and waiting for next release priority: high labels Nov 21, 2023
Copy link
Contributor

@noah-paige noah-paige left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@dlpzx dlpzx merged commit 99512dd into main Nov 21, 2023
8 checks passed
noah-paige added a commit that referenced this pull request Nov 28, 2023
### Feature or Bugfix
<!-- please choose -->
- Bugfix

### Detail
- When trying to update an Environment to use the latest Athena Version
(from this [previous
PR](#886)) and having CDK use
the data.all provided CDK Execution Policy ran into an `Update_Failed`
error on the Environment CloudFormation Stack due to missing permissions
for `athena:UpdateWorkGroup`
  - Added `athena:UpdateWorkGroup` to the CDK Exec Policy YAML file

### Relates
- N/A

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
  - Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@dlpzx dlpzx deleted the fix/upgrade-athena-engine-version branch December 18, 2023 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: high status: in-review This issue has been implemented and is currently in review and waiting for next release type: bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants