Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove policies-updater ECS task #1046

Merged
merged 1 commit into from
Feb 8, 2024
Merged

Remove policies-updater ECS task #1046

merged 1 commit into from
Feb 8, 2024

Conversation

dlpzx
Copy link
Contributor

@dlpzx dlpzx commented Feb 8, 2024

Feature or Bugfix

  • Bugfix

Detail

In data.all there are 7 ECS Tasks:
Tasks currently being used:

  • cdkproxy -- on demand - it deploys CDK stacks in Environment accounts
  • share-manager -- on demand - it executes sharing actions
  • stacks-updater -- on schedule every 1day - Every day environment and dataset stacks are updated

Tasks that need to be reviewed:

  • subscriptions -- on schedule every 15mins - it tries to poll message from subscriptions queue. The queue is empty and we are not posting any messages. We could consider subscriptions to be legacy at the moment.
  • catalog-indexer -- on schedule every 6hours - it reads all active items from RDS and indexes them in the Catalog. It does not look for deleted items.
  • tables-syncer -- on schedule every 15mins - it reads all active datasets. With boto3 it reads the Glue tables in that database and syncs the Glue tables with the registered tables in data.all. It upserts in OpenSearch and grant LF permissions.

Tasks that currently are not used and need to be removed:

  • policies-updater -- on schedule every 15mins - it reapplies shares on imported buckets. It is legacy from folder sharing based on bucket policies. It uses the generic ecs-tasks-role

In this PR the task policies-updater is removed

Tested in AWS

  • CICD pipeline succeeds
  • ECS CFN stack is updated and deleted policies-updater task. It also deletes log-group. All other tasks remain untouched

Relates

Security

Please answer the questions below briefly where applicable, or write N/A. Based on
OWASP 10.

  • Does this PR introduce or modify any input fields or queries - this includes
    fetching data from storage outside the application (e.g. a database, an S3 bucket)?
    • Is the input sanitized?
    • What precautions are you taking before deserializing the data you consume?
    • Is injection prevented by parametrizing queries?
    • Have you ensured no eval or similar functions are used?
  • Does this PR introduce any functionality or component that requires authorization?
    • How have you ensured it respects the existing AuthN/AuthZ mechanisms?
    • Are you logging failed auth attempts?
  • Are you using or adding any cryptographic features?
    • Do you use a standard proven implementations?
    • Are the used keys controlled by the customer? Where are they stored?
  • Are you introducing any new policies/roles/users?
    • Have you used the least-privilege principle? How?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Copy link
Contributor

@noah-paige noah-paige left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved

@dlpzx dlpzx merged commit 752e824 into main Feb 8, 2024
9 checks passed
@dlpzx dlpzx deleted the fix/remove_unused_tasks branch February 12, 2024 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants