Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ECS_EBSTA_SUPPORTED env variable to add EBSTA capabilities #4091

Merged
merged 5 commits into from
Feb 22, 2024

Conversation

prateekchaudhry
Copy link
Contributor

@prateekchaudhry prateekchaudhry commented Feb 9, 2024

Summary

This is a fix for issue #4089

As part of ecs-init start up, init tries to create /mnt/ecs/ebs directory. However depending on how instances are configured, /mnt can be sometimes be read only. This leads to ecs-agent container failing to be created with following error in ecs-init

[ERROR] could not create EBS mount directory: mkdir /mnt/ecs: read-only file system

This PR changes this by adding an environment variable ECS_EBSTA_SUPPORTED to ecs-agent. If set to false, agent will not advertise EBSTA capabilities. This works as follows

  • ecs-init/engine/engine.go - If the ebs mount directory /mnt/ecs/ebs fails to be created, we log this error in pre-start and continue
  • ecs-init/docker/docker.go - Detect if ebs mount directory has been successfully created. If not set an environment variable ECS_EBSTA_SUPPORTED as false (Set default true)
  • agent/app/agent_capability.go - Check the value of ECS_EBSTA_SUPPORTED and do not add EBSTA capabilities if the env var is set false. This is set as needed by the ecs-init changes. Agent defaults this to true for compatibility with older ecs-init versions

Testing

  • Mounting a Readonly file system at '/mnt' for an instance, able to launch agent and tasks
  • Run Functional tests on AL2 and AL2023 AMIs with ecs-init artifacts which also test default ebs functionality

New tests cover the changes:
No new tests added

Description for the changelog

Bugfix: Use ECS_EBSTA_SUPPORTED env variable to add EBSTA capabilities

Does this PR include breaking model changes? If so, Have you added transformation functions?
No

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

yinyic
yinyic previously approved these changes Feb 9, 2024
chienhanlin
chienhanlin previously approved these changes Feb 9, 2024
if err != nil {
return engineError("could not create EBS mount directory", err)
// Skip for External, EBS Task Attach is not supported for External launch type
if !config.RunningInExternal() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bug affects EC2 launch type too doesn't it? I am wondering if we should just log the failure and continue. Can we disable the EBS capability if we fail to create this directory?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, when running in external we're going to advertise the capability anyways here? even though we failed to make the mnt directory?

@prateekchaudhry prateekchaudhry force-pushed the ebs-external-dir-fix branch 2 times, most recently from 90513fe to 43f374f Compare February 13, 2024 23:42
@prateekchaudhry prateekchaudhry changed the title Do not create /mnt/ebs dir for external launch type Use ECS_EBSTA_SUPPORTED env variable to add EBSTA capabilities Feb 19, 2024
@prateekchaudhry prateekchaudhry merged commit f546d83 into aws:dev Feb 22, 2024
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants