Consume TMDS init function from ecs-agent module

[amogh09]

Summary

TMDS initialization was moved to ecs-agent module in #3660. Change TMDS initialization in agent module to consume the TMDS initialization function from ecs-agent module.

Implementation details

• Import tmds.NewServer function from ecs-agent module in agent/task_server_setup.go file and use it to initialize TMDS.
• Replace usage of AgentCredentialsPort constant with tmds.Port constant imported from ecs-agent module.
• Replace usage of LoggingHandler in introspection server with a LoggingHandler imported from ecs-agent module. LoggingHandler has also been moved to ecs-agent module. Remove the LoggingHandler in agent module.

Testing

Unit and functional tests are already in place to test TMDS. In addition to those, I performed some manual tests that are listed below.

• Built Agent from source and called the following endpoints and made sure that they are available.
  • v2 - credentials, container metadata, task metadata, taskWithTags, container stats, and task stats
  • v3 - container metadata, task metadata, taskWithTags, container stats, task stats, and associations
  • v4 - container metadata, task metadata, taskWithTags, container stats, task stats, and associations
  • task protection - GetTaskProtection and UpdateTaskProtection
• Performed stress tests on multiple endpoints and made sure that they are able to handle the same requests load as before.
• Performed stress test with request rate matching the configured rate limit (40 rps sustained, 100 rps burst) and checked that rate limiter is not triggered for both sustained and burst cases.
• Exceeded request rate limits and ensured that TMDS returns 429 Too Many Requests error when the limits are exceeded.
• Checked that audit logs are output to the audit log file as before for requests to v2 credentials endpoint and when rate limits are exceeded.
• Checked that logging middleware continues to work as before by setting the log level to DEBUG and observing Agent logs while sending requests to TMDS.
• Checked that logging middleware for Introspection server continues to work as before by setting the log level to DEBUG and observing Agent logs while sending requests to the Introspection Server.
• Collected profiling data for CPU and memory utilization and checked that the utilization is not obviously different than before.
• Called taskWithTags endpoint at a high request rate (for this endpoint) of 40rps to trigger throttling and checked that write timeout kicked in for the throttled requests and cut the connection after a timeout of 5 seconds as expected.

New tests cover the changes: no

Description for the changelog

Consume TMDS init function from ecs-agent module

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[mye956]

General Question!: Why was github.com/didip/tollbooth v4.0.2+incompatible moved here?

[amogh09]

This was done by go mod tidy. The reason is that agent module no longer has a direct dependency on tollbooth which is a library that provides request rate limiting middleware. ecs-agent module is responsible for adding the rate limiting middleware to TMDS, so agent module now has an indirect dependency on tollbooth while ecs-agent has a direct dependency as shown below.

amazon-ecs-agent/ecs-agent/go.mod

Line 8 in c83389c

github.com/didip/tollbooth v4.0.2+incompatible

[mye956]

Could you also add a link to the previous PR in the description?

[amogh09]

Yes, added a link to the previous PR to the description.