Permit configuring IMDS client behavior on timeout

[Mark-Simulacrum]

Describe the feature

The IMDS client in aws-config, both for internal usage (e.g., credential fetching) and as a public-facing client (e.g., to resolve instance metadata in user programs) should support being configured to an "expected to exist" mode where e.g. TCP connects are retried (unlike the current default).

Use Case

On EC2 instances, IMDS is not always 100% available; we periodically see short blips of unavailability in production, like with any other service. Retries avoid these bubbling out as e.g. service launch failures due to inability to provision credentials or discover local identity (instance ID).

Proposed Solution

At minimum, the client should expose a knob to enable retrying TCP failures (connect and read timeouts). Ideally, the solution would allow for us to specify that we do in fact expect a response and so the normal SDK behaviors should happen -- rather than having to chase those over time with more knobs.

The SDK defaults may make less sense for cases where IMDS may not be available, but explicit usage of the IMDS client seems like a good indicator of "I expect it to work" (at least as a default).

We would want to configure this for the implicit IMDS client created within credentials provider chains, but are OK explicitly threading a client into that state if needed.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

A note for the community

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue, please leave a comment

[landonxjames]

This behavior has now been added in version 1.5.15 of the aws-config crate. See the new retry_classifier method on the IMDS builder. Additionally, so that you don't have to implement the whole retry classifier yourself, we added a with_retry_connect_timeouts method to the ImdsResponseRetryClassifier that you can configure and pass in.

[ysaito1001]

The fix was released on 2025-01-23

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.