credential_process swallows stderr

[Nevon]

Describe the bug

This is a duplicate of the closed issue #3057. I'm opening a new issue to track that this is still broken in aws-cli, even if the root cause is within botocore.

When the aws cli launches a credential_process that writes to stderr, that stderr data is never displayed. This makes it impossible to use a credential process that prompts for MFA, since the prompt for that will be sent over stderr.

Expected Behavior

The credential process stderr should be connected to the aws cli stderr so that anything written to stderr from the credential process is displayed by the aws cli stderr.

Current Behavior

Output to stderr is not displayed. In my case, where the credential process displays an MFA prompt, that prompt is never shown to the user and it just looks like the process has hung.

Reproduction Steps

Example credential process:

#!/bin/bash
echo "Please enter fake MFA code:" >&2

read code

if (( ("$code" % 2) == "0" )); then
  echo "{
  \"Version\": 1,
  \"AccessKeyId\": \"an AWS access key\",
  \"SecretAccessKey\": \"your AWS secret access key\",
  \"SessionToken\": \"the AWS session token for temporary credentials\",
  \"Expiration\": \"ISO8601 timestamp when the credentials expire\"
}"
  exit 0
else
  echo "Invalid fake MFA code" >&2
  exit 1
fi

~/.aws/config:

[profile example]
credential_process = ~/fake-credential-process.sh

When used:

❯ aws --profile example sts get-caller-identity

# process now just sits there waiting for input with no visible prompt

Possible Solution

This is handled correctly in the other AWS SDKs, as far as I can tell, but not in Botocore. This is an open issue boto/botocore#1348.

The following two PRs both attempt to solve this:

• Remove redirection of stderr when calling credential_process boto/botocore#2091
• Allow stderr passthrough for credential_process boto/botocore#1835

Additional Information/Context

No response

CLI version used

aws-cli/1.25.82 Python/3.10.6 Darwin/21.6.0 botocore/1.27.81

Environment details (OS name and version, etc.)

macOS 12.6

[aBurmeseDev]

Hi @Nevon, thanks for reaching out.

Have you tried the workaround mentioned here?

As far as exposing stderr for prompts, it would need to be coordinated across SDKs which means the broader teams would need to review and decide on this. Given that this is a feature request that would affect multiple AWS SDKs, I created an issue in aws-sdk repo (aws/aws-sdk#358) for visibility.

Please feel free to check in and show your interest by adding a 👍 in the aws-sdk issue.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[lorengordon]

Fwiw, the golang sdk already does this, and it is glorious. Folks just shouldn't be using botocore/boto3, I guess? ;)