Automatically Enable AWS Go SDK Session Option SharedConfigState (AWS_SDK_LOAD_CONFIG environment variable)

[bflad]

Details

Currently, when working with Assumed IAM Role credentials following the Assuming an IAM Role in the AWS CLI documentation, the Terraform AWS Provider session logic requires the AWS_SDK_LOAD_CONFIG environment variable to set to a truthy value (e.g. AWS_SDK_LOAD_CONFIG=1). This is quite confusing user experience considering that the AWS CLI (boto) does not have a similar requirement.

For example, given this setup:

$ cat ~/.aws/config

[profile my-assume-role]
role_arn = arn:aws:iam::123456789012:role/my-assume-role
source_profile = my-assume-role-credentials

$ cat ~/.aws/credentials

[my-assume-role-credentials]
aws_access_key_id = ...
aws_secret_access_key = ...

$ cat main.tf

provider "aws" {
  region  = "us-east-1"
  version = "2.17.0"
}

data "aws_caller_identity" "current" {}

output "caller_arn" {
  value = "${data.aws_caller_identity.current.arn}"
}

$ export AWS_PROFILE=my-assume-role
$ terraform apply

Error: Error refreshing state: 1 error occurred:
  * provider.aws: No valid credential sources found for AWS Provider.
  Please see https://terraform.io/docs/providers/aws/index.html for more information on
  providing credentials for the AWS Provider

However, it works when the less documented environment variable is set:

$ export AWS_PROFILE=my-assume-role
$ export AWS_SDK_LOAD_CONFIG=1
$ terraform apply
data.aws_caller_identity.current: Refreshing state...

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

caller_arn = arn:aws:sts::123456789012:assumed-role/my-assume-role/1561748592973748000

We should be able to enable loading the shared configuration file automatically via the Options configuration SharedConfigState field.

This change should occur during a major version upgrade since it has the potential of unexpectedly changing authentication precedence and handling for some environments.

References

Among others:

• AWS_PROVIDER not working, explicit 'provider=xx' works fine #8779
• Document AWS_SDK_LOAD_CONFIG on aws provider #8451
• RFC Supporting credential_process for assuming roles using external credential providers #6913 (comment)
• Using terraform with assumed roles defined in shared configuration are ineffective #1184
• Terraform does not read AWS profile from environment variable #233

[jukie]

This would be super useful for my org. It looks like this it would just be change of the implementation here?
https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/config.go#L329-#L332

[bflad]

We will likely want to:

• In hashicorp/aws-sdk-go-base, appropriately enable the configuration via a new boolean flag that defaults to true and gets exposed for the Terraform AWS Provider and S3 Backend
• In Terraform AWS Provider and S3 Backend, update the dependency and set that flag to false by default (the existing behavior). Add a boolean configuration option to allow operators to set that flag to opt into setting true.
• In Terraform AWS Provider and S3 Backend during their next major version upgrade, change the default configuration value to be true instead of false and include note about change in the upgrade guides

This will allow folks to opt into the change now (if they prefer) and leave any potential for breaking changes caused by the behavior change to occur in a major version release. 👍

[bflad]

Support for this has been merged and will release with version 3.0.0 of the Terraform AWS Provider, likely in about two weeks. Please note this enhancement was also applied for the Terraform S3 Backend (part of Terraform CLI codebase) as part of 0.13.0-beta2.

[ghost]

This has been released in version 3.0.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!