Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix retry behavior in aws_ssoadmin_managed_policy_attachment for eventual consistency issue #19216

Conversation

jvitor03
Copy link
Contributor

@jvitor03 jvitor03 commented May 3, 2021

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #19215

This PR adds a retry behavior for Create and destroy in the aws_ssoadmin_managed_policy_attachment-consitency resource.

Output from acceptance testing:

No acceptance testes where created, because they are not reliable to cover eventual consistency issues: https://github.com/hashicorp/terraform-provider-aws/blob/main/docs/contributing/retries-and-waiters.md#eventual-consistency

Full trace log from the fix being used during a destroy: https://gist.github.com/jvitor03/89cb2d10213cebefcd76e95046118968

Log retry summary:

-----------------------------------------------------: timestamp=2021-05-03T01:28:54.732-0300
2021-05-03T01:28:54.732-0300 [INFO]  plugin.terraform-provider-aws: 2021/05/03 01:28:54 [DEBUG] [aws-sdk-go] {"__type":"ConflictException","Message":"There is a conflicting operation in process."}: timestamp=2021-05-03T01:28:54.732-0300
2021-05-03T01:28:54.732-0300 [INFO]  plugin.terraform-provider-aws: 2021/05/03 01:28:54 [DEBUG] [aws-sdk-go] DEBUG: Validate Response SSO Admin/DetachManagedPolicyFromPermissionSet failed, attempt 0/25, error ConflictException: There is a conflicting operation in process.: timestamp=2021-05-03T01:28:54.732-0300
2021-05-03T01:28:54.732-0300 [INFO]  plugin.terraform-provider-aws: 2021/05/03 01:28:54 [TRACE] Waiting 500ms before next try: timestamp=2021-05-03T01:28:54.732-0300
2021-05-03T01:28:54.896-0300 [INFO]  plugin.terraform-provider-aws: 2021/05/03 01:28:54 [DEBUG] [aws-sdk-go] DEBUG: Response SSO Admin/DeleteAccountAssignment Details:
---[ RESPONSE ]--------------------------------------

[...]

{"InstanceArn":"arn:aws:sso:::instance/ssoins-7223f43a8b17208a","ManagedPolicyArn":"arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess","PermissionSetArn":"arn:aws:sso:::permissionSet/ssoins-7223f43a8b17208a/ps-eb12a5e3a0d6a825"}
-----------------------------------------------------: timestamp=2021-05-03T01:28:55.935-0300
2021-05-03T01:28:56.638-0300 [INFO]  plugin.terraform-provider-aws: 2021/05/03 01:28:56 [DEBUG] [aws-sdk-go] DEBUG: Retrying Request SSO Admin/DetachManagedPolicyFromPermissionSet, attempt 1: timestamp=2021-05-03T01:28:56.638-0300
2021-05-03T01:28:56.638-0300 [INFO]  plugin.terraform-provider-aws: 2021/05/03 01:28:56 [DEBUG] [aws-sdk-go] DEBUG: Request SSO Admin/DetachManagedPolicyFromPermissionSet Details:
---[ REQUEST POST-SIGN ]-----------------------------

@jvitor03 jvitor03 requested a review from a team as a code owner May 3, 2021 19:45
@ghost ghost added size/S Managed by automation to categorize the size of a PR. service/ssoadmin Issues and PRs that pertain to the ssoadmin service. labels May 3, 2021
@github-actions github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label May 3, 2021
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome @jvitor03 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTING guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

@jvitor03 jvitor03 changed the title [WIP] Bugfix retry behavior in aws_ssoadmin_managed_policy_attachment for eventual consistency issue Bugfix retry behavior in aws_ssoadmin_managed_policy_attachment for eventual consistency issue May 3, 2021
@aabadia
Copy link

aabadia commented May 4, 2021

👍

@anGie44 anGie44 added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels May 4, 2021
@anGie44 anGie44 self-assigned this May 4, 2021
@anGie44
Copy link
Contributor

anGie44 commented May 4, 2021

Hi @jvitor03 , thank you for this PR and following the contribution guidelines for retry handling! With these eventual consistency errors happening on both create and delete, without a precise timeout as it can depend per account/user, we can instead add a connection retry handler in config.go to automatically retry when hitting the error described e.g.

client.ssoadminconn.Handlers.Retry.PushBack(func(r *request.Request) {
    if r.Operation.Name == "AttachManagedPolicyToPermissionSet" || r.Operation.Name == "DetachManagedPolicyFromPermissionSet" {
        if tfawserr.ErrCodeEquals(r.Error, ssoadmin.ErrCodeConflictException) {
            r.Retryable = aws.Bool(true)
	}
    }
}

@jvitor03
Copy link
Contributor Author

jvitor03 commented May 4, 2021

That looks way better to avoid duplicated code, I will do the change.
Thanks for the review!

@ghost ghost added size/XS Managed by automation to categorize the size of a PR. provider Pertains to the provider itself, rather than any interaction with AWS. and removed size/S Managed by automation to categorize the size of a PR. labels May 5, 2021
Copy link
Contributor

@anGie44 anGie44 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great @jvitor03 🚀

Output of acceptance tests:

--- PASS: TestAccAWSSSOAdminManagedPolicyAttachment_disappears (32.90s)
--- PASS: TestAccAWSSSOAdminManagedPolicyAttachment_disappears_permissionSet (24.52s)
--- PASS: TestAccAWSSSOAdminManagedPolicyAttachment_basic (32.00s)
--- PASS: TestAccAWSSSOAdminManagedPolicyAttachment_forceNew (57.42s)
--- PASS: TestAccAWSSSOAdminManagedPolicyAttachment_multipleManagedPolicies (82.14s)

@anGie44 anGie44 added this to the v3.39.0 milestone May 5, 2021
@anGie44 anGie44 merged commit e8f8cdf into hashicorp:main May 5, 2021
@ghost
Copy link

ghost commented May 7, 2021

This has been released in version 3.39.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@github-actions
Copy link

github-actions bot commented Jun 6, 2021

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 6, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. provider Pertains to the provider itself, rather than any interaction with AWS. service/ssoadmin Issues and PRs that pertain to the ssoadmin service. size/XS Managed by automation to categorize the size of a PR.
Projects
None yet
3 participants