provider/aws: DependencyViolation on aws_security_group with circular route dependencies

[JeanMertz]

I'm getting errors like this:

* aws_security_group.master: DependencyViolation: resource sg-2173f548 has a dependent object
    status code: 400, request id: []

When trying to delete through the AWS web interface, I get this notification when trying to remove:

This security group is referenced by another security group, and cannot be deleted until you change the other security group's rules.

This happens because of rules like this:

ingress {
  from_port = 5050
  to_port = 5050
  protocol = "tcp"
  security_groups = ["${aws_security_group.master_lb.id}"]
}

The problem is that there are circular dependencies which can't be resolved by removing one security group before the other, because they all have rules depending on other groups.

So instead, Terraform should first remove all rules, before removing the group itself.

[apparentlymart]

The handling of security groups referencing other security groups is quirky in a number of ways, which is why the separate aws_security_group_rule resource was introduced. It avoids the circular dependency issues by treating the rules and the groups as separate resources.

Would it work for you to switch to using separate rule resources to represent your configuration?

Which is not to say that what you saw isn't a bug, but I'm beginning to wonder if inline rules inside security groups ought to just be deprecated altogether since they have always acted kinda funky.

[JeanMertz]

@apparentlymart thanks for that. I indeed switched to using aws_security_group_rule to solve this problem.

However, unfortunately that resource has another problem where it doesn't recognise existing rules that don't have a cidr block, but instead reference an other source rule.

It seems to be related to #2584 and causes a second TF run to want to create "missing" rules, which fails due to duplication.

[catsby]

Hey Friends – security groups and security group rules have been through a lot since this issue was open, and I'm confident this issue is resolved. If you're still hitting it, please let us know!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.