Add permissions policy to the EKS nodegroup IAM role for Nomad Autosc…

…aler

README.md

@@ -58,10 +58,10 @@ We love contributions! Here is how to get started:
 - Once your work is complete, you may create a PR for your branch
 - Each commit is tested for formatting and syntax errors
 - We test PRs in our own test environments
-- PRs are reviewed and approved by members of CircleCI's Server team
+- PRs are reviewed and approved by members of CircleCI's [On-Prem team](https://github.com/orgs/CircleCI-Public/teams/on-prem)
 
 ### Reporting Issues
 
 - Feature requests or problems found may be reported by creating an issue
   within this repository
-- CircleCI's Server team will review and respond to issues.
+- CircleCI's [On-Prem team](https://github.com/orgs/CircleCI-Public/teams/on-prem) will review and respond to issues.

nomad-aws/README.md

@@ -33,7 +33,7 @@ module "nomad_clients" {
   subnet = "<< ID of subnet you want to run nomad clients in >>"
   vpc_id = "<< ID of VPC you want to run nomad client in >>"
 
-  server_endpoint = "<< hostname of server installation >>"
+  nomad_server_hostname = "<< hostname of server installation >>"
 
   dns_server = "<< ip address of your VPC DNS server >>"
   blocked_cidrs = [
@@ -46,6 +46,13 @@ module "nomad_clients" {
   }
   nomad_auto_scaler = false # If true, terraform will generate an IAM user to be used by nomad-autoscaler in CircleCI Server.
 
+  # If `nomad_auto_scaler` is enabled, `nodegroup_iam_role` should be populated with
+  # the EKS nodegroup IAM role name associated with the EKS cluster the Nomad Autoscaler is deployed on,
+  # and `create_nomad_nodegroup_iam_role_policy` should be set to `true`.
+  # This ensures an IAM policy is created with the minimum permissions required by the Nomad Autoscaler.
+  create_nomad_nodegroup_iam_role_policy = false
+  nodegroup_iam_role = ""
+
   # enable_irsa input will allow K8s service account to use IAM roles, you have to replace REGION, ACCOUNT_ID, OIDC_ID and K8S_NAMESPACE with appropriate value
   # for more info, visit - https://docs.aws.amazon.com/eks/latest/userguide/associate-service-account-role.html
   enable_irsa = {}
@@ -96,6 +103,7 @@ There are more examples in the [examples](./examples/) directory.
 | [aws_iam_access_key.nomad_asg_user](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) | resource |
 | [aws_iam_instance_profile.nomad_client_profile](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) | resource |
 | [aws_iam_role.nomad_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
+| [aws_iam_role_policy.nomad_nodegroup_iam_role_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource |
 | [aws_iam_user.nomad_asg_user](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) | resource |
 | [aws_iam_user_policy.nomad_asg_user](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy) | resource |
 | [aws_key_pair.ssh_key](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/key_pair) | resource |
@@ -112,6 +120,7 @@ There are more examples in the [examples](./examples/) directory.
 |------|-------------|------|---------|:--------:|
 | <a name="input_basename"></a> [basename](#input\_basename) | Name used as prefix for AWS resources | `string` | `""` | no |
 | <a name="input_blocked_cidrs"></a> [blocked\_cidrs](#input\_blocked\_cidrs) | List of CIDR blocks to block access to from within jobs, e.g. your K8s nodes.<br>You won't want to block access to external VMs here.<br>It's okay when your dns\_server is within a blocked CIDR block, you can use var.dns\_server to create an exemption. | `list(string)` | n/a | yes |
+| <a name="input_create_nomad_nodegroup_iam_role_policy"></a> [create\_nomad\_nodegroup\_iam\_role\_policy](#input\_create\_nomad\_nodegroup\_iam\_role\_policy) | Indicates whether the `nomad_nodegroup_iam_role_policy` should be created | `bool` | `false` | no |
 | <a name="input_disk_size_gb"></a> [disk\_size\_gb](#input\_disk\_size\_gb) | The volume size, in GB to each nomad client's /dev/sda1 disk. | `number` | `100` | no |
 | <a name="input_dns_server"></a> [dns\_server](#input\_dns\_server) | If the IP address of your VPC DNS server is within one of the blocked CIDR blocks you can create an exemption by entering the IP address for it here | `string` | n/a | yes |
 | <a name="input_docker_network_cidr"></a> [docker\_network\_cidr](#input\_docker\_network\_cidr) | IP CIDR to be used in docker networks when running job on nomad client.<br>This CIDR block should not be the same as your VPC CIDR block.<br>i.e - "10.10.0.0/16" or "172.32.0.0/16" or "192.168.0.0/16" | `string` | `"10.10.0.0/16"` | no |
@@ -123,6 +132,7 @@ There are more examples in the [examples](./examples/) directory.
 | <a name="input_machine_image_names"></a> [machine\_image\_names](#input\_machine\_image\_names) | Strings to filter image names for nomad virtual machine images. | `list(string)` | <pre>[<br>  "ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"<br>]</pre> | no |
 | <a name="input_machine_image_owners"></a> [machine\_image\_owners](#input\_machine\_image\_owners) | List of AWS account IDs that own the images to be used for nomad virtual machines. | `list(string)` | <pre>[<br>  "099720109477",<br>  "513442679011"<br>]</pre> | no |
 | <a name="input_max_nodes"></a> [max\_nodes](#input\_max\_nodes) | Maximum number of nomad clients to create. Must be greater than or equal to nodes | `number` | `5` | no |
+| <a name="input_nodegroup_iam_role"></a> [nodegroup\_iam\_role](#input\_nodegroup\_iam\_role) | Specifies the IAM role name for the EKS nodegroup that is associated with the EKS cluster where the Nomad Autoscaler is deployed.<br>    This should be populated if `nomad_auto_scaler` is enabled.<br>    It ensures that an IAM policy with the minimum permissions required by the Nomad Autoscaler is created.<br>    Note that this is dependent on `create_nomad_nodegroup_iam_role_policy` being set to `true`. | `string` | `""` | no |
 | <a name="input_nodes"></a> [nodes](#input\_nodes) | Number of nomad clients to create | `number` | n/a | yes |
 | <a name="input_nomad_auto_scaler"></a> [nomad\_auto\_scaler](#input\_nomad\_auto\_scaler) | If set to true, A Nomad User or A Role will be created based on enable\_irsa variable values | `bool` | `false` | no |
 | <a name="input_nomad_server_hostname"></a> [nomad\_server\_hostname](#input\_nomad\_server\_hostname) | Hostname of RPC service of Nomad control plane (e.g circleci.example.com) | `string` | n/a | yes |

nomad-aws/nomad-autoscaler.tf

@@ -38,4 +38,12 @@ resource "aws_iam_role" "nomad_role" {
     })
   }
   tags = local.tags
-}
+}
+
+resource "aws_iam_role_policy" "nomad_nodegroup_iam_role_policy" {
+  count = var.create_nomad_nodegroup_iam_role_policy ? 1 : 0
+
+  name   = "${var.basename}-nomad-nodegroup-iam-role-policy"
+  role   = var.nodegroup_iam_role
+  policy = templatefile("${path.module}/template/nomad_nodegroup_iam_role_policy.tpl", {})
+}

nomad-aws/template/nomad_nodegroup_iam_role_policy.tpl

@@ -0,0 +1,18 @@
+{
+	"Version": "2012-10-17",
+	"Statement": [
+		{
+			"Effect": "Allow",
+			"Action": [
+				"autoscaling:CreateOrUpdateTags",
+				"autoscaling:DescribeAutoScalingGroups",
+				"autoscaling:DescribeScalingActivities",
+				"autoscaling:TerminateInstanceInAutoScalingGroup",
+				"autoscaling:UpdateAutoScalingGroup"
+			],
+			"Resource": [
+				"*"
+			]
+		}
+	]
+}

nomad-aws/variables.tf

@@ -123,7 +123,7 @@ variable "role_name" {
   default     = null
 }
 
-# Check for IRSA Role (more details)  - https://docs.aws.amazon.com/eks/latest/userguide/create-service-account-iam-policy-and-role.html
+# Check for IRSA Role (more details)  - https://docs.aws.amazon.com/eks/latest/userguide/associate-service-account-role.html
 #   enable_irsa  = {
 #                  oidc_principal_id  = "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/oidc.eks.<REGION>.amazonaws.com/id/<OIDC_ID>"
 #                  oidc_eks_variable  = "oidc.eks.<REGION>.amazonaws.com/id/<OIDC_ID>:sub"
@@ -141,6 +141,24 @@ variable "enable_irsa" {
   description = "If passed a valid OIDC MAP, terraform will create K8s Service Account Role to be used by nomad autoscaler."
 }
 
+variable "create_nomad_nodegroup_iam_role_policy" {
+  description = "Indicates whether the `nomad_nodegroup_iam_role_policy` should be created"
+  type        = bool
+  default     = false
+}
+
+variable "nodegroup_iam_role" {
+  type    = string
+  default = ""
+
+  description = <<EOF
+    Specifies the IAM role name for the EKS nodegroup that is associated with the EKS cluster where the Nomad Autoscaler is deployed.
+    This should be populated if `nomad_auto_scaler` is enabled.
+    It ensures that an IAM policy with the minimum permissions required by the Nomad Autoscaler is created.
+    Note that this is dependent on `create_nomad_nodegroup_iam_role_policy` being set to `true`.
+  EOF
+}
+
 variable "disk_size_gb" {
   type        = number
   default     = 100