adding dynamics to allow for other expiration types (#21)

* adding dynamic to allow for other expiration types for non versioned buckets * adding support for abort_incomplete_multipart_upload_days * Fixing tags in lifecycle when multipart upload is enabled Co-authored-by: PePe (Jose) Amengual <[email protected]>
cloudposse · Apr 6, 2020 · fe66c46 · fe66c46
1 parent 444f956
commit fe66c46
Show file tree

Hide file tree

Showing 4 changed files with 87 additions and 9 deletions.
diff --git a/README.md b/README.md
@@ -123,16 +123,21 @@ Available targets:
 
 | Name | Description | Type | Default | Required |
 |------|-------------|:----:|:-----:|:-----:|
+| abort_incomplete_multipart_upload_days | Maximum time (in days) that you want to allow multipart uploads to remain in progress | number | `5` | no |
 | acl | The canned ACL to apply. We recommend `private` to avoid exposing sensitive information | string | `private` | no |
 | allow_encrypted_uploads_only | Set to `true` to prevent uploads of unencrypted objects to S3 bucket | bool | `false` | no |
 | allowed_bucket_actions | List of actions the user is permitted to perform on the S3 bucket | list(string) | `<list>` | no |
 | attributes | Additional attributes (e.g. `1`) | list(string) | `<list>` | no |
 | delimiter | Delimiter to be used between `namespace`, `environment`, `stage`, `name` and `attributes` | string | `-` | no |
+| enable_glacier_transition | Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files | bool | `true` | no |
 | enabled | Set to false to prevent the module from creating any resources | bool | `true` | no |
 | environment | Environment, e.g. 'prod', 'staging', 'dev', 'pre-prod', 'UAT' | string | `` | no |
+| expiration_days | Number of days after which to expunge the objects | number | `90` | no |
 | force_destroy | A boolean string that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable | bool | `false` | no |
+| glacier_transition_days | Number of days after which to move the data to the glacier storage tier | number | `60` | no |
 | kms_master_key_arn | The AWS KMS master key ARN used for the `SSE-KMS` encryption. This can only be used when you set the value of `sse_algorithm` as `aws:kms`. The default aws/s3 AWS KMS master key is used if this element is absent while the `sse_algorithm` is `aws:kms` | string | `` | no |
 | lifecycle_rule_enabled | Enable or disable lifecycle rule | bool | `false` | no |
+| lifecycle_tags | Tags filter. Used to manage object lifecycle events | map(string) | `<map>` | no |
 | name | Solution name, e.g. 'app' or 'jenkins' | string | `` | no |
 | namespace | Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' | string | `` | no |
 | noncurrent_version_expiration_days | Specifies when noncurrent object versions expire | number | `90` | no |
@@ -142,6 +147,7 @@ Available targets:
 | region | If specified, the AWS region this bucket should reside in. Otherwise, the region used by the callee | string | `` | no |
 | sse_algorithm | The server-side encryption algorithm to use. Valid values are `AES256` and `aws:kms` | string | `AES256` | no |
 | stage | Stage, e.g. 'prod', 'staging', 'dev', OR 'source', 'build', 'test', 'deploy', 'release' | string | `` | no |
+| standard_transition_days | Number of days to persist in the standard storage tier before moving to the infrequent access tier | number | `30` | no |
 | tags | Additional tags (e.g. `map('BusinessUnit','XYZ')` | map(string) | `<map>` | no |
 | user_enabled | Set to `true` to create an IAM user with permission to access the bucket | bool | `false` | no |
 | versioning_enabled | A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket | bool | `false` | no |
@@ -218,6 +224,10 @@ We deliver 10x the value for a fraction of the cost of a full-time engineer. Our
 
 Join our [Open Source Community][slack] on Slack. It's **FREE** for everyone! Our "SweetOps" community is where you get to talk with others who share a similar vision for how to rollout and manage infrastructure. This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build totally *sweet* infrastructure.
 
+## Discourse Forums
+
+Participate in our [Discourse Forums][discourse]. Here you'll find answers to commonly asked questions. Most questions will be related to the enormous number of projects we support on our GitHub. Come here to collaborate on answers, find solutions, and get ideas about the products and services we value. It only takes a minute to get started! Just sign in with SSO using your GitHub account.
+
 ## Newsletter
 
 Sign up for [our newsletter][newsletter] that covers everything on our technology radar.  Receive updates on what we're up to on GitHub as well as awesome new projects we discover. 
@@ -333,6 +343,7 @@ Check out [our other projects][github], [follow us on twitter][twitter], [apply
   [testimonial]: https://cpco.io/leave-testimonial?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/terraform-aws-s3-bucket&utm_content=testimonial
   [office_hours]: https://cloudposse.com/office-hours?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/terraform-aws-s3-bucket&utm_content=office_hours
   [newsletter]: https://cpco.io/newsletter?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/terraform-aws-s3-bucket&utm_content=newsletter
+  [discourse]: https://ask.sweetops.com/?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/terraform-aws-s3-bucket&utm_content=discourse
   [email]: https://cpco.io/email?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/terraform-aws-s3-bucket&utm_content=email
   [commercial_support]: https://cpco.io/commercial-support?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/terraform-aws-s3-bucket&utm_content=commercial_support
   [we_love_open_source]: https://cpco.io/we-love-open-source?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/terraform-aws-s3-bucket&utm_content=we_love_open_source

diff --git a/docs/terraform.md b/docs/terraform.md
@@ -2,16 +2,21 @@
 
 | Name | Description | Type | Default | Required |
 |------|-------------|:----:|:-----:|:-----:|
+| abort_incomplete_multipart_upload_days | Maximum time (in days) that you want to allow multipart uploads to remain in progress | number | `5` | no |
 | acl | The canned ACL to apply. We recommend `private` to avoid exposing sensitive information | string | `private` | no |
 | allow_encrypted_uploads_only | Set to `true` to prevent uploads of unencrypted objects to S3 bucket | bool | `false` | no |
 | allowed_bucket_actions | List of actions the user is permitted to perform on the S3 bucket | list(string) | `<list>` | no |
 | attributes | Additional attributes (e.g. `1`) | list(string) | `<list>` | no |
 | delimiter | Delimiter to be used between `namespace`, `environment`, `stage`, `name` and `attributes` | string | `-` | no |
+| enable_glacier_transition | Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files | bool | `true` | no |
 | enabled | Set to false to prevent the module from creating any resources | bool | `true` | no |
 | environment | Environment, e.g. 'prod', 'staging', 'dev', 'pre-prod', 'UAT' | string | `` | no |
+| expiration_days | Number of days after which to expunge the objects | number | `90` | no |
 | force_destroy | A boolean string that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable | bool | `false` | no |
+| glacier_transition_days | Number of days after which to move the data to the glacier storage tier | number | `60` | no |
 | kms_master_key_arn | The AWS KMS master key ARN used for the `SSE-KMS` encryption. This can only be used when you set the value of `sse_algorithm` as `aws:kms`. The default aws/s3 AWS KMS master key is used if this element is absent while the `sse_algorithm` is `aws:kms` | string | `` | no |
 | lifecycle_rule_enabled | Enable or disable lifecycle rule | bool | `false` | no |
+| lifecycle_tags | Tags filter. Used to manage object lifecycle events | map(string) | `<map>` | no |
 | name | Solution name, e.g. 'app' or 'jenkins' | string | `` | no |
 | namespace | Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' | string | `` | no |
 | noncurrent_version_expiration_days | Specifies when noncurrent object versions expire | number | `90` | no |
@@ -21,6 +26,7 @@
 | region | If specified, the AWS region this bucket should reside in. Otherwise, the region used by the callee | string | `` | no |
 | sse_algorithm | The server-side encryption algorithm to use. Valid values are `AES256` and `aws:kms` | string | `AES256` | no |
 | stage | Stage, e.g. 'prod', 'staging', 'dev', OR 'source', 'build', 'test', 'deploy', 'release' | string | `` | no |
+| standard_transition_days | Number of days to persist in the standard storage tier before moving to the infrequent access tier | number | `30` | no |
 | tags | Additional tags (e.g. `map('BusinessUnit','XYZ')` | map(string) | `<map>` | no |
 | user_enabled | Set to `true` to create an IAM user with permission to access the bucket | bool | `false` | no |
 | versioning_enabled | A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket | bool | `false` | no |

diff --git a/main.tf b/main.tf
@@ -23,19 +23,43 @@ resource "aws_s3_bucket" "default" {
   }
 
   lifecycle_rule {
-    id      = module.label.id
-    enabled = var.lifecycle_rule_enabled
-    prefix  = var.prefix
-    tags    = module.label.tags
-
-    noncurrent_version_transition {
-      days          = var.noncurrent_version_transition_days
-      storage_class = "GLACIER"
-    }
+    id                                     = module.label.id
+    enabled                                = var.lifecycle_rule_enabled
+    prefix                                 = var.prefix
+    tags                                   = var.lifecycle_tags
+    abort_incomplete_multipart_upload_days = var.abort_incomplete_multipart_upload_days
 
     noncurrent_version_expiration {
       days = var.noncurrent_version_expiration_days
     }
+
+    dynamic "noncurrent_version_transition" {
+      for_each = var.enable_glacier_transition ? [1] : []
+
+      content {
+        days          = var.noncurrent_version_transition_days
+        storage_class = "GLACIER"
+      }
+    }
+
+    transition {
+      days          = var.standard_transition_days
+      storage_class = "STANDARD_IA"
+    }
+
+    dynamic "transition" {
+      for_each = var.enable_glacier_transition ? [1] : []
+
+      content {
+        days          = var.glacier_transition_days
+        storage_class = "GLACIER"
+      }
+    }
+
+    expiration {
+      days = var.expiration_days
+    }
+
   }
 
   # https://docs.aws.amazon.com/AmazonS3/latest/dev/bucket-encryption.html

diff --git a/variables.tf b/variables.tf
@@ -129,3 +129,40 @@ variable "noncurrent_version_expiration_days" {
   default     = 90
   description = "Specifies when noncurrent object versions expire"
 }
+
+variable "standard_transition_days" {
+  type        = number
+  default     = 30
+  description = "Number of days to persist in the standard storage tier before moving to the infrequent access tier"
+}
+
+variable "glacier_transition_days" {
+  type        = number
+  default     = 60
+  description = "Number of days after which to move the data to the glacier storage tier"
+}
+
+variable "enable_glacier_transition" {
+  type        = bool
+  default     = true
+  description = "Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files"
+}
+
+variable "expiration_days" {
+  type        = number
+  default     = 90
+  description = "Number of days after which to expunge the objects"
+}
+
+variable "abort_incomplete_multipart_upload_days" {
+  type        = number
+  default     = 5
+  description = "Maximum time (in days) that you want to allow multipart uploads to remain in progress"
+}
+
+variable "lifecycle_tags" {
+  type        = map(string)
+  description = "Tags filter. Used to manage object lifecycle events"
+  default     = {}
+}
+