Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3deeparchive support #78

Merged
merged 7 commits into from
Feb 22, 2021
Merged

S3deeparchive support #78

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8d76cc1
Enable S3 Glacier deep archive storage tier
Feb 22, 2021
bfe9907
Enable S3 Deep Archive in variables
Feb 22, 2021
cc91bd6
Readme up to date
Feb 22, 2021
4d5cc59
Auto Format
cloudpossebot Feb 22, 2021
f17f387
Back to defaults
Feb 22, 2021
89dacde
Merge branch 's3deeparchive' of https://github.com/bcoddens/terraform…
Feb 22, 2021
c622535
Auto Format
cloudpossebot Feb 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 25 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,26 @@ Available targets:
|------|---------|
| aws | >= 2.0 |

## Modules

| Name | Source | Version |
|------|--------|---------|
| s3_user | cloudposse/iam-s3-user/aws | 0.15.1 |
| this | cloudposse/label/null | 0.24.1 |

## Resources

| Name |
|------|
| [aws_iam_policy_document](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/data-sources/iam_policy_document) |
| [aws_iam_policy](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/iam_policy) |
| [aws_iam_role_policy_attachment](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/iam_role_policy_attachment) |
| [aws_iam_role](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/iam_role) |
| [aws_partition](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/data-sources/partition) |
| [aws_s3_bucket_policy](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/s3_bucket_policy) |
| [aws_s3_bucket_public_access_block](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/s3_bucket_public_access_block) |
| [aws_s3_bucket](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/s3_bucket) |

## Inputs

| Name | Description | Type | Default | Required |
Expand All @@ -199,9 +219,11 @@ Available targets:
| block\_public\_policy | Set to `false` to disable the blocking of new public policies on the bucket | `bool` | `true` | no |
| context | Single object for setting entire context at once.<br>See description of individual variables for details.<br>Leave string and numeric variables as `null` to use default value.<br>Individual variable settings (non-null) override settings in context object,<br>except for attributes, tags, and additional\_tag\_map, which are merged. | `any` | <pre>{<br> "additional_tag_map": {},<br> "attributes": [],<br> "delimiter": null,<br> "enabled": true,<br> "environment": null,<br> "id_length_limit": null,<br> "label_key_case": null,<br> "label_order": [],<br> "label_value_case": null,<br> "name": null,<br> "namespace": null,<br> "regex_replace_chars": null,<br> "stage": null,<br> "tags": {}<br>}</pre> | no |
| cors\_rule\_inputs | Specifies the allowed headers, methods, origins and exposed headers when using CORS on this bucket | <pre>list(object({<br> allowed_headers = list(string)<br> allowed_methods = list(string)<br> allowed_origins = list(string)<br> expose_headers = list(string)<br> max_age_seconds = number<br> }))</pre> | `null` | no |
| deeparchive\_transition\_days | Number of days after which to move the data to the glacier deep archive storage tier | `number` | `90` | no |
| delimiter | Delimiter to be used between `namespace`, `environment`, `stage`, `name` and `attributes`.<br>Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. | `string` | `null` | no |
| enable\_current\_object\_expiration | Enables the expiration of current objects | `bool` | `true` | no |
| enable\_glacier\_transition | Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files | `bool` | `true` | no |
| enable\_deeparchive\_transition | Enables the transition to AWS Glacier Deep Archive which can cause unnecessary costs for huge amount of small files | `bool` | `false` | no |
| enable\_glacier\_transition | Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files | `bool` | `false` | no |
| enable\_standard\_ia\_transition | Enables the transition to STANDARD\_IA | `bool` | `false` | no |
| enabled | Set to false to prevent the module from creating any resources | `bool` | `null` | no |
| environment | Environment, e.g. 'uw2', 'us-west-2', OR 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no |
Expand All @@ -220,8 +242,9 @@ Available targets:
| logging | Bucket access logging configuration. | <pre>object({<br> bucket_name = string<br> prefix = string<br> })</pre> | `null` | no |
| name | Solution name, e.g. 'app' or 'jenkins' | `string` | `null` | no |
| namespace | Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' | `string` | `null` | no |
| noncurrent\_version\_deeparchive\_transition\_days | Number of days to persist in the standard storage tier before moving to the glacier deeparchive access tier | `number` | `60` | no |
| noncurrent\_version\_expiration\_days | Specifies when noncurrent object versions expire | `number` | `90` | no |
| noncurrent\_version\_transition\_days | Number of days to persist in the standard storage tier before moving to the glacier tier infrequent access tier | `number` | `30` | no |
| noncurrent\_version\_glacier\_transition\_days | Number of days to persist in the standard storage tier before moving to the glacier infrequent access tier | `number` | `30` | no |
| policy | A valid bucket policy JSON document. Note that if the policy document is not specific enough (but still valid), Terraform may view the policy as constantly changing in a terraform plan. In this case, please make sure you use the verbose/specific version of the policy | `string` | `""` | no |
| prefix | Prefix identifying one or more objects to which the rule applies | `string` | `""` | no |
| regex\_replace\_chars | Regex to replace chars with empty string in `namespace`, `environment`, `stage` and `name`.<br>If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. | `string` | `null` | no |
Expand Down Expand Up @@ -252,7 +275,6 @@ Available targets:
| user\_enabled | Is user creation enabled |
| user\_name | Normalized IAM user name |
| user\_unique\_id | The user unique ID assigned by AWS |

<!-- markdownlint-restore -->


Expand Down
28 changes: 25 additions & 3 deletions docs/terraform.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,26 @@
|------|---------|
| aws | >= 2.0 |

## Modules

| Name | Source | Version |
|------|--------|---------|
| s3_user | cloudposse/iam-s3-user/aws | 0.15.1 |
| this | cloudposse/label/null | 0.24.1 |

## Resources

| Name |
|------|
| [aws_iam_policy_document](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/data-sources/iam_policy_document) |
| [aws_iam_policy](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/iam_policy) |
| [aws_iam_role_policy_attachment](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/iam_role_policy_attachment) |
| [aws_iam_role](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/iam_role) |
| [aws_partition](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/data-sources/partition) |
| [aws_s3_bucket_policy](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/s3_bucket_policy) |
| [aws_s3_bucket_public_access_block](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/s3_bucket_public_access_block) |
| [aws_s3_bucket](https://registry.terraform.io/providers/hashicorp/aws/2.0/docs/resources/s3_bucket) |

## Inputs

| Name | Description | Type | Default | Required |
Expand All @@ -26,9 +46,11 @@
| block\_public\_policy | Set to `false` to disable the blocking of new public policies on the bucket | `bool` | `true` | no |
| context | Single object for setting entire context at once.<br>See description of individual variables for details.<br>Leave string and numeric variables as `null` to use default value.<br>Individual variable settings (non-null) override settings in context object,<br>except for attributes, tags, and additional\_tag\_map, which are merged. | `any` | <pre>{<br> "additional_tag_map": {},<br> "attributes": [],<br> "delimiter": null,<br> "enabled": true,<br> "environment": null,<br> "id_length_limit": null,<br> "label_key_case": null,<br> "label_order": [],<br> "label_value_case": null,<br> "name": null,<br> "namespace": null,<br> "regex_replace_chars": null,<br> "stage": null,<br> "tags": {}<br>}</pre> | no |
| cors\_rule\_inputs | Specifies the allowed headers, methods, origins and exposed headers when using CORS on this bucket | <pre>list(object({<br> allowed_headers = list(string)<br> allowed_methods = list(string)<br> allowed_origins = list(string)<br> expose_headers = list(string)<br> max_age_seconds = number<br> }))</pre> | `null` | no |
| deeparchive\_transition\_days | Number of days after which to move the data to the glacier deep archive storage tier | `number` | `90` | no |
| delimiter | Delimiter to be used between `namespace`, `environment`, `stage`, `name` and `attributes`.<br>Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. | `string` | `null` | no |
| enable\_current\_object\_expiration | Enables the expiration of current objects | `bool` | `true` | no |
| enable\_glacier\_transition | Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files | `bool` | `true` | no |
| enable\_deeparchive\_transition | Enables the transition to AWS Glacier Deep Archive which can cause unnecessary costs for huge amount of small files | `bool` | `false` | no |
| enable\_glacier\_transition | Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files | `bool` | `false` | no |
| enable\_standard\_ia\_transition | Enables the transition to STANDARD\_IA | `bool` | `false` | no |
| enabled | Set to false to prevent the module from creating any resources | `bool` | `null` | no |
| environment | Environment, e.g. 'uw2', 'us-west-2', OR 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no |
Expand All @@ -47,8 +69,9 @@
| logging | Bucket access logging configuration. | <pre>object({<br> bucket_name = string<br> prefix = string<br> })</pre> | `null` | no |
| name | Solution name, e.g. 'app' or 'jenkins' | `string` | `null` | no |
| namespace | Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' | `string` | `null` | no |
| noncurrent\_version\_deeparchive\_transition\_days | Number of days to persist in the standard storage tier before moving to the glacier deeparchive access tier | `number` | `60` | no |
| noncurrent\_version\_expiration\_days | Specifies when noncurrent object versions expire | `number` | `90` | no |
| noncurrent\_version\_transition\_days | Number of days to persist in the standard storage tier before moving to the glacier tier infrequent access tier | `number` | `30` | no |
| noncurrent\_version\_glacier\_transition\_days | Number of days to persist in the standard storage tier before moving to the glacier infrequent access tier | `number` | `30` | no |
| policy | A valid bucket policy JSON document. Note that if the policy document is not specific enough (but still valid), Terraform may view the policy as constantly changing in a terraform plan. In this case, please make sure you use the verbose/specific version of the policy | `string` | `""` | no |
| prefix | Prefix identifying one or more objects to which the rule applies | `string` | `""` | no |
| regex\_replace\_chars | Regex to replace chars with empty string in `namespace`, `environment`, `stage` and `name`.<br>If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. | `string` | `null` | no |
Expand Down Expand Up @@ -79,5 +102,4 @@
| user\_enabled | Is user creation enabled |
| user\_name | Normalized IAM user name |
| user\_unique\_id | The user unique ID assigned by AWS |

<!-- markdownlint-restore -->
23 changes: 21 additions & 2 deletions main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,20 @@ resource "aws_s3_bucket" "default" {
for_each = var.enable_glacier_transition ? [1] : []

content {
days = var.noncurrent_version_transition_days
days = var.noncurrent_version_glacier_transition_days
storage_class = "GLACIER"
}
}

dynamic "noncurrent_version_transition" {
for_each = var.enable_deeparchive_transition ? [1] : []

content {
days = var.noncurrent_version_deeparchive_transition_days
storage_class = "DEEP_ARCHIVE"
}
}

dynamic "transition" {
for_each = var.enable_glacier_transition ? [1] : []

Expand All @@ -41,6 +50,17 @@ resource "aws_s3_bucket" "default" {
}
}

dynamic "transition" {
for_each = var.enable_deeparchive_transition ? [1] : []

content {
days = var.deeparchive_transition_days
storage_class = "DEEP_ARCHIVE"
}
}



dynamic "transition" {
for_each = var.enable_standard_ia_transition ? [1] : []

Expand Down Expand Up @@ -227,4 +247,3 @@ resource "aws_s3_bucket_public_access_block" "default" {
ignore_public_acls = var.ignore_public_acls
restrict_public_buckets = var.restrict_public_buckets
}

23 changes: 20 additions & 3 deletions variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,10 +85,16 @@ variable "prefix" {
description = "Prefix identifying one or more objects to which the rule applies"
}

variable "noncurrent_version_transition_days" {
variable "noncurrent_version_glacier_transition_days" {
type = number
default = 30
description = "Number of days to persist in the standard storage tier before moving to the glacier tier infrequent access tier"
description = "Number of days to persist in the standard storage tier before moving to the glacier infrequent access tier"
}

variable "noncurrent_version_deeparchive_transition_days" {
type = number
default = 60
description = "Number of days to persist in the standard storage tier before moving to the glacier deeparchive access tier"
}

variable "noncurrent_version_expiration_days" {
Expand Down Expand Up @@ -122,12 +128,23 @@ variable "glacier_transition_days" {
description = "Number of days after which to move the data to the glacier storage tier"
}

variable "deeparchive_transition_days" {
type = number
default = 90
description = "Number of days after which to move the data to the glacier deep archive storage tier"
}

variable "enable_glacier_transition" {
type = bool
default = true
default = false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bcoddens why to change default behaviour?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct,just tested it.
When you have in variables.tf:

variable "enable_glacier_transition" {
type = bool
default = true
description = "Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files"
}

== The default

variable "enable_deeparchive_transition" {
type = bool
default = false
description = "Enables the transition to AWS Glacier Deep Archive which can cause unnecessary costs for huge amount of small files"
}

And this in your main.tf:

enable_glacier_transition = "false"
enable_deeparchive_transition = "true"

It works fine according to the plan output:

      + noncurrent_version_expiration {
          + days = 90
        }

      + noncurrent_version_transition {
          + days          = 60
          + storage_class = "DEEP_ARCHIVE"
        }

      + transition {
          + days          = 90
          + storage_class = "DEEP_ARCHIVE"
        }
    }

description = "Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files"
}

variable "enable_deeparchive_transition" {
type = bool
default = false
description = "Enables the transition to AWS Glacier Deep Archive which can cause unnecessary costs for huge amount of small files"
}
variable "enable_standard_ia_transition" {
type = bool
default = false
Expand Down