Cannot add lifecycle_configuration_rules with expires and without

[ByJacob]

Found a bug? Maybe our Slack Community can help.

Describe the Bug

I'd like to add policies where one is with the expiration parameter and the other is not.

Expected Behavior

The policy will be added correctly

Steps to Reproduce

Steps to reproduce the behavior:

module "s3_example" {
  source             = "cloudposse/s3-bucket/aws"
  version            = "0.49.0"
  name               = local.bucket_name
  enabled            = true
  user_enabled       = false
  versioning_enabled = true

  logging = {
    bucket_name = module.s3_logs.bucket_id
    prefix      = "logs/"
  }
  lifecycle_configuration_rules = [
       {
           abort_incomplete_multipart_upload_days = 1
           enabled                                = true
           expiration                             = null
           filter_and                             = {
               prefix = "prefix1"
            }
           id                                     = "prefix1"
           noncurrent_version_expiration          = {
               newer_noncurrent_versions = 2
               noncurrent_days           = 30
            }
           noncurrent_version_transition          = []
           transition                             = [
               {
                   days          = 7
                   storage_class = "GLACIER"
                },
            ]
        },
       {
           abort_incomplete_multipart_upload_days = 1
           enabled                                = true
           expiration                             = {
               days                         = 93
               expired_object_delete_marker = false
            }
           filter_and                             = {
               prefix = "prefix2"
            }
           id                                     = "prefix2"
           noncurrent_version_expiration          = {
               newer_noncurrent_versions = 2
               noncurrent_days           = 14
            }
           noncurrent_version_transition          = []
           transition                             = [
               {
                   days          = 3
                   storage_class = "GLACIER"
                },
            ]
        }
    ]

  tags = local.common_tags
}

Screenshots

Error in terminal:

│   on main.tf line 275, in module "s3_client_backup":
│  275:   lifecycle_configuration_rules = [
│  ....
│  282:   ]
│ 
│ The given value is not suitable for child module variable
│ "lifecycle_configuration_rules" defined at
│ ../../modules/s3/variables.tf:136,1-41: element types must all match for
│ conversion to list.

Environment (please complete the following information):

Anything that will help us triage the bug will help. Here are some ideas:

• OS: Linux
• Version 0.49.0

Additional Context

By dynamic typing it requires that the items in the list will always be the same (expires will consist of the same keys)
https://www.terraform.io/language/expressions/type-constraints#dynamic-types-the-any-constraint

Workaround:

expiration = {
       days = 99999
       expired_object_delete_marker = false
    }

[nitrocode]

This is more of a terraform limitation then a module limitation.

I don't see how this is possible to be honest.

The workaround makes sense to me. Does -1 for days work?

    expiration = {
       days = -1
       expired_object_delete_marker = false
    }

[ByJacob]

In general, what they changed, they probably tested only on simple examples. 😄
When I set -1 I get a strange message:

│ Error: error updating S3 Bucket Lifecycle Configuration (bucket_name): InvalidRequest: ExpiredObjectDeleteMarker cannot be specified with Object Size.
│ 	status code: 400, request id: PLACEHOLDER, host id: PLACEHOLDER
│ 
│   with module.s3_internal_backup.aws_s3_bucket_lifecycle_configuration.default[0],
│   on ../../modules/s3/lifecycle.tf line 147, in resource "aws_s3_bucket_lifecycle_configuration" "default":
│  147: resource "aws_s3_bucket_lifecycle_configuration" "default" {

I try set expired_object_delete_marker to false and null, but it don't help.
But setting large values works fine.
It is possible that this is a problem with the object_size_greater_than parameter which I set to 1 to be able to configure the filter (I need to use the prefix parameter, but because it is in a block and there must be 2 parameters)

[Nuru]

@ByJacob There are a few issues here, none of which are specific to this module or something we can fix.

Terraform requires all the elements of a list to have the same type. This means complex objects need to have the same structure. To fix the terraform error

The given value is not suitable for child module variable

you need to replace expiration = null with

expiration                             = {
  days                         = null
  expired_object_delete_marker = null
}

This fixes Terraform's complaint and will cause apply to make changes. In my testing, the apply fails with

│ Error: error creating S3 Lifecycle Configuration for bucket (eg-test-s3-lifecycle-test): MalformedXML: The XML you provided was not well-formed or did not validate against our published schema

I have reported this as hashicorp/terraform-provider-aws#23882 and, while trying to find a workaround, reported hashicorp/terraform-provider-aws#23883 and hashicorp/terraform-provider-aws#23884 as well.

I want to wait and see what kind of response we get from AWS before going and further with this. Ideally they will fix the bugs and we will not need to make any changes in this module.

[Nuru]

Looks like this will be fixable once AWS provider v4.9.0 is released. See hashicorp/terraform-provider-aws#23893

[Nuru]

This was an upstream bug fixed in AWS Terraform provider v4.9.0