S3 bucket - not found

[martaver]

First up, my deepest thanks for the great examples and useful modules you guys have laid out!

I'm trying to use this module to bootstrap an S3-backed cloudfront distribution. When I'm applying this module, though, I get an error trying to create the associated origin S3 bucket:

module.cdn.data.aws_s3_bucket.selected: data.aws_s3_bucket.selected: Failed getting S3 bucket: NotFound: Not Found

If I understand the module's code correctly, the module should be creating the S3 bucket if no origin_bucket parameter is provided. This is my basic configuration:

module "cdn" {
  source           = "git::https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn.git?ref=master"
  namespace        = "${var.namespace}"
  stage            = "${var.environment}"
  name             = "${var.name}"
  aliases          = ["app.${var.domain_name}"]
  parent_zone_name = "${var.domain_name}"
}

So with my configuration, shouldn't this module be creating the S3 bucket itself and returning its name/arn?

[osterman]

Out of curiousity, can you try a previous release of this module. We recently made some big changes and I am concerned that broke it.

e.g. try 0.1.7

[osterman]

(yes, it should create the bucket)

[Jamie-BitFlight]

This was last tested using the example code in the example folder and it created. But it is a good idea to test the previous version!
Sometimes, you may notice that there are creation delays, and that running the terraform apply over the top of a previous apply may also fix it.

[mike-zipit]

Yep, 0.1.7 works (creating bucket), 0.2.0 bails with a bucket not found

[danlamanna]

Yep, 0.1.7 works (creating bucket), 0.2.0 bails with a bucket not found

Can confirm the same happened here, running Terraform v0.11.7.

[osterman]

@danlamanna we'll try to find some more time to get back to this.

Pull requests welcome =)

[jschaul]

I can confirm the same issue; strangely only happens when adding 2 blocks of module "cdn" { ... and module "cdn2" { with different names respectively, the second CDN module fails with S3 bucket not found.

[osterman]

@jschaul oh wierd! Thanks for sharing.

@Jamie-BitFlight see this?

[garzparz]

Any idea what a work around could be? No Found error as well here.
I have no CDN in the account, fresh account. This module wont deploy as is, only building a single CDN.

Terraform v0.11.8

namespace = "${var.dept}"
stage = "${var.env}"
name = "${var.dept}-${var.app}-cdn"
compress = "true"
forward_cookies = "all"
use_regional_s3_endpoint = "true"
origin_force_destroy = "true"
allowed_methods = ["GET", "HEAD", "OPTIONS"]
cors_allowed_headers = [""]
cors_allowed_methods = ["GET", "HEAD"]
cors_allowed_origins = [""]
cors_expose_headers = ["ETag"]

[osterman]

Ok! We definitely need to get this fixed. For now, can you use an earlier release of the module?

[Jamie-BitFlight]

Addressing this now

[Jamie-BitFlight]

@garzparz what region is this in?

[Jamie-BitFlight]

Can someone please see if my new branch regional_support resolves your issues?

module "cdn" {
  source           = "git::https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn.git?ref=regional_support"
  namespace        = "${var.namespace}"
  stage            = "${var.environment}"
  name             = "${var.name}"
  aliases          = ["app.${var.domain_name}"]
  parent_zone_name = "${var.domain_name}"
}

[Jamie-BitFlight]

@martaver can you try the new branch too?

[aknysh]

@Jamie-BitFlight
I tested the master branch yesterday and it worked (in us-west-2).
So I think it maybe depends on the region, or some race conditions happen.

Does this exists in all regions?

variable "static_s3_bucket" {
  description = <<DOC
aws-cli is a bucket owned by amazon that will perminantly exist
It allows for the data source to be called during the destruction process without failing.
It doesn't get used for anything else, this is a safe workaround for handling the fact that
if a data source like the one `aws_s3_bucket.selected` gets an error, you can't continue the terraform process
which also includes the 'destroy' command, where is doesn't even need this data source!
Don't change this bucket name, its a variable so that we can provide this description.
And this works around a problem that is an edge case.
DOC

  default = "aws-cli"
}

[garzparz]

@Jamie-BitFlight I tested your new module, same issue. Deployed to an account created by Landing Zone. So we have a master aws account, that has two AWS organizations accounts under it. I have deployed your module successfully under one AWS org account, but when using a second AWS org account from same master, I get the Not Found error. Not sure if my env could be contributing.

Is there some quick hack I could make to my fork to get past this? Function logic in terraform is a little tricky in your module.

[Jamie-BitFlight]

Would you mind giving me a snippet of how you are using the module? So I can test it using a similar set up?

[garzparz]

@Jamie-BitFlight I'm creating my own origen bucket now and will try passing that in, my config below was assuming the S3 bucket would be created by the module.

module "cdn" {
source = "git::https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn.git?ref=regional_support"
namespace = "abc"
stage = "dev"
name = "aaa-bbb-cdn"
compress = "true"
forward_cookies = "all"
allowed_methods = ["GET", "HEAD", "OPTIONS"]
cors_allowed_headers = [""]
cors_allowed_methods = ["GET", "HEAD"]
cors_allowed_origins = [""]
cors_expose_headers = ["ETag"]

tags = {
terraform = "true"
env = "dev"
dept = "aaa"
app = "bbb"
cost-center = "0"
}
}

[garzparz]

Quick update, I had a terraform state file with no owner for this new module. Deleted, removed entry form lock table for new module and rebuilt, the original error I had reported is now gone. I can build the cdn letting it create S3 buckets.

[Jamie-BitFlight]

Thanks!

[osterman]

@martaver @mike-zipit @jschaul @danlamanna does @Jamie-BitFlight's PR fix your issues? Are you still encountering the issues?

[seanfdnn]

I'm experiencing the same problem with both master and the branch. I'm using the syntax exactly as in the example. I've tried applying on top of an apply with the same problem. It's in us-west-2, in a sub-account (not master account)

[osterman]

@aknysh can you see if you can reproduce this? Seems to be affecting a lot of folks :/