Add location attribute to google_tags_tag_binding

[robzr]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

Some tag bindings require the location to be specified, ex: Cloud SQL instances. Currently trying to bind a tag to a Cloud SQL instance fails:

│ Error: Error creating TagBinding: googleapi: Error 400: field [binding.resource] has issue [Must be a valid One Platform resource name of a tag-compatible global resource. Did you forget to specify the correct location?]
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.BadRequest",
│     "fieldViolations": [
│       {
│         "description": "Must be a valid One Platform resource name of a tag-compatible global resource. Did you forget to specify the correct location?",
│         "field": "binding.resource"
│       }
│     ]
│   }
│ ]

New or Affected Resource(s)

resource.google_tags_tag_binding

Potential Terraform Configuration

resource "google_tags_tag_binding" "this" {
  location  = "us-east1"
  parent    = "//sqladmin.googleapis.com/projects/${var.gcp_project}/instances/${module.sql.primary.name}"
  tag_value = "tagValues/1234567890"
}

References

[rileykarson]

Note: Cloud SQL Instances are supported, per https://cloud.google.com/resource-manager/docs/tags/tags-supported-services. I can't see a guide on attaching one, though, and the Tags API doesn't support a location field. Is it possible that they're using a different format, or you have a REST response from a tag that has been successfully attached to a SQL instance?

[alexbacchin-asx]

The location attribute is associated with the API endpoint, not to the the payload. I tried to run the CloudSQL gcloud command and I got this error.

gcloud crashed (ConnectionError): HTTPSConnectionPool(host='australia-southeast1-cloudresourcemanager.googleapis.com', port=443): Max retries exceeded with url: /v3/tagBindings?alt=json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f76467f7390>: Failed to establish a new connection: [Errno 101] Network is unreachable',))

[alexbacchin-asx]

I am not familiar with the contributing process, but we need to include an optional location parameter to the tags api definition
https://github.com/GoogleCloudPlatform/magic-modules/blob/main/mmv1/products/tags/api.yaml

Similar to cloudrun
https://github.com/GoogleCloudPlatform/magic-modules/blob/main/mmv1/products/cloudrun/api.yaml

[lazcamus]

Debugging this with gcloud --log-http on a compute instance. You can create a tagbinding with gcloud IF you provide --location.

I run:

$ gcloud resource-manager  --log-http  tags bindings create --tag-value=tagValues/111458724321 --parent=//compute.googleapis.com/projects/camus-infra/zones/us-central1-a/instances/laz-testing-nat --location=us-central1-a

which first calls the compute instances API to pull down the instance info, then POSTs this json payload to https://us-central1-a-cloudresourcemanager.googleapis.com/v3/tagBindings?alt=json:

{"parent": "//compute.googleapis.com/projects/camus-infra/zones/us-central1-a/instances/2094347600019513919", "tagValue": "tagValues/111458724321"}`

So gcloud is doing 2 tricks: looking up the numeric ID of the instance to include in the resource string, and posting to a regional hostname instead of cloudresourcemanager.googleapis.com

This doesn't seem to be covered in the API docs at https://cloud.google.com/resource-manager/reference/rest/v3/tagBindings/create

[kpocius]

I understand the issue is on the API side and not much can be done until location is provided, just wanted to leave a note for the sake of completion, that another affected service is GKE.

[alexbacchin-asx]

@kpocius the gcloud tag binding create documentation states location is only required for non global resources. Based on
@lazcamus, the location just point the regional api endpoint and it is not part of the payalod

[melinath]

b/247163900

[alextodicescu]

I came across this as well, using null local-exec as a workaround for now
eg.

resource "null_resource" "bucket_tagging" {
  for_each = var.bucket_names
  provisioner "local-exec" {
    command = "gcloud resource-manager tags bindings create --tag-value=${var.tag_value} --parent=//storage.googleapis.com/projects/_/buckets/${each.value} --location=${var.location}"
  }
}

I'll have a look next week to see if can contribute with an PR.

[nphilbrook]

I'll have a look next week to see if can contribute with an PR.

@alextodicescu I'm curious if you were able to make any progress on this? I am going to take a stab at it tomorrow as part of my employer's Open Source Hack Day.

[alextodicescu]

I haven't made much progress, no

been chatting about what changes are required here: #12738 (comment)

[melinath]

@AarshDhokai has been working on this but there are technical complications that may require refactoring parts of mmv1 core.

[melinath]

@AarshDhokai could you leave a comment here so I can assign it to you?

[nphilbrook]

I noticed this in the cloud run YAML:
https://github.com/GoogleCloudPlatform/magic-modules/blob/main/mmv1/products/cloudrun/api.yaml#L21
It looks like this is doing something similar? But I have not dug any further than this single file (e.g. haven't even peeped the generated golang, sorry).

[nphilbrook]

As per Riley over at #12738 (comment) it seems like one perspective from the provider maintainers would be to hand write these resources for now. @melinath are you on board with that approach?

[AarshDhokai]

Hey @melinath, just commenting so that you can assign the issue.

[melinath]

As per Riley over at #12738 (comment) it seems like one perspective from the provider maintainers would be to hand write these resources for now. @melinath are you on board with that approach?

yep looks like that's the plan for now.

[nphilbrook]

Hey @melinath, just commenting so that you can assign the issue.

Hey @AarshDhokai wondering what your timeline looks like on this issue. No pressure, just our team has an immediate use for this feature, so I was going to work on it next week if you don't have immediate plans. Don't want to duplicate any work :) Thanks!

[oboukili]

Even if it's clearly indicated that this is supported on a best-effort basis, It would seem the API client the google_tags_tag_binding resource is using is not supporting this resource_manager_custom_endpoint provider attribute unfortunately.

I was hoping to work around the current issue by instantianting specific regional provider aliases. Which would have been a cleaner workaround that calling the gcloud CLI through a null_resource.

provider "google" {
  alias = "regional-endpoint-workaround"

  resource_manager_custom_endpoint = format("https://%s-cloudresourcemanager.googleapis.com/v3/", local.region)"
}

resource "google_tags_tag_binding" "cloudrun_public_website_all_users_ingress" {
  parent = format("//run.googleapis.com/projects/%s/locations/%s/services/%s",
    data.google_cloud_run_service.public_website.project,
    data.google_cloud_run_service.public_website.location,
    data.google_cloud_run_service.public_website.name,
  )
  tag_value = format("tagValues/%s", data.google_tags_tag_value.all_users_ingress_true.name)
  provider  = google.regional-endpoint-workaround
}

Unfortunately, the global endpoint is still used:

2023-01-12T13:19:02.038+0100 [INFO]  provider.terraform-provider-google_v4.48.0_x5: 2023/01/12 13:19:02 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
POST /v3/tagBindings?alt=json HTTP/1.1
Host: cloudresourcemanager.googleapis.com
User-Agent: Terraform/1.3.6 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/4.48.0
...

[oboukili]

Update: so looking at the code, the resource uses a specific tags endpoint (actually targeting the resource manager v3 endpoint), which, while undocumented, can be fortunately overridden.

The following works:

provider "google" {
  alias = "regional-endpoint-workaround"

  tags_custom_endpoint        = format("https://%s-cloudresourcemanager.googleapis.com/v3/", local.region)
}

resource "google_tags_tag_binding" "cloudrun_public_website_all_users_ingress" {
  parent = format("//run.googleapis.com/projects/%s/locations/%s/services/%s",
    google_project.public_website.number, # permadiff if using the project id
    data.google_cloud_run_service.public_website.location,
    data.google_cloud_run_service.public_website.name,
  )
  tag_value = format("tagValues/%s", data.google_tags_tag_value.all_users_ingress_true.name)
  provider  = google.regional-endpoint-workaround
}

Use with caution and only for specific tag resources that need this workaround. Also, unsure what would happen if Google decided to use the resourcemanager v4+ endpoint to manage tags all of a sudden. Better wait for a proper fix.

[ghabian]

for anybody testing out location-scoped bindings (we released it as part of v4.51.0). Zonal resources are still broken since the operation check is still going to the global TagsBasePath instead of the overridden TagsLocationBasePath.

The tests included in the PR only test Cloud Run bindings (regional resource) but my testing with zonal resources (compute instances) is still failing

[ghabian]

tested that zonal bindings work now on >=4.57 (specifically compute instances)

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.