Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update google_dataflow_flex_template_job to send sdkpipeline parameters via environment field #6357

Conversation

modular-magician
Copy link
Collaborator

Fixes hashicorp/terraform-provider-google#14679

Since the above issue was introduced, the flex template job resource has had the problem where fields can be sent via both an environment block and the parameters one -- which could lead to permadiffs due to the environment block fields not being flagged as computed, and would lead to guaranteed errors on max_workers and num_workers because they would be sent via both blocks always even when only included in parameters if the user applied a plan through the permadiff (as integer nils are equivalent to 0 within Terraform).

This has been resolved by:

  • Flagging all environment block fields as computed (which while resulting in testing complications, reflects their actual functionality with the parameter block)
  • Customize diff to synchronize parameter fields and their corresponding environment{} variable
  • Creating an error message if users attempt to update fields in the environment{} variable if the corresponding parameter option was also supplied, as an override value being entered for a computed field would lead to a disjointed state due to logic in above bullet being cancelled out. Additionally this would present false functionality in that the create/update commands will always take a parameter-value as the value to be sent.
  • During Create and Update the called resourceDataflowFlexJobSetupEnv() function will now update the ResourceData to assign the parameter values to their corresponding environment variables, and remove those parameters from the parameters block sent to the API.

Plus resolved a bonus undocumented issue

  • Updating max_worker's name within the read command to properly pull its value from the API

This technically will not be a breaking change!

Release Note Template for Downstream PRs (will be copied)

`dataflow`: fixed permadiff when SdkPipeline values are supplied via parameters. 
`dataflow`: fixed max_workers read value permanently displaying as 0.
`dataflow`: fixed issue causing error message when max_workers and num_workers were supplied via parameters.

Derived from GoogleCloudPlatform/magic-modules#9031

…rs via environment field (hashicorp#9031)

* updated dataflow problem with sdkpipelineresources

* removing leftover comments, removing unused diffsuppressfunc that was not implemented in final fix

* functionized repeated code

* formatting/bugs

* extra line

* bad var

* comments + update test

Signed-off-by: Modular Magician <[email protected]>
@modular-magician modular-magician merged commit dd58dbd into hashicorp:FEATURE-BRANCH-major-release-5.0.0 Sep 22, 2023
diviner524 pushed a commit to diviner524/k8s-config-connector that referenced this pull request Nov 16, 2023
Make the same changes in
hashicorp/terraform-provider-google-beta#6357
to update google_dataflow_flex_template_job to send SdkPipeline parameters
via environment block fields, and to fix the issue that the returned
maxWorkers was always 0.
diviner524 pushed a commit to diviner524/k8s-config-connector that referenced this pull request Nov 17, 2023
Make the same changes in
hashicorp/terraform-provider-google-beta#6357
to update google_dataflow_flex_template_job to send SdkPipeline parameters
via environment block fields, and to fix the issue that the returned
maxWorkers was always 0.
diviner524 pushed a commit to diviner524/k8s-config-connector that referenced this pull request Nov 29, 2023
Make the same changes in
hashicorp/terraform-provider-google-beta#6357
to update google_dataflow_flex_template_job to send SdkPipeline parameters
via environment block fields, and to fix the issue that the returned
maxWorkers was always 0.
@modular-magician modular-magician deleted the downstream-pr-18480ba1c4d41638ea5aec6cc5fa004fdcbbc5aa branch November 16, 2024 03:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant