DXE-3308 IVM Import Contract Discrepancies and IVM State Confusion when applying while activation in progress

[hypnotisttom]

Terraform Version

v1.3.1

Problem Statement

When attempting to import the IVM resources, they all accept the contract with or without the ctr_ prefix; however, if you provide the prefix for the policy set, drift will be detected wanting it removed, and if you don't provide the prefix for the polices, drift will be detected asking you to put it on.

When applying a policy that activates on the production network, and then subsequently changing the policy and applying that, an api error returns while the activation rolls out (expected). However, the change is accepted to staging, and then doesn't hit production. The state file is updated, so no drift is detected on subsequent runs. That then leaves us in a situation where there is no reasonable way to promote a staged policy to the production network. Not placing it in the state file wouldn't accurately reflect the fact it rolled out to staging either.

Affected Resource(s)

• akamai_imaging_policy_set
• akamai_imaging_policy_image
• akamai_imaging_policy_video

Expected Behavior

Regarding import, I believe the way the other resources work is they either drop or add the ctr_ prefix in the provider to match the way it wants to be added to the state file.

Regarding the state issue, if an error occurs, normally I would say the state should not be updated so that the drift is detected with a subsequent plan; however, what is occurring here is the staging change occurs, but the production one does not, and as a result, there is no way that I know of to re-trigger the production deployment. So, not touching state would be out of sync with staging, but touching state prevents drift for being able to trigger the production run.

Actual Behavior

Regarding the import, both methods (with or without the ctr_ prefix) are accepted via import; however, the policyset will show drift if you provide the prefix (it wants it without), whereas the policy will show drift if you don't provide the prefix (it wants the prefix provided).

Regarding the state issue, after the run fails due to an existing roll out on the production network, subsequent plans show no drift. So, we have a situation where staging is the latest version, but prod is a version behind, with no way to trigger a subsequent production deployment.

Steps to Reproduce

Import Issue

1. Import a akamai_imaging_policy_set with the ctr_ prefix.
2. Run a plan
3. Notice the drift asking to remove it
4. Import a akamai_imaging_policy_image or a akamai_imaging_policy_video without the ctr_ prefix.
5. Run a plan
6. Notice the drift asking to add it in

State Issue

1. Apply a config that activates on production (policy_image or policy_video)
2. Change the policy in the terraform hcl
3. Apply that new config
4. Receive error that activation is in progress
5. Re-run apply - no drift
6. After 1 hour estimated, notice how the new policy is on staging but not on production, and there's no reasonable way to use terraform to roll it out to the production network

[lsadlon]

Hi @hypnotisttom

Thank you for raising this issue.
We were able to reproduce first issue with ctr_ prefix, but unfortunately wasn't able to reproduce second one. Could you share more detail about that? Preferably HCL configuration without sensitive data.

BR,
Lukasz

[hypnotisttom]

Sure @lsadlon.

Resource:

resource "akamai_imaging_policy_image" "this" {
  activate_on_production = var.activate_on_production
  contract_id            = data.akamai_contract.this.id
  policy_id              = local.policy_id
  policyset_id           = local.policyset_id
  json                   = jsonencode(var.policy)
}

activate_on_production being set to true for both runs.

Example first policy:

          breakpoints:
            widths:
              - 500
              - 720
              - 1080
              - 1366
              - 1920
              - 5000
          output:
            perceptualQuality: mediumHigh
          transformations:
            - aspect: ignore
              height:
                var: a
              transformation: Resize
              type: normal
              width:
                var: b
          variables:
            - defaultValue: "500"
              name: a
              type: number
            - defaultValue: "500"
              name: b
              type: number

We then plan and apply this version. That kicks off a "rolling out" to production.

Then I planned and attempted to apply:

          breakpoints:
            widths:
              - 500
              - 720
              - 1080
              - 1366
              - 1920
              - 5000
          output:
            perceptualQuality: mediumHigh
          transformations:
            - aspect: ignore
              height:
                var: h
              transformation: Resize
              type: normal
              width:
                var: w
          variables:
            - defaultValue: "500"
              name: h
              type: number
            - defaultValue: "500"
              name: w
              type: number

(changing the variable names to w/h)

This then returned the API error saying something along the lines of an operation in progress and asked to retry in an hour (by giving an explicit time).

When I then checked in the UI, staging had the latest version, but prod didn't. After roll out, prod remained "out of sync" with staging.

When I attempted to replan either during the rolling out period or after roll out completed, I got no changes detected.

Please let me know if you are able to reproduce using this.

If you aren't able to reproduce, please let me know what behavior you are seeing: are you not getting the api error, are you not getting the "no changes" message?

For me, this is like a chicken-and-egg scenario. If staging rolls out but prod doesn't, I should see drift for prod, but state needs to know about staging.

The "working state" I think that makes sense (which is not what is happening) is if my second apply is accepted (no api error), queued, and then rolled out after the first roll out completes. So, I'm really curious when you say it can't be reproduced, what behavior you are seeing?

Personally, I would prefer a separate activation resource for at least production this way state between the networks is separate, and I can orchestrate it separately. Having to update the definition of a resource for activation makes ci/cd rather tricky. I do realize the provider is bound by the API limitations as well.

Please let me know if that helps or if you need anything further.

Thanks!

[lsadlon]

Hi @hypnotisttom

Thanks for details. Finally we were able to reproduce this issue. I created internal ticket for it.

BR,
Lukasz

[hypnotisttom]

Glad to hear it!

I'm curious on what approach you'll take to resolve (once you have a plan), as this one is a bit tricky. Perhaps the separate production activation resource?

Thanks for taking a look,

Tom

[lsadlon]

Hi,

We decide to split contract prefix and state issue for tracking proposes. You can track contract prefix issue in this ticket, and state issue in this new one: #493

BR,
Lukasz

[lkowalsk-akamai-com]

This issue should be fixed now with Akamai Terraform Provider 5.5.0 release