Zarf doesn't update zarf-registry secrets

[YrrepNoj]

The zarf-registry secret stores the credentials for the docker registry zarf hosts. If you run zarf destroy --confirm && zarf init --confirm a new docker registry is created and a new secret is created in the zarf namespace with valid credentials to to access the registry however any preexisting zarf-registry secrets in other namespaces are not updated leading to an imagePullBackOff if you try to deploy something else in that namespace.

Steps To Reproduce:

11520  ./zarf-mac-intel init --confirm
11521  ./zarf-mac-intel package deploy zarf-package-big-bang-single-package-demo.tar.zst --confirm.     // pods come up healthy
11522  ./zarf-mac-intel destroy --confirm

11523  ./zarf-mac-intel init --confirm
11524  ./zarf-mac-intel package deploy  zarf-package-big-bang-single-package-demo.tar.zst --confirm   // pods fail to pull image

[jeff-mccoy]

Can you confirm this is true if you add the remove components flag on destroy? We should still fix it either way, but that will help to know.

[YrrepNoj]

When doing zarf destroy --confirm --remove-components I'm able to re-init and deploy packages that have deploying to their own namespace. However anything that is deploying to the default namespace is still going to see the imagePullBackoff (such as our game example package).

[mahomedalid]

@YrrepNoj @jeff-mccoy

We are having some strange behavior that may be related to this, I see that this issue was fixed so probably not, I just want to double confirm. So the questions:

• should zarf init be run only once either at the cluster creation or some time later, and then we can do multiple zarf deploys without need to do zarf init again?
• Is the last question still true even if we switch kube contexts between different clusters that may or not have zarf initialized?

[jeff-mccoy]

@YrrepNoj is working on the docs currently, so sorry they suck 😭. But the idea is:

1. zarf init once per kube context--it's actually possible to treat it as idempotent, but there are some flaky behaviors around the registry bootup so we don't officially support that right now.
2. zarf package deploy uses the zarf secret to run it's operations in a given kube context. Unless you are only copying files, this will be needed--if it's missing Zarf will tell you.

Does that clarify? I plan on doing some more videos on this very soon to explain things better.

[vyta]

@jeff-mccoy Currently, zarf package deploy doesn't use the zarf secret to push images to he zarf registry. PushZarfImages() relies on local ~/.docker/config in order to push images via a tunnel to the zarf docker registry. So in essence, you must run zarf init and zarf package deploy from the same machine w/ the same user. Is this by design?

Context: we are looking at a use case where we "prep" a cluster for zarf packages, and do the deployment(s) as part of a separate workflow.

[jeff-mccoy]

😱 no that is not by design. This must have been missed when we moved this into the cluster secret. You do need to run init before trying to do k8s things with deploy in zarf (note if it's just file or scripts you don't need to have a zarf cluster yet, that's how the k3s and eks packages work).

The tests missed this because they are running on the same machine, we should blow those away @YrrepNoj during tests to verify functionality. Also we should do this for git secrets too.

[YrrepNoj]

I'll make an issue and look at getting this address by the end of the week. Thanks for pointing this out @vyta!

[vyta]

@YrrepNoj Just checking in on this.. I have a fix on my fork, happy to create an issue + PR if the work isn't already in flight

[YrrepNoj]

@vyta I would gladly take a look at the solution in your fork if you already have a fix prepared. I just created the issue for it that you can tag in the PR.

#477