Can't run website in a container via make container-serve due to the image absence

[shurup]

This is a Bug Report

Problem:

Running make container-serve in kubernetes/website:main (to run the website locally in a container) leads to an error:

Unable to find image 'gcr.io/k8s-staging-sig-docs/k8s-website-hugo:v0.133.0-af5f894e895c' locally
docker: Error response from daemon: manifest for gcr.io/k8s-staging-sig-docs/k8s-website-hugo:v0.133.0-af5f894e895c not found: manifest unknown: Failed to fetch "v0.133.0-af5f894e895c" from request "/v2/k8s-staging-sig-docs/k8s-website-hugo/manifests/v0.133.0-af5f894e895c".
See 'docker run --help'.
make: *** [Makefile:119: container-serve] Error 125

It happens on Linux/amd64. This behaviour is confirmed by a few people.

Proposed Solution:

Following the #49444 discussion, it should be fixed not by prior executing make container-image (to build images locally), but by the availability of Hugo images that can be pulled from GCR instead. As @sftim noted in Slack, they might be absent due to a recent Docsy upgrade. We need to have them back.

[niranjandarshann]

I am also facing the same issue.

[sftim]

/triage accepted
/priority important-soon

Only affects contributors, not website visitors, but we should get a fix in place; we may need to revisit how we build and publish the container image(s)

[sftim]

Until we fix this, there are two workarounds you can use:

• reference an older container image
  make container-serve CONTAINER_IMAGE=gcr.io/k8s-staging-sig-docs/k8s-website-hugo:v0.133.0-a5ef70d3da97
• build your own image
  make container-image # only needed one time
make container-serve

---

If you like helping out, SIG Docs can guide you towards working on a longer term fix. The best place to offer help is the #sig-docs channel on Slack.

[SayakMukhopadhyay]

Got the link to the failing prow job on the 15th Jan https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/post-website-push-image-k8s-website-hugo/1879257326494420992

A cursory glance makes me think that the issue might have something to do with npm and package.json but I will need to look into it more.

EDIT: Maybe its related to ed1cc81

[sftim]

We should check that the container build works for AArch64 (ideally: we also add some CI checks to reject PRs that break image builds)

I think it builds fine for AMD64, but maybe only for AMD64. In the cloud we build a multiarch image.

[ameukam]

#49529 is an attempt to debug the issue.
See logs: https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/post-website-push-image-k8s-website-hugo/1882444436592398336

[sftim]

I (still) suspect that a local AArch64 build would also fail.

[ameukam]

Yeah. I see:

#24 84.50 npm error npm error ld-linux-aarch64.so.1: /root/.npm/_cacache/tmp/git-cloneXXXXXXMFelHl/node_modules/hugo-extended/vendor/hugo: Not a valid dynamic program
#24 84.50 npm error npm error ✖ Hugo installation failed. :(
#24 84.50 npm error npm error node:internal/errors:984
#24 84.50 npm error npm error   const err = new Error(message);

Looks like an issue with NPM rather than the infrastructure. Possible the alpine image no longer have this lib for aarch64