A file placed in a sym-linked folder (/var/run) during build disappears from the finished image

[mhvelplund]

Actual behavior

A file placed in a sym-linked folder (/var/run) during build disappears from the finished image.

Expected behavior

The same Dockerfile built locally with Docker does not remove the file. It correctly appears in the target folder /run.

To Reproduce
Steps to reproduce the behavior:

Building with Docker:

$ IMAGE_NAME=django_app:local
$ docker build . -t $IMAGE_NAME
$ docker run -u root -t --rm $IMAGE_NAME ls -l /var/run/
total 8
-rw-r--r-- 1 root    root       0 Aug 27 09:25 adduser
drwxrwxr-x 2 oralvis oralvis 4096 Aug 27 09:26 gunicorn
drwxrwxrwt 2 root    root    4096 Jun 12 02:00 lock

Building with Kaniko:

$ IMAGE_NAME=django_app:kaniko
$ docker run \
 -v $PWD:/build \
 gcr.io/kaniko-project/executor:latest \
 --dockerfile=Dockerfile \
 --tar-path /build/django_app.tar \
 --no-push \
 --context /build \
 --destination $IMAGE_NAME && \
$ docker load -i django_app.tar && \
$ docker run -u root -t --rm $IMAGE_NAME ls -l /var/run/
total 4
-rw-r--r-- 1 root root    0 Aug 27 11:50 adduser
drwxrwxrwt 2 root root 4096 Aug 14 02:00 lock

... the gunicorn file is missing?

Additional Information

• Dockerfile (slightly trimmed for anonymity):

FROM python:3-slim-bookworm
ENV DEBIAN_FRONTEND=noninteractive
EXPOSE 8000
RUN adduser --disabled-password --gecos "" app && \
    apt update -q &&  apt upgrade -y --no-install-recommends && \
    apt install -y --no-install-recommends libgl1-mesa-glx libglib2.0-0 && \
    rm -rf /var/lib/apt/lists/* && \
    pip install --no-cache-dir --upgrade pip && \
    pip install --no-cache-dir gunicorn && \
    mkdir -pv /var/run/gunicorn/ && \
    chown -cR app:app /var/run/gunicorn/  && \
    chmod -R 775 /var/run/gunicorn/
WORKDIR /app
COPY --chown=app:app requirements.txt .
RUN pip install -r requirements.txt
USER app
COPY --chown=app:app django_app /app/django_app
WORKDIR /app/django_app
RUN python manage.py collectstatic --no-input
CMD [ "gunicorn", "-c", "gunicorn.config.py"]

• Kaniko Image (fully qualified with digest):

$ docker inspect gcr.io/kaniko-project/executor:latest
[
    {
        "Id": "sha256:03375da0f864ee91d092750766b3c9fecab70aa1b0ebde6021b369b1cb8b097a",
        "RepoTags": [
            "gcr.io/kaniko-project/executor:latest"
        ],
        "RepoDigests": [
            "gcr.io/kaniko-project/executor@sha256:55182847c4c0b3a5b4b285ec7a1ee26749777b15339fae66fd20c1fe62af8e3e"
        ],
        "Parent": "",
        "Comment": "buildkit.dockerfile.v0",
        "Created": "2023-08-15T21:18:36.362296939Z",
        ...

Triage Notes for the Maintainers

Description	Yes/No
Please check if this a new feature you are proposing
Please check if the build works in docker but not in kaniko
Please check if this error is seen when you use --cache flag
Please check if your dockerfile is a multistage dockerfile

[kevingigiano]

I ran into the same exact issue.
I don't know what the fix is but I have a work around.
Since /var/run is a symlink to /run, I decided to try putting my directory directly in /run.

This does not work:
RUN mkdir /var/run/foobar && chmod 777 /var/run/foobar
Workaround:
RUN mkdir /run/foobar && chmod 777 /run/foobar

Once the build completed, I pulled the image and my folder showed up in /var/run/foobar

Hope this helps someone else.

[mhvelplund]

I arrived at the same work-around but forgot to post an update :)