Kaniko deletes all content inside $HOME directory when using multiple FROM elements inside Dockerfile.

[GamePlayer-8]

Actual behavior
When I'm using multiple FROM <image name> inside Dockerfile kaniko somehow deletes everything inside $HOME directory (at least when running on root). Causing the push option of the image to error since the credentials are stored in $HOME/.docker.

Expected behavior
Kaniko should build an image with multiple FROM <image name> elements just fine, without deleting content inside $HOME and pushing the image without any problem.

To Reproduce
Steps to reproduce the behavior:

1. Install kaniko inside i.e. alpine container.
2. Install docker
3. Create credentials file with docker login
4. Create an example Dockerfile outside of $HOME directory:

FROM alpine AS builder

RUN echo "First image."

FROM alpine

COPY --from=builder /bin/sh .

RUN echo "Second image."

CMD ["/bin/sh"]

5. Build a Docker image and try to push it onto the specified docker registry (from docker login). Use command kaniko --context . --dockerfile ./Dockerfile --destination your.registry/username/package
6. kaniko should crash and after doing ls -a $HOME the directory should be shown as empty.

Description	Yes/No
Please check if this a new feature you are proposing
Please check if the build works in docker but not in kaniko
Please check if this error is seen when you use --cache flag
Please check if your dockerfile is a multistage dockerfile

[JeromeJu]

Thanks for the issue @GamePlayer-8 . Would you mind providing a bit more detail on the exact path of the secrets being in $HOME and the context?

[GamePlayer-8]

The path of secrets of $HOME is $HOME/.docker/config.json. In the exact case /root/.docker/config.json.

The context of config.json is:

{
    "auths": {
        "<registry_url>": {
        "auth": "<base64 encoded auth data>"
        }
    }
}

I've decided to change $HOME path to /kaniko and kaniko wasn't removed the content of /kaniko. The problem occurs when running as root and having the default $HOME configured at /root.

[JeromeJu]

Thanks for the pointers on this from @aaron-prindle , for this issue, the root cause could be that $HOME fell under the ignored list dirs and would be cleared/ignored after the RUN. Please feel free to correct/supplement if there are anything missing. 🙏

[cyprianbergoniatmo]

We're seeing more files being deleted.
We have kaniko installed in an image. We also install other executables in / and in /usr/local/bin. After running kaniko, files are missing in / and in /usr/local/bin. For example /crane and everything in the /usr/local/bin folder.
home is /root. Unrelated but to remove a possible variable, for the container logs and execution below, the /certs folder was mounted.

Before build:

/workspace # ls /
bin        crane      etc        kaniko     media      opt        root       sbin       sys        usr        workspace
certs      dev        home       lib        mnt        proc       run        srv        tmp        var
/workspace # ls /usr/local/bin
2to3               idle3              pip3               pydoc3.12          python3-config     wheel
2to3-3.12          idle3.12           pip3.12            python             python3.12
crane              jinjanate          pydoc              python-config      python3.12-config
idle               pip                pydoc3             python3            scannercli

After build:

/workspace # ls  /
bin     dev     home    lib     mnt     proc    run     sh      sys     usr
certs   etc     kaniko  media   opt     root    sbin    srv     tmp     var
/workspace # ls /usr/local/bin -la
total 8
drwxr-xr-x    2 root     root          4096 Nov  6 17:16 .
drwxr-xr-x    5 root     root          4096 Nov  6 17:16 ..

[manics]

Here's a minimal example using the latest gcr.io/kaniko-project/executor:debug image- /bin is missing at the end.

podman run -it --rm --entrypoint /busybox/sh gcr.io/kaniko-project/executor:debug -c '/kaniko/executor version && ls -l /bin && mkdir /src && echo -e "FROM scratch\nFROM scratch" > /src/Dockerfile && cat /src/Dockerfile && /kaniko/executor --context /src --no-push --cleanup && ls -l /bin'

Kaniko version :  v1.17.0
total 0
lrwxrwxrwx    1 0        0               11 Oct 18 17:06 sh -> /busybox/sh
FROM scratch
FROM scratch
INFO[0000] No base image, nothing to extract            
INFO[0000] No base image, nothing to extract            
INFO[0000] Built cross stage deps: map[]                
INFO[0000] No base image, nothing to extract            
INFO[0000] Executing 0 build triggers                   
INFO[0000] Building stage 'scratch' [idx: '0', base-idx: '-1'] 
INFO[0000] Skipping unpacking as no commands require it. 
INFO[0000] Deleting filesystem...                       
INFO[0000] No base image, nothing to extract            
INFO[0000] Executing 0 build triggers                   
INFO[0000] Building stage 'scratch' [idx: '1', base-idx: '-1'] 
INFO[0000] Skipping unpacking as no commands require it. 
INFO[0000] Deleting filesystem...                       
INFO[0000] Skipping push to container registry due to --no-push flag 
ls: /bin: No such file or directory

[barrykp]

I have run into what I believe is a similar issue.

I have reproduced this using the executor:debug container with sha: executor@sha256:b6a520e10252cfbb1898cac88d05f4c7c838cf615d1a588ec63a0d415db6c4c0

Before running kaniko I can ls /bin/sh and it's there.
Afterwards it is gone.

[atorrembo]

May be related to #2576 ?

[DeanHnter]

Im still experiencing this issue when running kaniko in an alpine base image. I tried multiple alternate ways to ignore some directories but no luck it deletes files in /bin /sbin /usr etc, in the end the container continues to work however the environment breaks in various ways

[rcollette]

Duplicate of #1572

[mzihlmann]

kaniko removes everything outside of the /kaniko dir when switching over to the second build stage, so if your credentials are stored outside this directory they are indeed gone. You can safe a file across boundaries by putting it on the ingore list:
https://github.com/GoogleContainerTools/kaniko#flag---ignore-path

--ignore-path=/root/.netrc

as an example

[SebastienTolron]

Is it possible to ignore the whole / or It might create problems ?

I am using the debug image so I can use busybox and some function. But after the multistage build I can't do anything.

This is a problem in a Gitlab Ci environment.

What do you recommend ?

I can't even make a ls after a multistage build !

Thanks

[jahusa02]

Facing the same issue. After cleanup I can't even ls or any other command