missing provenance errors when context does not exist

[nicks]

Description

If I try to build from the API without a context, I get an error like:

missing provenance for fuqamtlkfg8ef41sv5ivh9bn6

Repro steps

I wrote a little tool for debugging the buildkit api.

1. Create a Dockerfile like this:

FROM alpine

2. Checkout my buildkitapi debugger

git clone [email protected]:tilt-dev/tilt
cd tilt
go install ./cmd/buildkitapi

there's nothing special in this script, it uses the docker client libraries to talk to the ImageBuild api in docker engine

https://github.com/tilt-dev/tilt/blob/master/cmd/buildkitapi/main.go

3. In the directory with your dockerfile, run:

$ buildkitapi --context missing-dir --legacy
{"errorDetail":{"message":"missing provenance for l4rd8x27he7k4hcsmi9gq40w3"},"error":"missing provenance for l4rd8x27he7k4hcsmi9gq40w3"}

Expected result

No error

Additional info

This is a recent regression that started happening with Docker Engine 24.0 / Docker Desktop 4.20

Boiled down from tilt-dev/tilt#6125

[jedevc]

So I can reproduce this 🎉

This is a bit fiddly. Here's my guess as to what's happening:

• The context directory doesn't exist. If we try and access it, bad things happen.
• On build, we try to sync over the .dockerignore file. This adds it to the provenance bridge. However, we silently drop failures to read the .dockerignore file.
• When generating provenance, we try and read from the provenance bridge... but since we hit an error, we never managed to get the provenance recorded (even though the result is there).

I think the provenance code expects frontend calls to solve to be propagated, instead of silently being discarded (as we do for the dockerignore). Not quite sure what we expect to do here.

I wonder if we just should skip this provenance case? And not skip past when we're recorded an error?

Or maybe we should propagate certain types of dockerignore read failures, instead of the broad err == nil?

buildkit/frontend/dockerui/config.go

Lines 375 to 380 in 40c5efa

	dt, err = ref.ReadFile(ctx, client.ReadRequest{
	Filename: bctx.filename + ".dockerignore",
	})
	if err == nil {
	bc.dockerignore = dt
	}

[jedevc]

To add on - I think the correct call is that the client should be checking that the directory needs to exist - but also, there are plenty of edge-cases where you could imagine a directory being deleted halfway through a build or similar, so we still need to fix this.

[jedevc]

Managed to write a failing test case: b35f95a. Since the context isn't accessed in the Dockerfile, then the build shouldn't fail.