Error when http_archive has wrong sha256 now reports prefix not found in the zip

[alexeagle]

Maybe a caching bug?

I changed my WORKSPACE to update the version in my http_archive

http_archive(
    name = "build_bazel_rules_nodejs",
    sha256 = "e9bc013417272b17f302dc169ad597f05561bb277451f010043f4da493417607",
    strip_prefix = "rules_nodejs-0.7.0",
    url = "https://github.com/bazelbuild/rules_nodejs/archive/0.7.0.zip",
)

now I get

ERROR: error loading package '': Encountered error while reading extension file 'defs.bzl': no such package '@build_bazel_rules_nodejs//': Prefix rules_nodejs-0.7.0 was given, but not found in the zip

I tried releasing a new tag just to get around this, but it still fails.

Sorry, at ng-conf, don't have time to make a minimal repro right now. Possible this is not reproducible, will try to follow up later.

[laszlocsomor]

Please provide some repro, the repo you were working in, and the Bazel version.

[alexeagle]

I will try to repro (have seen this twice now) - one hint (maybe) is that the sha256 was also wrong. Next time I see this happen I'll try removing the sha256 attribute and see if that makes any difference.

[alexeagle]

Okay, now I understand it a bit better.

http_archive(
    name = "build_bazel_rules_nodejs",
    url = "https://github.com/bazelbuild/rules_nodejs/archive/0.7.0.zip",
    strip_prefix = "rules_nodejs-0.7.0",
    sha256 = "7406bea7954e1c906f075115dfa176551a881119f6820b126ea1eacb09f34a1a",
)

-> ERROR: error loading package '': Encountered error while reading extension file 'defs.bzl': no such package '@build_bazel_rules_nodejs//': Prefix rules_nodejs-0.7.0 was given, but not found in the zip

I expected the error message to tell me the sha256 is wrong, and what the actual one is, so I can update myself. That's what used to happen.

If I put the correct sha256sum, then it works.

[alexeagle]

and it's reproducible on linux too, so it's not platform-specific, sorry for the false lead there

[aehlig]

Looking into the issue, I'm pretty sure it is the same cache issue/user error (depending on your point of view) as #5144. A minimal example that explains what is going on is the following (on a bazel with empty repository cache). ``` WRKDIR=$(mktemp -d "${TEST_TMPDIR}/testXXXXXX") cd "${WRKDIR}" mkdir ext_repo touch ext_repo/WORKSPACE touch ext_repo/BUILD tar cvf ext.tar ext_repo rm -rf ext_repo touch empty mkdir main cd main cat > WORKSPACE <<EOF load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive", "http_file") http_archive( name = "ext", url = "file://${WRKDIR}/ext.tar", strip_prefix = "ext_repo", # actually the sha1 sum of the empty string sha256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", ) http_file( name = "empty", urls = ["file://${WRKDIR}/empty"], sha256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", ) EOF ``` Then, the first `bazel build @ext//...` fails with the expected error message `Checksum was ... but wanted ...`. Then doing a `bazel build @empty/...` succeeds (as expected) and has the side effect that bazel now knows how to obtain a file with SHA256 sum e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855. So, the next `bazel build @ext//...` will not try to download the file (we have it in cache anyway!) but instead try to unpack it---and fail with `Prefix ext_repo was given, but not found in the archive` which is the correct description for the archive with the given SHA256 sum. As mentioned in #5144, in my opinion, the long-term solution would be to make progress on the `WORKSPACE.resolved` proposal. Once hashes are no longer added by hand, the problem of manually written WORKSPACES with hashes referring to other files (e.g., earlier versions) will go away. I am not sure, we need a quick interim solution.

…

-- Klaus Aehlig Google Germany GmbH, Erika-Mann-Str. 33, 80636 Muenchen Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Geschaeftsfuehrer: Paul Terence Manicle, Halimah DeLaine Prado

[alexeagle]

Sure, I'm getting used to downloading the file and running sha256sum myself, this is low severity.

[qzmfranklin]

I am getting this same issue after upgrading to bazel 0.16

I remembered that in a previous version (possibly 0.12), it will report mismatched sha256. I liked that behavior better.

[ittaiz]

I have a feeling there always be some need for hashes to be added by hand even if solely for the use case of quickly iterating by myself (WORKSPACE.resolved IIRC is now discussed with respect to sync which is time consuming). I think the more accurate error message is indeed better.

…

On Mon, 8 Oct 2018 at 16:51 Erick J ***@***.***> wrote: "I am not sure, we need a quick interim solution." to echo @qzmfranklin <https://github.com/qzmfranklin> - reverting to the more accurate error message of a mismatching shasum would be extremely helpful — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#5045 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIF42n7lp22KwFYge7nZf8zUOU9oDXks5ui7q5gaJpZM4TaeGz> .

[aehlig]

I think the more accurate error message is indeed better.

Sure, but as explained in #6089, I don't see a way of improving the error message without degrading the non-error case by unwanted fetches.

[ittaiz]

I agree

…

On Wed, 7 Nov 2018 at 10:59 Klaus Aehlig ***@***.***> wrote: I think the more accurate error message is indeed better. Sure, but as explained in #6089 <#6089>, I don't see a way of improving the error message without degrading the non-error case by unwanted fetches. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5045 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIF7NbTRnHJJKv0pI1ZnhJKh_LIwElks5usqD_gaJpZM4TaeGz> .

[jrajahalme]

Just hit this by changing the archive I'm depending on to a newer one, but failing to update the SHA256, so in the hindsight it is obvious that the newer archive was not found from the old zip file.

@aehlig Would it be too much to ask to have some more context in the error message, e.g., "Existing zip with SHA256 ... was used but the contents does not match with the expected, maybe use a different SHA256?"

[dslomov]

Let's attempt to redownload the file and check the checksum if the prefix is not found

[aehlig]

The current state (after aff8319) is that in that case (URL and prefix changed, but hash not updated), bazel will

• report that the prefix wasn't found and show the available prefixes,
• report which files where taken from cache based on which provided hashes, and
• will fail the build.

After offline discussion with @dslomov, we decided to leave it at this state for the moment, and not attempt to proactively redownload the file.