update-repos sets the name argument in lowercase.

[acehko]

For example, a program imports github.com/Shopify/sarama.

Gopkg.tml:

[[constraint]]
  name = "github.com/Shopify/sarama"
  version = "1.15.0"

BUILD.bazel:

load("@io_bazel_rules_go//go:def.bzl", "go_library")

go_library(
    name = "go_default_library",
    srcs = ["test.go"],
    importpath = "github.com/test/test",
    visibility = ["//visibility:public"],
    deps = [ "@com_github_Shopify_sarama//:go_default_library"],
)

WORKSPACE:

go_repository(
    name = "com_github_shopify_sarama",
    commit = "3b1b38866a79f06deddf0487d5c27ba0697ccd65",
    importpath = "github.com/Shopify/sarama",
)

Currently the update-repos command sets the name argument in lowercase.
It should be com_github_Shopify_sarama.

[jayconrod]

I read through the workspace naming guidelines again. I couldn't find anything that dictates that workspace names should be lowercase, but it certainly seems to be conventional. Gazelle has been generating lowercase names for a while, and I'm concerned that changing it could break something.

Could you try manually changing the name of the go_repository, then running gazelle update-repos again? Gazelle should respect the names that are already there.

[acehko]

Yes, manually changing the name works.

The problem is the inconsistency between running bazel run //:gazelle and gazelle update-repos.
bazel run //:gazelle generates the dependency name like this:@com_github_Shopify_sarama//:go_default_library. Uppercase letters are preserved.
While gazelle update-repos generates the repository name in lowercase: com_github_shopify_sarama.

The dependencies in the BUILD files should also be in lowercase.

[jayconrod]

I couldn't reproduce this. Could you show me a small test case where this happens using Gazelle at tip?

In a minimal repo with a file that just imports this library, if I run gazelle update-repos github.com/Shopify/sarama, then gazelle, I get this repository rule:

go_repository(
    name = "com_github_shopify_sarama",
    commit = "f7466ea37d994396e26da2a8b7ca33e6691abafd",
    importpath = "github.com/Shopify/sarama",
)

and this library rule:

go_library(
    name = "go_default_library",
    srcs = ["lib.go"],
    importpath = "example.com/repo",
    visibility = ["//visibility:public"],
    deps = ["@com_github_shopify_sarama//:go_default_library"],
)

If I capitalize Shopify in the workspace rule manually, then run gazelle again, it updates the dependency to "@com_github_Shopify_sarama//:go_default_library".

In both cases, the same code is used to generate the external repository name. However, there's some new functionality that reads repository names and import paths from WORKSPACE and uses those in dependencies. I believe that was in #107, which is not in a tagged release yet.

[acehko]

I made a test repo here: https://github.com/acehko/gazelle-test.
Commands that I run:

go get -u github.com/bazelbuild/bazel-gazelle/cmd/gazelle
dep ensure
gazelle update-repos -from_file=Gopkg.lock
gazelle -go_prefix=github.com/acehko/gazelle-test

[jayconrod]

Still works for me (mostly). Only difference was that I removed the vendor directory after running dep.

From WORKSPACE:

go_repository(
    name = "com_github_shopify_sarama",
    commit = "3b1b38866a79f06deddf0487d5c27ba0697ccd65",
    importpath = "github.com/Shopify/sarama",
)

From BUILD.bazel:

go_library(
    name = "go_default_library",
    srcs = ["main.go"],
    importpath = "github.com/acehko/gazelle-test",
    visibility = ["//visibility:private"],
    deps = ["@com_github_shopify_sarama//:go_default_library"],
)

I am seeing a related issue in the build though:

ERROR: /usr/local/google/home/jayconrod/.cache/bazel/_bazel_jayconrod/8bc7749987d449d593b5342bdc6ac4be/external/com_github_pierrec_lz4/BUILD.bazel:3:1: no such package '@com_github_pierrec_xxHash//xxHash32': The repository could not be resolved and referenced by '@com_github_pierrec_lz4//:go_default_library'.
ERROR: Analysis of target '//:go_default_library' failed; build aborted: Loading failed.

[jayconrod]

I figured out what's happening with that error message. It looks like I added the lower casing in #47 (then forgot that it changed). rules_go is still pointing to an older version of Gazelle from before this change. So go_repository will generate the old capitalized names, and any newer version of Gazelle will not.

This will be fixed when rules_go updates to a newer version of Gazelle. PR #1279 does this but has been blocked on the removal of wtool.

go_repository should really be moved to this repository or removed entirely. That version skew is causing problems.

[astratto]

Hey! I'm actually seeing this issue with bazel run :gazelle, where a dependency is changed to lowercase.

I'm using the latest versions of both rules_go and bazel-gazelle.

http_archive(
    name = "io_bazel_rules_go",
    url = "https://github.com/bazelbuild/rules_go/archive/cc0b6dd26fc79f48482ebf1792009c4b6a0d3242.tar.gz",
    strip_prefix = "rules_go-cc0b6dd26fc79f48482ebf1792009c4b6a0d3242",
    sha256 = "548c975c3281db38dacacd6cb490d2e15081607bf93b07829624b1f0fdaf3303",
)

http_archive(
    name = "bazel_gazelle",
    url = "https://github.com/bazelbuild/bazel-gazelle/archive/6f36218e9eb6c9e9b4974fe0b20d893cb0b6c53c.tar.gz",
    strip_prefix = "bazel-gazelle-6f36218e9eb6c9e9b4974fe0b20d893cb0b6c53c",
    sha256 = "7b4f3b09c160d4ee8e9d19371a7ad782c3336d9fb3576d4e9f933b0343757c26",
)

The failure is (redacted a bit):

(11:24:00) ERROR: /home/vagrant/go/src/github.com/DataDog/app/BUILD.bazel:3:1: no such package '@com_github_datadog_mylib//': The repository could not be resolved and referenced by '//app:go_default_library'
(11:24:00) ERROR: Analysis of target '//app:go_default_library' failed; build aborted: no such package '@com_github_datadog_mylib//': The repository could not be resolved

The correct path would be @com_github_DataDog_mylib and I can clearly see the diff after bumping bazel-gazelle.

+++ b/BUILD.bazel
@@ -30,8 +30,8 @@ go_library(
...
-        "@com_github_DataDog...:go_default_library",
+        "@com_github_datadog...:go_default_library",

[jayconrod]

Hmm, I can't reproduce this with the commits you mentioned. Could you maybe create a small example repo that demonstrates this?

Here are the files I'm trying with:

WORKSPACE

http_archive(
    name = "io_bazel_rules_go",
    url = "https://github.com/bazelbuild/rules_go/archive/cc0b6dd26fc79f48482ebf1792009c4b6a0d3242.tar.gz",
    strip_prefix = "rules_go-cc0b6dd26fc79f48482ebf1792009c4b6a0d3242",
    sha256 = "548c975c3281db38dacacd6cb490d2e15081607bf93b07829624b1f0fdaf3303",
)

http_archive(
    name = "bazel_gazelle",
    url = "https://github.com/bazelbuild/bazel-gazelle/archive/6f36218e9eb6c9e9b4974fe0b20d893cb0b6c53c.tar.gz",
    strip_prefix = "bazel-gazelle-6f36218e9eb6c9e9b4974fe0b20d893cb0b6c53c",
    sha256 = "7b4f3b09c160d4ee8e9d19371a7ad782c3336d9fb3576d4e9f933b0343757c26",
)

load("@io_bazel_rules_go//go:def.bzl", "go_register_toolchains", "go_repository", "go_rules_dependencies")

go_rules_dependencies()

go_register_toolchains()

load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")

gazelle_dependencies()

go_repository(
    name = "com_github_Shopify_sarama",
    commit = "bfaa60f9e0d0b6781a78ed5f01e0426615a5ea13",
    importpath = "github.com/Shopify/sarama",
)

BUILD.bazel

load("@io_bazel_rules_go//go:def.bzl", "go_library")

# gazelle:prefix example.com/repo

load("@bazel_gazelle//:def.bzl", "gazelle")

gazelle(name = "gazelle")

go_library(
    name = "go_default_library",
    srcs = ["lib.go"],
    importpath = "example.com/repo",
    visibility = ["//visibility:public"],
    deps = ["@com_github_Shopify_sarama//:go_default_library"],
)

lib.go

package lib

import _ "github.com/Shopify/sarama"

Running bazel run //:gazelle has no effect. If I change the repo name in WORKSPACE, running Gazelle again will change the deps to the new name.

[astratto]

Thanks @jayconrod for looking into this! Strange, I can reproduce it consistently internally.
I'll let you know if I manage to come up with a small test repo.

[astratto]

Here's an example that shows the issue: https://github.com/astratto/test-case-gazelle

Let me know if you need more info or if I'm doing something absolutely stupid 😄

[jayconrod]

Thanks, this makes more sense now. I think Gazelle is working as intended here, but I can see room for some improvements.

When Gazelle generates a rule for //testapp, it transforms imports in sources (in this case github.com/BurntSushi/toml) into Bazel labels. You can read more about this in the design doc.

• If there is a library in the current repository with the same importpath, that library will be used. This is why Gazelle resolved the dependency to the vendor directory before it was removed.
• Otherwise, Gazelle will blindly transform the import string into a label, according to the -external flag. By default, This will go to an external repository.
• Gazelle will figure out the part of the import path that corresponds to the repository root. In this, case it's the entire import path. This part is transformed into a repo name. If there is already a go_repository in WORKSPACE with the same importpath, the name of that repo will be used. Otherwise, the name is transformed to com_github_burntsushi_toml.
• Gazelle is not aware of repositories declared inside macros like go_deps. It's not feasible to evaluate them at run-time. So it will proceed as if those repositories don't exist.

One way to fix this is by running bazel run //:gazelle -- update-repos -from_file=Gopkg.lock. That will generate go_repository rules from Gopkg.lock and add them to WORKSPACE. It will update rules with the same import paths that are already there. This would be instead of go_deps.

I think a needed improvement here is the ability to tell Gazelle about a repository declared in a macro. I've filed #132 to track that.

[astratto]

Thanks for the exhaustive explanation Jay, this is very precious info!
I'll have to dig deeper to understand what changed in behaviour between 0.8.0 and 0.9.0+, because that code works great with the old version.

I might have misunderstood, but the complexity of instructing Gazelle about repositories declared in a macro (i.e., go_deps) simply moves the burden of manually defining directives in commands to Gazelle instead of explicitly in the WORKSPACE, right?

I mean, my current workflow with Gazelle 0.8.0 is: dep ensure -> Gopkg.lock + rules_go_dep to automatically generate a Gopkg.bzl.
In this way, you basically ignore go_repository directives and it's automagically taken care of by Gazelle. [1]
What #132 would provide is a way to say "go_deps would generate X, Y, Z, so please Gazelle take notes"?

[1] I'm over-simplifying a bit here, there's still some tweaks required to add information to the BUILD.bazel files generated, but that's a topic for another conversation :)

[jayconrod]

So basically, Gazelle needs some way to discover external repositories. We basically need a mapping between repository names and import paths. Currently, Gazelle just builds that mapping by parsing (not evaluating) WORKSPACE and looking for go_repository rules. Gazelle has no understanding of other rules, and it does not know about rules declared within macros.

What I was thinking about with #132 would be a way to write comments in WORKSPACE that informs Gazelle of other repositories. You would probably need to write these by hand, at least in cases where Gazelle's default repository name differs from other tools (i.e., capitalization).

This is only important for differing repository names now, but in the future, Gazelle will index libraries in external repositories and use that index to resolve dependencies. So it will be more important to have complete information when that's working.

[astratto]

Okay, that makes sense. Thanks!

My 0.02$: I think annotations in WORKSPACE for corner cases not detected might work nicely enough in my case.
For the time being I guess I'll try to stick with the older version then.

[neakor]

We are also seeing this issue. We generate the WORKSPACE file so our tooling currently has to perform a separate string replacement to get this right. The workflow we use is to generate a go.mod file then run bazel run //:gazelle -- update-repos -from_file=go.mod --to_macro=vendor_deps.bzl.

The example we have is the github.com/DATA-DOG/go-sqlmock package. The generated go_repository looks like

go_repository(
        name = "com_github_data_dog_go_sqlmock",
        importpath = "github.com/DATA-DOG/go-sqlmock",
        sum = "h1:Shsta01QNfFxHCfpW6YH2STWB0MudeXXEWMr20OEh60=",
        version = "v1.5.0",
    )

Notice the name field has the data_dog as lowercase. Instead it should be uppercase.

[jayconrod]

@neakor The case of generated names is intended to be lower case. At one point there was a stronger Bazel convention around that. In any case, changing it now would create inconsistencies that would pretty thoroughly break go_repository.

You can change the names manually if you prefer. Later invocations of update-repos will match based on the importpath attribute, so they'll update the renamed repositories instead of adding new ones.

Closing this issue since there's nothing left to do here.