Get serious about go module hygiene

[jchesterpivotal]

The problem

Right now I basically ignore the business of keeping in sync with the upstream code from Concourse (go-concourse, atc etc). This is in part because Concourse itself is sprawled over multiple repositories that are not really synced with each other within the repositories. The syncing is achieved as a by-product of manufacturing BOSH releases, but not applied back to the repositories in a way that go modules can easily consume.

At the moment I have a little bit of a Frankenstein's go.mod file to try and approximately pin versions. This works, but has problems:

• I'm trying to work out what to pin to at a git-commit level. I might get it wrong.
• Concourse keeps moving forward and so too might my upstream dependencies, but many users will want to scrape older versions of Concourse. What's the compatibility matrix going to be like? Or will I take the fly policy and wash my hands of it?

A solution

The Concourse team are aware of the problems of multi-repo sprawl and are working towards consolidating into a monorepo.

When that work lands and stabilises, I anticipate that all my direct dependencies on Concourse code will be safely consumable as a single version line in go.mod. I will be able to introduce pipeline jobs to monitor and update this resource to keep in sync with the upstream.

Open questions

• Do I abandon vendoring? It provides a useful guarantee of stability and reproducibility, but doesn't seem to be the "approved" way to work with go modules. It also makes the repo much heavier than it needs to be.
  • Following from which: if I drop vendoring, do I prune history or accept the cloning cost forever?

Todos

None currently; the issue is blocked on the upstream work.

[jchesterpivotal]

Looks like the answer to "Do I abandon vendoring?" was yes. Not sure about whether to bother pruning, though.

[jchesterpivotal]

• As a step to help bring this about, investigate adding go.mod files to concourse/concourse subdirectories and opening a PR.