Remove vendor directory.

[wlynch]

Changes

This was something previously brought up in #1607, but was deferred to
allow for continued discussion.

Pros of making this change:

• Reduces size of incoming PRs when new dependencies are used.
• Removes confusion around source of truth for dependencies (e.g. vendor
  vs mod). This was an issue that Use vendored version of plumbing instead of go get 🦸 #1763 solved by pinning to vendor, this
  will solve the same problem differently by always deferring to the
  version in go.mod.
• Allows easier use of upstream and dependent types without the use of vendor tools - see https://gist.github.com/wlynch/325293bc3fbc488d3b3d319f3a93bbea for an
  example of why this is difficult today.

Risks of making this change:

• Dependencies will not be present in initial clone. They will be
  fetched dynamicly when needed, which may cause workflow distruptions.
• This breaks compatibility with older versions of Go (though for
  certain projects like triggers, we already require Go >= 1.13).

Third-party obligations (for things like making source available in
images) have already been addressed in #1607.

As with #1607, this change is more easily viewed by running git diff master -- ':!third_party' ':!vendor'

This should have no functional change for users (and ideally, even devs).

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

• Includes tests (if functionality changed/added)
• [n/a] Includes docs (if user facing)
• Commit messages follow commit message best practices

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

• If you are adding a new binary/image to the cmd dir, please update
  the release Task to build and release this image.

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

[tekton-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign abayer
You can assign the PR to them by writing /assign @abayer in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[wlynch]

/cc @bobcatfish @imjasonh @vdemeester

[wlynch]

/hold

[tekton-robot]

@wlynch: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-tekton-pipeline-integration-tests	93bf39b	link	/test pull-tekton-pipeline-integration-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[vdemeester]

/hold

Pros of making this change:

* Reduces size of incoming PRs when new dependencies are used.

* Removes confusion around source of truth for dependencies (e.g. vendor
  vs mod). This was an issue that #1763 solved by pinning to vendor, this
  will solve the same problem differently by always deferring to the
  version in go.mod

Yes, this is kinda of a problem with go.mod. With vendor we should use -mod=vendor on the CI to make sure we are using it as soruce of truth.

* Allows easier use of upstream and dependent types without the use of
  vendor - see
  https://gist.github.com/wlynch/325293bc3fbc488d3b3d319f3a93bbea for an
  example of why this is difficult today.

What was the setup for this ^^. go env, …

Risks of making this change:

* Dependencies will not be present in initial clone. They will be
  fetched dynamicly when needed, which may cause workflow distruptions.

* This breaks compatibility with older versions of Go (though for
  certain projects like triggers, we already require Go >= 1.13).

This is already the case with pipeline, we require go 1.13 too (the error wrapping thing %w is used in the codebase).

There is additional problems, although more on go modules themselves that vendor or not:

• Some tooling and "downstream" integration require vendor (or at least, we do at Red Hat).
• go.mod is kinda unpredictable in what it gives, depending on the go version. This is a problem today with the vendor folder (that can change depending on who did what with which go version). This is something that worries me a bit.

As an example on this "unpredictability", cleaning my go setup (pkg, cache, …), I, somehow can't re-run successfully ./hack/update-codegen.sh

~/s/g/t/cli fix-thirdparty !102 ⛄ λ ./hack/update-deps.sh
Error: cannot register licenses from archive: open /home/vincent/pkg/mod/github.com/google/[email protected]/licenses/licenses.db: no such file or directory
Usage:
  licenses save <package> [flags]Flags:
      --force              Delete the destination directory if it already exists.
  -h, --help               help for save
      --save_path string   Directory into which files should be saved that are required by license termsGlobal Flags:
      --alsologtostderr                  log to standard error as well as files
      --confidence_threshold float       Minimum confidence required in order to positively identify a license. (default 0.9)
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --logtostderr                      log to standard error instead of files
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          log level for V logs
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered loggingF1223 15:11:26.148430  531560 main.go:43] cannot register licenses from archive: open /home/vincent/pkg/mod/github.com/google/[email protected]/licenses/licenses.db: no such file or directory

This worries me as, at least with a vendor folder, I have the guarantee it will work (as the code to compile it, …) is there, on the working directory.

Also go.mod and "tooling" acts in a really weird way, or more accurately, right now, our CI is a bit out-of-sync. We are using a tools.go to depend to tooling, which means few things

1. tooling dependencies become the project dependency, leading to, sometimes, dependency problems/conflicts…
2. depending on if you do a go install from the project or not, you will end up with a different version of the tool. This is true for the CI, where we install the tooling while creating the image it will run into. This has the advantage of not getting/installing it each time, but the disatvantage of having a version of the tooling that has nothing to do with what is present in the go.mod. This is probably what causes the error above.

[wlynch]

* Allows easier use of upstream and dependent types without the use of
  vendor - see
  https://gist.github.com/wlynch/325293bc3fbc488d3b3d319f3a93bbea for an
  example of why this is difficult today.

What was the setup for this ^^. go env, …

This is the same main.go that you see, with no other files. Notably this would work if you created a go.mod or used dep, since the go tool would know how to handle and override these. This is only a minor win, but would help dependencies get up and running faster since if pipelines depended on the scm type directly tools could theoretically use it without go mod.

[wlynch]

• go.mod is kinda unpredictable in what it gives, depending on the go version. This is a problem today with the vendor folder (that can change depending on who did what with which go version). This is something that worries me a bit.

This is only for the initial version set though, right? This shouldn't change as long as you don't try and explicitly update (e.g. go get -u) a dependency.

[vdemeester]

This is only for the initial version set though, right? This shouldn't change as long as you don't try and explicitly update (e.g. go get -u) a dependency.

Well that’s my understanding. But depending on your go version, it doesn’t get the same deps, and.. on minor version like 1.13.1 vs 1.13.5 🤓

[bobcatfish]

I'm thinking we close this for now until we make decisions around the vendor dir?

[jerop]

As an example on this "unpredictability", cleaning my go setup (pkg, cache, …), I, somehow can't re-run successfully ./hack/update-codegen.sh

~/s/g/t/cli fix-thirdparty !102 ⛄ λ ./hack/update-deps.sh
Error: cannot register licenses from archive: open /home/vincent/pkg/mod/github.com/google/[email protected]/licenses/licenses.db: no such file or directory
Usage:
  licenses save <package> [flags]Flags:
      --force              Delete the destination directory if it already exists.
  -h, --help               help for save
      --save_path string   Directory into which files should be saved that are required by license termsGlobal Flags:
      --alsologtostderr                  log to standard error as well as files
      --confidence_threshold float       Minimum confidence required in order to positively identify a license. (default 0.9)
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --logtostderr                      log to standard error instead of files
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          log level for V logs
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered loggingF1223 15:11:26.148430  531560 main.go:43] cannot register licenses from archive: open /home/vincent/pkg/mod/github.com/google/[email protected]/licenses/licenses.db: no such file or directory

This worries me as, at least with a vendor folder, I have the guarantee it will work (as the code to compile it, …) is there, on the working directory.

@vdemeester I'm running into the same issue, do you remember how you solved it? Asking 2+ years later 😅

[vdemeester]

For what is worth, it seems it's fixed by installing go-licences outside of the project folder (so that it doesn't use go.mod, etc.).