feat!: update `github-releases` datasource digest computation to use git tag, and preserve existing digest semantics as separate datasource #20178

devversion · 2023-02-02T16:28:02Z

Changes

The github-releases datasource has been copied into a new datasource called github-release-attachments.
The github-releases general datasource is updated to use the underlying Git tag of a GitHub release entry for digest computation.

BREAKING CHANGE: Regex Manager configurations relying on the github-release data-source with digests will have different digest semantics. The digest will now always correspond to the underlying Git SHA of the release/version. The old behavior can be preserved by switching to the github-release-attachments datasource.

Fixes #20160. Fixes #19552.

Context

See: #20160, #19552, and #20175

Currently the github-releases datasource getDigest method is rather magical and depends/requires an existing digest and version. This makes it difficult to use in some situations where pinning is enabled but no initial digest is explicitly known. In such cases, Renovate just silently stops updating: see. #20160.

Additionally, some managers like bazel never pass a digest because they handle the digest on their own. i.e. for Bazel updates with github releases, the actual artifact checksum is used as digest and needs to use an algorithm as determined by the manager.

The current approach of getDigest (introduced with #6422) involves consulting the current digest and iterating through the releases of the currentValue to figure out which artifact is used for the digest. The artifact is then mapped back to the new release (with some trickery to also ignore version names in the filename..). This is somewhat reasonable but the data source generally does not have a clear insight into what GitHub artifact(s) are used by the manager. Or, alternatively, a manager might not even want the digest of an individual artifact. e.g. as seen in: #19550

Documentation (please check one with an [x])

I have updated the documentation, or
No documentation update is required (NOTE: to my knowledge this was never documented)

How I've tested my work (please select one)

I have verified these changes via:

Code inspection only, or
Newly added/modified unit tests, or
No unit tests but ran on a real repository, or
Both unit tests + ran on a real repository

viceice

can we do this in a less breaking way?

eg if there is no digest or the existing digest points to the git sha, use new behavior. if existing digest points to an attachment, use current behavior. fallback to new behavior if current digest can't be found.

devversion · 2023-02-02T21:50:23Z

I considered doing this, but it felt like this would potentially make things unnecessarily slow, magical and I felt like long-term this might be the best way forward. Happy to make that change if you feel like it

viceice · 2023-02-02T21:58:31Z

the problem is, we've a lot users who rely on current behavior. so if we change it without an alternative we would make them very angry about renovate.

devversion · 2023-02-02T22:11:49Z

yeah, I see that, but do we actually know that many users rely on this digesting? I haven't seen many issues for this data source, and I would have expected some given the current issues with the digesting.

My understanding is that this digest logic was just added for the custom regex manager use-case and other data sources like for Gitlab don't support it either. I don't feel strongly at all though- happy to continue supporting that mechanism

rarkins · 2023-02-03T06:17:10Z

The last time I looked at this, it seemed like a mistake we should eventually correct. I forget where/why it's being used though - can we revisit that to understand the impact?

devversion · 2023-02-09T15:47:37Z

Hi all, I've copied the existing datasource into github-release-attachments as suggested by @viceice, and then made the breaking change for digest computation in github-releases.

please let me know if you'd prefer to have this separate PRs. I've kept two commits to make review a little easier.

viceice · 2023-02-09T17:17:33Z

change to feature. please target v35 branch because of the breaking changes. will try to review later.

devversion · 2023-02-09T17:26:37Z

Thanks, done. The CI failures seem to be because 5a6a046 is not part of the v35 branch yet. Happy to cherry-pick this into here.

rarkins · 2023-02-10T14:29:15Z

v35 has now been brought up to date with main, and this branch up to date with v35

lib/modules/datasource/github-releases/index.ts

lib/util/github/tags.ts

…ebot#19942) Changes `rangeStrategy` default value from `'replace'` to `'auto'`. Changes `auto` behavior so that `update-lockfile` is preferred if the manager supports the `updateLockedDependency()` function. Closes renovatebot#19800 BREAKING CHANGE: Renovate will now default to updating locked dependency versions. To revert to previous behavior, configure rangeStrategy=replace.

…novatebot#19958) Sets new defaults: - `prConcurrentLimit`: 10 (instead of 0) - `prHourlyLimit`: 2 (instead of 0) Closes renovatebot#19800 BREAKING CHANGE: Renovate now defaults to applying hourly and concurrent PR limits. To revert to unlimited, configure them back to `0`.

Set default GOPROXY value to match `go`'s own default. Closes renovatebot#20040 BREAKING CHANGE: Renovate will now use go's default `GOPROXY` settings. To avoid using the public proxy, configure `GOPROXY=direct`.

rarkins

Sorry, requires another conflict resolution

…leases

lib/modules/datasource/github-release-attachments/index.spec.ts

devversion · 2023-02-22T13:46:39Z

Thanks for reviewing & merging. Is there any rough idea when v35 will be around (no promises needed, thx!)?

…git tag, and preserve existing digest semantics as separate datasource (#20178) Co-authored-by: RahulGautamSingh <[email protected]> Co-authored-by: Rhys Arkins <[email protected]> Co-authored-by: Michael Kriese <[email protected]> Co-authored-by: HonkingGoose <[email protected]>

…t file digest (#20178) The github-releases datasource has been copied into a new datasource called github-release-attachments. The github-releases general datasource is updated to use the underlying Git tag of a GitHub release entry for digest computation. Fixes #20160, Fixes #19552 BREAKING CHANGE: Regex Manager configurations relying on the github-release data-source with digests will have different digest semantics. The digest will now always correspond to the underlying Git SHA of the release/version. The old behavior can be preserved by switching to the github-release-attachments datasource.

Renovate v35 had a breaking change in how it computes the digest of GitHub releses. Previously, it would use the digest of the release attachements, but now it's just using a git sha as the "digest". This commit changes the data source for the Hubble artifacts to use the data source which preserves the old behavior. Ref: renovatebot/renovate#20178 Signed-off-by: Sebastian Wicki <[email protected]>

This was referenced Feb 2, 2023

Github-releases datasource should use tag digest, not file digest #19552

Closed

GitHub releases datasource prevents updates when pinning is enabled #20160

Closed

viceice reviewed Feb 2, 2023

View reviewed changes

devversion force-pushed the fix-digest-github-releases branch 2 times, most recently from 43b4f57 to 26b8b94 Compare February 9, 2023 13:46

devversion marked this pull request as ready for review February 9, 2023 15:44

devversion changed the title ~~refactor: replace github-releases datasource getDigest to consult underlying Git tag~~ refactor: update github-releases datasource digest computation to use git tag, and preserve existing digest semantics as separate datasource Feb 9, 2023

devversion requested a review from viceice February 9, 2023 15:47

devversion force-pushed the fix-digest-github-releases branch from 26b8b94 to c1e8379 Compare February 9, 2023 17:24

devversion changed the base branch from main to v35 February 9, 2023 17:24

rarkins force-pushed the v35 branch from a5577c3 to 11c0960 Compare February 10, 2023 14:28

rarkins added the breaking Breaking change, requires major version bump label Feb 11, 2023

rarkins approved these changes Feb 11, 2023

View reviewed changes

viceice reviewed Feb 13, 2023

View reviewed changes

lib/modules/datasource/github-releases/index.ts Outdated Show resolved Hide resolved

lib/util/github/tags.ts Show resolved Hide resolved

rarkins requested a review from viceice February 14, 2023 07:28

RahulGautamSingh and others added 6 commits February 17, 2023 08:16

feat(config)!: add new option constraintsFiltering (renovatebot#19992)

527f752

feat(cache): default cacheHardTtlMinutes to 24 hours (renovatebot#20079)

6e08957

feat(go)!: default GOPROXY (renovatebot#20081)

0705159

Set default GOPROXY value to match `go`'s own default. Closes renovatebot#20040 BREAKING CHANGE: Renovate will now use go's default `GOPROXY` settings. To avoid using the public proxy, configure `GOPROXY=direct`.

fix(regex): don’t escape forward slash in fileMatch (renovatebot#19314)

1283705

devversion requested a review from viceice February 20, 2023 19:41

rarkins requested changes Feb 21, 2023

View reviewed changes

Merge remote-tracking branch 'upstream/v35' into fix-digest-github-re…

b93cae4

…leases

devversion requested review from rarkins and removed request for viceice February 21, 2023 14:24

rarkins requested a review from viceice February 21, 2023 18:07

viceice reviewed Feb 22, 2023

View reviewed changes

lib/modules/datasource/github-release-attachments/index.spec.ts Outdated Show resolved Hide resolved

lib/modules/datasource/github-release-attachments/index.spec.ts Outdated Show resolved Hide resolved

cleanup code that was touched in this PR

e92d103

devversion requested review from viceice and removed request for rarkins February 22, 2023 09:34

fix lint import ordering

ea804d8

viceice approved these changes Feb 22, 2023

View reviewed changes

viceice requested a review from rarkins February 22, 2023 10:36

rarkins approved these changes Feb 22, 2023

View reviewed changes

viceice merged commit a1b2e5d into renovatebot:v35 Feb 22, 2023

gandro mentioned this pull request Mar 20, 2023

renovate: Fix Hubble release digest regex cilium/cilium#24477

Merged

github-actions bot locked as resolved and limited conversation to collaborators Mar 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat!: update `github-releases` datasource digest computation to use git tag, and preserve existing digest semantics as separate datasource #20178

feat!: update `github-releases` datasource digest computation to use git tag, and preserve existing digest semantics as separate datasource #20178

devversion commented Feb 2, 2023 •

edited

Loading

viceice left a comment

devversion commented Feb 2, 2023

viceice commented Feb 2, 2023

devversion commented Feb 2, 2023 •

edited

Loading

rarkins commented Feb 3, 2023

devversion commented Feb 9, 2023

viceice commented Feb 9, 2023

devversion commented Feb 9, 2023

rarkins commented Feb 10, 2023

rarkins left a comment

devversion commented Feb 22, 2023

feat!: update github-releases datasource digest computation to use git tag, and preserve existing digest semantics as separate datasource #20178

feat!: update github-releases datasource digest computation to use git tag, and preserve existing digest semantics as separate datasource #20178

Conversation

devversion commented Feb 2, 2023 • edited Loading

Changes

Context

Documentation (please check one with an [x])

How I've tested my work (please select one)

viceice left a comment

Choose a reason for hiding this comment

devversion commented Feb 2, 2023

viceice commented Feb 2, 2023

devversion commented Feb 2, 2023 • edited Loading

rarkins commented Feb 3, 2023

devversion commented Feb 9, 2023

viceice commented Feb 9, 2023

devversion commented Feb 9, 2023

rarkins commented Feb 10, 2023

rarkins left a comment

Choose a reason for hiding this comment

devversion commented Feb 22, 2023

feat!: update `github-releases` datasource digest computation to use git tag, and preserve existing digest semantics as separate datasource #20178

feat!: update `github-releases` datasource digest computation to use git tag, and preserve existing digest semantics as separate datasource #20178

devversion commented Feb 2, 2023 •

edited

Loading

devversion commented Feb 2, 2023 •

edited

Loading