bors-ng is being retired

[adamgreig]

See https://bors.tech/newsletter/2023/05/01/tmib-76/ for details.

We currently use bors on about 32 repositories, mainly as a convenient way to ensure the master branch always passes CI. GitHub have recently added a new merge queue feature that provides many of the same features within the GitHub UI, which is why bors-ng is being retired.

We discussed this in today's meeting and a few options were suggested:

• Do nothing for now, wait until bors-ng actually gets shut down
• Test out GH's merge queue on one or two repos first, to help inform a decision
• See if we can use rust-lang's hosted Homu instance, though noted that it currently only runs on 15 repos so we'd be tripling the number of repos it manages and we probably don't need all its features for most of our repos
• Host bors/homu ourselves

From feedback in the meeting it sounds like migrating to GHMQ is relatively straightforward (certainly easier than our previous move from Travis CI to GHA for CI), but features we'd particularly miss include bors try to run CI on a merge commit, and bors d+ to delegate merge permission to the PR author.

I think it's probably worth trying out GHMQ but I'm not against someone asking the rust-lang infra team what they think of us using homu too. We are already set up as rust-lang teams so the permissions should be easy to configure, at least.

[reitermarkus]

features we'd particularly miss include bors try

I don't think there is much to miss about bors try. CI should just run on every commit, GitHub Actions has a concurrency option to allow cancelling CI for previous commits (in case of long-running CI matrices).

and bors d+

You can have somewhat similar behaviour if you don't dismiss approvals when new commits are pushed. That would be the same as doing bors d+ on every approved PR though and probably not an option in combination with auto-merge.

[adamgreig]

I don't think there is much to miss about bors try. CI should just run on every commit, GitHub Actions has a concurrency option to allow cancelling CI for previous commits (in case of long-running CI matrices).

The slight difference is that bors try runs CI on the merge commit to master, rather than on the PR branch commit. Still, I don't think this is usually an issue.

[kevswims]

The slight difference is that bors try runs CI on the merge commit to master, rather than on the PR branch commit. Still, I don't think this is usually an issue.

I believe GH Actions supports running CI on the resultant merge commit. See the original issue for it here: actions/checkout#15 The one limitation that I think exists is that it won't force a re-run if the target branch changes.

Note that I haven't used this feature since all the GH Actions I typically work on at day job enforce fast-forward merges only which removes this risk.

[reitermarkus]

The slight difference is that bors try runs CI on the merge commit to master

That's also the case when using the merge queue feature, which will catch this when actually trying to merge it, right?

[Emilgardis]

One thing when migrating I just encountered, bors interprets a skipped required check as failed, merge queue sees it as a success.

We aggregate the bunch of jobs needed into one check for bors to check, and if a need fails that check will be marked as skipped, hence merge queue will complete.

The way to fix this is to do

    if: always()
    steps:
      - run: jq --exit-status 'all(.result == "success")' <<< '${{ toJson(needs) }}'

to check that all needs are success

made a note here

[adamgreig]

Thanks for catching that, good point. Hopefully there ends up being a neater way than the jq snippet though...

I've noticed another difference: currently we usually configure bors to not allow merging if certain labels (like needs-decision) are present, but afaict there's no way to do anything similar with GH branch protection or merge queues. The labels haven't been that important to us recently really so it's probably not a huge concern.

Edit: we were using bors to delete branches once they're merged, but GitHub now has a setting for this too.

[Urhengulas]

Hopefully there ends up being a neater way than the jq snippet though...

To add perspective: For the knurling-rs repositories we also did aggregate the multiple jobs into one for bors. But now we removed this check and just mark all checks as required in the branch protection rules.

[romancardenas]

Did you have to check them one by one? I haven't found a "check all" button. Therefore, the "all in one" job is still handy.

Also, do you have optional checks? For instance, nightly builds or nightly clippy. For riscv, those checks are now allowed to fail.

[Urhengulas]

Did you have to check them one by one? I haven't found a "check all" button. Therefore, the "all in one" job is still handy.

That is true. I needed to check them individually, but in our case this isn't too much.

Also, do you have optional checks? For instance, nightly builds or nightly clippy. For riscv, those checks are now allowed to fail.

We don't. Are you saying that you decided that it is okay if they fail or that through the switch to the merge queue they are no allowed to fail?

[romancardenas]

Maybe I'm wrong, but what I understand from here is that the matrix marks nightly-related jobs as experimental, so even if one or more of these fail, the workflow is allowed to "succeed". However, the stable and the MRSV builds MUST succeed.

[Emilgardis]

notriddle posted some more information here for the plan

There are a few things that I'd like to see happen, which means it's not scheduled, but it is planned:

• I'd like to see GitHub's built-in Merge Queue leave beta.
• I'd like to see GHMQ become available on personal repos.
• I'd like to see some of the bigger projects that use the public bors-ng instance switch to either self-hosting or GHMQ.
• I'd like to see someone launch a fork of this project (I'm not handing control of the public instance over to anybody, though; that would be irresponsible).

I'll be waiting for most of these things to happen, but not necessarily all of them.