Why doesn't CI run on PR merged into main?

[andyfaff]

For many project PRs the most useful CI run is to run a test suite against the package as if the PR was merged into main. However, this doesn't seem to be the default clone setup for cirrus-ci runs where CIRRUS_PR is defined. For CIRRUS_PR runs the CI seems to be run against the PR branch itself. Is this correct?

For the scipy project we've overridden the clone_script:

clone_script: |
    if [ -z "$CIRRUS_PR" ]; then
      # if you're not in a PR then clone against the branch name that was pushed to.
      git clone --recursive --branch=$CIRRUS_BRANCH https://x-access-token:${CIRRUS_REPO_CLONE_TOKEN}@github.com/${CIRRUS_REPO_FULL_NAME}.git $CIRRUS_WORKING_DIR
      git reset --hard $CIRRUS_CHANGE_IN_REPO
    else
      # it's a PR so clone the main branch then merge the changes from the PR
      git clone --recursive https://x-access-token:${CIRRUS_REPO_CLONE_TOKEN}@github.com/${CIRRUS_REPO_FULL_NAME}.git $CIRRUS_WORKING_DIR
      git fetch origin pull/$CIRRUS_PR/head:pull/$CIRRUS_PR
      git merge --no-commit pull/$CIRRUS_PR
    fi

Thus if one is running CI against a PR we clone the main repo, fetch the change-set for the PR, then no-commit merge the change-set into main, then run the test suite.

• why isn't this the default clone for a CIRRUS_PR run?
• is this the best way of doing the clone?

cc @rgommers

[rptb1]

I also have this issue.

For GitHub CI, when a build is trigged for a pull request, the build is run on a trial merge to the destination. See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request .

For Travis CI, the same thing happens. See https://docs.travis-ci.com/user/pull-requests/ :

Travis CI builds a pull request when it is first opened, and whenever commits are added to the pull request. Rather than build the commits that have been pushed to the branch the pull request is from, we build the merge between the source branch and the upstream branch.

This results in two results appearing in the pull request "checks" section for each GitHub CI and Travis CI event. For example:

Cirrus CI only produces one result: for the unmerged branch.

The trial merge can be fetched from GitHub at refs/pull/:prNumber/merge.

Without this, we haven't seen the results of a merge before we commit to master, and have to add extra steps to our merging procedure.

[rptb1]

I also have this issue.

Current workaround https://github.com/Ravenbrook/mps/blob/0442b6089ddd0aa33d5bb0de3f620a958d4b33c6/.cirrus.yml#L23-L37 . Work in progress.

# Override clone script to get the *trial merge* of a pull request
# with its base (usually master) rather than the branch.  See
# <https://cirrus-ci.org/guide/tips-and-tricks/#custom-clone-command>.
# See also
# <https://github.com/cirruslabs/cirrus-ci-docs/discussions/1110#discussion-4613478>.

clone_script_template: &CLONE_SCRIPT_TEMPLATE
  clone_script: |
    set -x
    git clone --depth=1 --branch=$CIRRUS_BRANCH https://x-access-token:${CIRRUS_REPO_CLONE_TOKEN}@github.com/${CIRRUS_REPO_FULL_NAME}.git $CIRRUS_WORKING_DIR
    if test -n "$CIRRUS_PR"; then
      cd $CIRRUS_WORKING_DIR &&
      git fetch --depth=1 origin pull/$CIRRUS_PR/merge &&
      git checkout FETCH_HEAD
    fi

I'm not recommending this. You still only get one build. And it feels pretty hacky to override the clone like this. For example, it causes Cirrus CI to report the wrong hash and branch for the build on pages like https://cirrus-ci.com/build/5385021104586752 .

I feel like this is solving the problem at the wrong level. It means more scripts for everyone to maintain, which isn't great.

[rptb1]

For GitHub CI, when a build is trigged for a pull request, the build is run on a trial merge to the destination. See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request .

Note that this says:

GITHUB_SHA for this event is the last merge commit of the pull request merge branch.

Whereas https://cirrus-ci.org/guide/writing-tasks/#environment-variables just says:

CIRRUS_CHANGE_IN_REPO | Git SHA

but doesn't explicitly say what it's the SHA of exactly. That documentation could be improved, at least.

[rptb1]

Also relevant is the brand new Merge Queue facility from GitHub, which changes things even more. See https://github.blog/changelog/2023-02-08-pull-request-merge-queue-public-beta/