Multiple comments on the PR - is this by design?

[grahammcculloch]

jest-coverage-report-action is generating a new comment every time the action is run (on a new push to the branch). Is this by design? Is there a way to configure it to update the existing comment? It is cluttering up our PR timelines quite a bit!

[ArtiomTr]

Hello @grahammcculloch 👋,

Action automatically updates comment, if comment "tags" match. Tag is a md5 hash of stringified options: working-directory, test-script, coverage-file and base-coverage-file. If any of these options change, the new comment will appear.

To verify, that your comment "tags" match, you can check the source of comments. At the beginning of each comment, there is a line:

                           Here is the value to verify ↓
<!-- jest coverage report action for options with hash "md5 hash" -->

Here is a screenshot how to view comment source:

Could you please check your comments tags? I need this information to verify, if it is a bug or not.

[grahammcculloch]

@ArtiomTr thanks for the reply. I checked the comments as you suggested and they do indeed all have the same md5 hash in the top comment line.

[ArtiomTr]

@grahammcculloch, thanks for verifying the bug. Last request - could you please send me your configuration (.yml file)? It would help me a lot. Thank you in advance.

[grahammcculloch]

Sure:

name: Test

on:
  push:
    branches-ignore:
      - main

env:
  CI: true

jobs:
  test:
    name: Lint, unit test and report coverage
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 0

      - uses: actions/setup-node@v2
        with:
          node-version: 16.x
      - name: Install dependencies
        run: npm i

      - name: Lint
        run: |
          npm run lint
          npm run lint:css

      - uses: ArtiomTr/[email protected]
        with:
            github-token: ${{ secrets.GITHUB_TOKEN }}
            test-script: npm run test:coverage
            skip-step: install
            annotations: none

Our repo is private so unfortunately I can't point you directly at it.

[raspo]

I have also noticed something similar happening on a project where I am getting new comments every time I push a new commit.

I created a test pr on my fork to verify this and here is what I noticed:

1. I opened the PR with a branch where tests were failing
2. The first comment on the PR appeared
3. I pushed another commit that would still break the tests
4. A second comment appeared on the PR, with a matching "tag"
5. I pushed another commit fixing tests
6. The previous comment got updated correctly
7. Every other commit I pushed continued to update that comment

---

This is just a simple test but it seems like the logic that updates the previous comment only works after the first successful report. I think that if I kept pushing test-breaking commits, I would have got more and more comments on that PR

---

Here is a screenshot to show the matching tags:

[ArtiomTr]

Hello @raspo 👋,

Thank you for investigating this problem further! I have one suspicion, what could cause the bug. I looked at PR in your fork, and saw some strange behavior:

The action attaches comment not to the PR, but for the commit. Could you please try to run the action with Debug-action value instead of v2 version? To do this, modify .github/workflows/test.yml file a bit:

name: Coverage

on:
    pull_request:
        paths-ignore:
            - 'docs/**'
    workflow_dispatch:

jobs:
    coverage:
        runs-on: ubuntu-latest
        name: Coverage report
        steps:
            - name: Checkout
              uses: actions/checkout@v2
            - name: Test coverage
              uses: ArtiomTr/jest-coverage-report-action@Debug-action # Use debug version
              with:
                  annotations: failed-tests
                  test-script: npm run test:coverage

And then, could you send a screenshot of the action console? It is located here:

Thank you in advance!

What I think, is that GitHub changed context.eventName values, and check for pull_request no longer works for that. Here is the line of code, that could cause this issue:

jest-coverage-report-action/src/run.ts

Line 24 in 9ce24ef

const isInPR = context.eventName === 'pull_request';

[raspo]

TL;DR
I don't think this is a bug. This happens when you have a workflow that triggers on push instead of pull_request:

name: My workflow

on:
  push:

jobs: [...]

---

@ArtiomTr I tried to do what you suggested and the logs show this:

Interestingly, I also tried to open another PR to try and recreate my previous scenario, but this time using Debug-action from the start... and I couldn't reproduce it: the one comment kept getting update regularly.

So I started thinking what did I do differently this time, and I remembered that when I opened my first PR on my fork, the actions didn't trigger initially because they were not enabled for the forked repo and I think I accidentally triggered one manually; this should explain why the commend was attached to the commit and not the PR.

By looking at the wokflow posted by @grahammcculloch I now realize that this is happening because he had it trigger on push instead of pull_request, which is also how I originally configured my workflow in the project I was alluding to 🤦

[ArtiomTr]

Thank you for resolving the issue!

[grahammcculloch]

Thanks @raspo! I will try changing my GitHub action to trigger on pull_request instead of push!