Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(runner): ensure sequential suite overrides sequence.concurrent #6653

Merged
merged 4 commits into from
Oct 12, 2024

Conversation

dsyddall
Copy link
Contributor

@dsyddall dsyddall commented Oct 6, 2024

Description

Fixes #6652

Currently, when running Vitest with the --sequence.concurrent CLI option or with the sequence.concurrent config option set to true, describe.sequential has no effect and tests run concurrently.

This PR fixes the issue by ensuring that passing { concurrent: false } to the options of a suite does not only cancel the effect of passing { concurrent: true } but causes { sequential: true } to be set on the options, so that when the value of options.sequential is checked in the task function, it overrides the value specified in the runner config.

The test that originally checked for this behaviour became invalid with changes to the runner in Vitest 2.0.0, so this PR also updates the test to correctly check for the behaviour in v2.0.0 onwards.

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

@dsyddall dsyddall force-pushed the fix-override-sequence-concurrent branch from eaced80 to 6c45eaa Compare October 6, 2024 17:06
@hi-ogawa
Copy link
Contributor

hi-ogawa commented Oct 9, 2024

Thanks for the PR!
It looks like your issue went unnoticed since vi.setConfig({ sequence: { concurrent } }) is not supported properly (type is available but doc says this https://vitest.dev/api/vi.html#vi-setconfig) Can you also add a new test in test/config to cover the case like your repro?

@dsyddall
Copy link
Contributor Author

dsyddall commented Oct 9, 2024

Yes, that looks correct. It looks like this option was removed in the v1 documentation (there was no such restriction in the v0 documentation when the test was added). Per your suggestion, I'll update this PR to remove the existing test and replace it with one in test/config.

It looks like the existing test is the only usage of the sequence.concurrent config property - since the docs have specified that it shouldn't be used since v1, perhaps after replacing the existing test, we can also remove the sequence.concurrent property from the RuntimeConfig type?

@hi-ogawa
Copy link
Contributor

hi-ogawa commented Oct 9, 2024

we can also remove the sequence.concurrent property from the RuntimeConfig type?

Yeah, I think so. Can you confirm @sheremet-va?

@sheremet-va
Copy link
Member

To me, this seems like a regression. The docs started mentioning it just for clarification purposes, the behaviour of setConfig never changed, but how the config is used might've changed. I don't see why it shouldn't work in principle 🤔

@hi-ogawa
Copy link
Contributor

hi-ogawa commented Oct 10, 2024

Maybe this is because config.sequence.concurrent is picked up too early in default suite?

function createDefaultSuite(runner: VitestRunner) {
const config = runner.config.sequence
const api = config.shuffle ? suite.shuffle : suite
return api('', { concurrent: config.concurrent }, () => {})
}

I think looking up config.sequence.concurrent could be delayed until describe(...) or it(...) are called and at that point we can use config.sequence.concurrent overwritten by vi.setConfig.

Copy link

netlify bot commented Oct 10, 2024

Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit 439a2ed
🔍 Latest deploy log https://app.netlify.com/sites/vitest-dev/deploys/6709c902ea72ca0007e055a7
😎 Deploy Preview https://deploy-preview-6653--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@dsyddall dsyddall force-pushed the fix-override-sequence-concurrent branch from 0e83780 to 74c5f1e Compare October 10, 2024 13:49
@dsyddall
Copy link
Contributor Author

Okay, that makes sense - in that case, I'll leave the type as it is and I've updated the PR to replace the existing test with a new test in test/config, as suggested.

Copy link
Contributor

@hi-ogawa hi-ogawa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, nice tests!

We can investigate vi.setConfig({ sequence }) separately. I'll create a new issue later.

@hi-ogawa hi-ogawa merged commit 5e6de27 into vitest-dev:main Oct 12, 2024
17 checks passed
@dsyddall dsyddall deleted the fix-override-sequence-concurrent branch October 14, 2024 18:42
renovate bot added a commit to mmkal/eslint-plugin-mmkal that referenced this pull request Oct 14, 2024
##### [v2.1.3](https://github.com/vitest-dev/vitest/releases/tag/v2.1.3)

#####    🐞 Bug Fixes

-   Fix error diff of `toBeNaN, toBeUndefined, toBeNull, toBeTruthy, toBeFalsy`  -  by [@hi-ogawa](https://github.com/hi-ogawa) in vitest-dev/vitest#6697 [<samp>(e0027)</samp>](vitest-dev/vitest@e002758c)
-   **browser**:
    -   Provide aria role intellisense  -  by [@sheremet-va](https://github.com/sheremet-va) in vitest-dev/vitest#6629 [<samp>(f36ea)</samp>](vitest-dev/vitest@f36eac77)
    -   Not.toBeInTheDocument works with locators API  -  by [@sheremet-va](https://github.com/sheremet-va) in vitest-dev/vitest#6634 [<samp>(8bef5)</samp>](vitest-dev/vitest@8bef5d2b)
    -   Use `/mockServiceWorker.js` instead of `/__vitest_msw__`  -  by [@sheremet-va](https://github.com/sheremet-va) in vitest-dev/vitest#6687 [<samp>(4b2ce)</samp>](vitest-dev/vitest@4b2ce07f)
    -   Don't override the esbuild option in the browser mode  -  by [@sheremet-va](https://github.com/sheremet-va) in vitest-dev/vitest#6692 [<samp>(d131f)</samp>](vitest-dev/vitest@d131fd64)
-   **expect**:
    -   Preserve prototype in `toMatchObject` diff  -  by [@hi-ogawa](https://github.com/hi-ogawa) in vitest-dev/vitest#6620 [<samp>(d289e)</samp>](vitest-dev/vitest@d289e7eb)
-   **mocker**:
    -   Specify correct spy dependency  -  by [@sheremet-va](https://github.com/sheremet-va) [<samp>(7e958)</samp>](vitest-dev/vitest@7e9584aa)
-   **runner**:
    -   Ensure sequential suite overrides sequence.concurrent  -  by [@dsyddall](https://github.com/dsyddall) and [@hi-ogawa](https://github.com/hi-ogawa) in vitest-dev/vitest#6653 [<samp>(5e6de)</samp>](vitest-dev/vitest@5e6de274)
-   **vitest**:
    -   Deprecate UserConfig in favor of ViteUserConfig  -  by [@sheremet-va](https://github.com/sheremet-va) in vitest-dev/vitest#6626 [<samp>(496bd)</samp>](vitest-dev/vitest@496bd251)
    -   Don't hang with maxConcurrency 0  -  by [@sheremet-va](https://github.com/sheremet-va) in vitest-dev/vitest#6627 [<samp>(946d8)</samp>](vitest-dev/vitest@946d8bb1)
    -   Deprecate old task types and node-reliant types  -  by [@sheremet-va](https://github.com/sheremet-va) in vitest-dev/vitest#6632 [<samp>(00045)</samp>](vitest-dev/vitest@000459a9)
    -   Fix `<empty line>` logs when interleaving `console.log/error`  -  by [@hi-ogawa](https://github.com/hi-ogawa) in vitest-dev/vitest#6644 [<samp>(9ece3)</samp>](vitest-dev/vitest@9ece3952)
    -   Show rollup error details as test error  -  by [@hi-ogawa](https://github.com/hi-ogawa) in vitest-dev/vitest#6686 [<samp>(47dde)</samp>](vitest-dev/vitest@47dde76f)
    -   Always inline setup files  -  by [@sheremet-va](https://github.com/sheremet-va) in vitest-dev/vitest#6689 [<samp>(19d64)</samp>](vitest-dev/vitest@19d64e20)
    -   Use `fast-glob` instead of `tinyglobby` in Vitest  -  by [@sheremet-va](https://github.com/sheremet-va) in vitest-dev/vitest#6688 [<samp>(70baa)</samp>](vitest-dev/vitest@70baaaa9)
-   **web-worker**:
    -   Share mocker with main executor  -  by [@hi-ogawa](https://github.com/hi-ogawa) in vitest-dev/vitest#6623 [<samp>(fe7b8)</samp>](vitest-dev/vitest@fe7b8aee)

#####    🏎 Performance

-   Reuse full name in reported tasks, update generator types  -  by [@sheremet-va](https://github.com/sheremet-va) in vitest-dev/vitest#6666 [<samp>(7a0de)</samp>](vitest-dev/vitest@7a0de0a7)

#####     [View changes on GitHub](vitest-dev/vitest@v2.1.2...v2.1.3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Regression: When sequence.concurrent is true, tests in a sequential suite run concurrently
3 participants