Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🛠️ Repo: Improving test isolation in Workflow #78

Open
matiassimon opened this issue Jan 28, 2024 · 1 comment · May be fixed by #79
Open

🛠️ Repo: Improving test isolation in Workflow #78

matiassimon opened this issue Jan 28, 2024 · 1 comment · May be fixed by #79
Labels
area: repository tooling status: accepting prs Please, send a pull request to resolve this! 🙏 type: feature New feature or request

Comments

@matiassimon
Copy link

Summary:

The current workflow runs all tests together in a single job, causing two main issues:

  1. Tracking Difficulty:

    • It's challenging to distinguish which examples are functioning correctly and which aren't.
  2. Dependency Interference:

    • Shared system dependencies might lead to misleading test results; an example might pass due to a dependency installed by a different example.

Proposed Solutions:

  1. Matrix Strategy:

    • Consider replacing Lerna with a matrix strategy in the workflow, iterating over each package's folder independently.
    strategy:
      matrix:
        folder:
          - example1
          - example2
          - example3
        node-version:
          - 16
          - 18
  2. Separate Repositories:

    • Alternatively, use individual repositories for each example, each with its own workflows. Add them as submodules to a main repository.
@matiassimon matiassimon linked a pull request Jan 28, 2024 that will close this issue
@JoshuaKGoldberg JoshuaKGoldberg changed the title Improving test isolation in Workflow 🛠️ Repo: Improving test isolation in Workflow Jul 4, 2024
@JoshuaKGoldberg
Copy link
Member

Oh! Sorry for taking so long to get to this @matiassimon! We'd just started working on Mocha in January (mochajs/mocha#5027) and have mostly been focusing on the main repo. But this examples repo needs love too - and we should have responded sooner. Sorry!

+1 to this issue as described. It's annoying having them run in fail-fast series and interfering with each other.

@JoshuaKGoldberg JoshuaKGoldberg added type: feature New feature or request status: accepting prs Please, send a pull request to resolve this! 🙏 area: repository tooling labels Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: repository tooling status: accepting prs Please, send a pull request to resolve this! 🙏 type: feature New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants