!!!! That's a super serious issue. I checked out your branch at commit 7f51e7b5188d8b95d60ebbf9cdb8880b7da569b1 and I don't seem to be able to reproduce this, though: cargo nextest run produced:

Summary [  84.746s] 292 tests run: 281 passed, 11 failed, 0 skipped

(I had to press Ctrl-C because moon_cli::run_node_test inherits_moon_env_vars was taking a long time, over 60 seconds).

This is with cargo-nextest 0.9.45 on Linux x86_64.

What version of cargo-nextest are you using and on what platform?

I did another test run and this is what I got:

     Summary [  11.585s] 292 tests run: 283 passed, 9 failed, 0 skipped
        FAIL [   0.005s]                      moon_cli helpers::test::setup_color::forced_color::forces_over_no_color
        FAIL [   0.004s]                      moon_cli helpers::test::setup_color::forced_color::forces_via_arg
        FAIL [   0.054s]         moon_cli::docker_test prune_node::focuses_for_pnpm
        FAIL [   0.038s]           moon_cli::node_test run_script::works_with_pnpm
        FAIL [   0.125s]       moon_cli::run_node_test pnpm::can_install_a_dep
        FAIL [   0.125s]       moon_cli::run_node_test pnpm::can_run_a_deps_bin_hoisted
        FAIL [   0.096s]       moon_cli::run_node_test pnpm::can_run_a_deps_bin_isolated
        FAIL [   0.108s]       moon_cli::run_node_test pnpm::can_run_a_script
        FAIL [   0.388s]     moon_cli::run_system_test unix::handles_ls

I also tried with cargo nextest run --workspace and got a much larger number of failing tests:

https://gist.github.com/sunshowers/9149ee5b7c113ae364b68b56430cbe48

Maybe it's just that you were missing the --workspace at the end? This is a well-known issue with cargo (happens while setting default-members) and one I've increasingly become unhappy about.

Maybe --workspace is it. This would explain the increase in bugs lately :[ I was under the assumption nextest did this automatically, but if not, giant whoops on my part!

For reference, here's a PR where I've been going through and fixing then: moonrepo/moon#492

Looks like --workspace fixed it, sorry for this dumb issue :P

If possible, it may be nice to display a warning if Cargo.toml is using workspaces but --workspaces is not passed.

Yeah, you're the third person in the last 2 weeks to run into this. I'm going to repurpose this as a feature request to add that warning.

I was under the assumption nextest did this automatically, but if not, giant whoops on my part!

As a general rule, cargo nextest run is equivalent to cargo test -- nextest doesn't add any embellishments to the step where Cargo builds tests, just the run step.

Good to know. Thanks for the quick response 👍

I started writing #701 but quickly realized it's a lot of work enumerating all the different cases. If you or someone else would like to do it, please comment here or over there!