General problem with pytest_ignore_collect

[blueyed]

pytest_ignore_collect is used to deselect whole files, but that causes problems currently when -k or -x is used: testmon considers the file as stable later, and would e.g. not run remaining tests that were not run before when using -x, or deselected using -k.

I think files should only be considered stable/updated when no arguments were used maybe?

Given its problems I'd rather not have this feature on by default, i.e. pytest would still visit all files, and only later testmon deselects items.
While it is a nice benefit for projects with many files, it only saves some seconds after all, while causing lots of (annoying) issues.

[blueyed]

I suggest adding an option "testmon_skip_file_collection", defaulting to "auto", which would skip it if -k, -m, or maybe just anything is in config._origargs.
See #136.

[tarpas]

Ideally testmon would generate the test items from cache instead of importing the python files. I don't use -x or -k very often, and I appreciate the speed that testnom pytest_ingore_collect brings so I'm not sure about your suggestion.

@blueyed wouldn't you take stab at generating the test items, so that they can be filtered afterwards and we would have the advantage or both approaches?

[blueyed]

IMHO importing/collecting is fast enough, and certainly much less annoying than false results / testmon getting confused.

I do not plan to work on this anytime soon anyway - my current project has fast tests, and my local branch with the different PRs/fixes here works otherwise.. ;)

[tarpas]

This was one of the main goals of the rewrite. However testmon is interrupted or the tests filtered it should keep a consistent database.