Skip to content

Commit

Permalink
closes #164
Browse files Browse the repository at this point in the history
Collection error incorrectly deleted unstable tests completely, so they were not executed on next runs at all.
  • Loading branch information
tarpas committed Apr 28, 2021
1 parent 5ef40c4 commit f08e250
Show file tree
Hide file tree
Showing 3 changed files with 45 additions and 2 deletions.
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
name="pytest-testmon",
description="selects tests affected by changed files and methods",
long_description=long_description,
version="1.1.0",
version="1.1.1",
license="AGPL",
platforms=["linux", "osx", "win32"],
packages=[
Expand Down
43 changes: 43 additions & 0 deletions test/test_testmon.py
Original file line number Diff line number Diff line change
Expand Up @@ -1270,6 +1270,49 @@ def test_2():
)


class TestPytestCollectionPhase:
def test_sync_after_collectionerror(self, testdir):
testdir.makepyfile(
test_a="""
def test_0():
pass
def test_2():
pass
"""
)
testdir.runpytest_inprocess(
"--testmon",
)
testdir.makepyfile(
test_a="""
def test_0():
pass
def test_2():
try:
"""
)
testdir.runpytest_inprocess(
"--testmon",
)

This comment has been minimized.

Copy link
@blueyed

blueyed Apr 28, 2021

Contributor

Would be good to have some basic assertion on the result, i.e. that it really triggered a collection error.

This comment has been minimized.

Copy link
@tarpas

tarpas Apr 30, 2021

Author Owner

Could you elaborate? Was it difficult to spot that test_2 is broken (and will definitely lead to collection error)? I would rather solve that with a comment.

This comment has been minimized.

Copy link
@tarpas

tarpas Apr 30, 2021

Author Owner

The test is not supposed to test if a broken py file leads to collection error in pytest so I think such an assertion would be a distraction.

This comment has been minimized.

Copy link
@blueyed

blueyed Apr 30, 2021

Contributor

@tarpas
I've meant that the fix is specifically for handling the case of a collection error in pytest.
If it for example would cause another error there already earlier (without triggering the hooks that you are expecting / that lead to the bug) you might be testing something else.
Or if the broken file would not be collected for some reason / another bug, etc...

The test is still good to have, but should get hardened from what I can tell.

testdir.makepyfile(
test_a="""
def test_0():
pass
def test_2():
print(1)
"""
)
result = testdir.runpytest_inprocess("--testmon", "-v")
result.stdout.fnmatch_lines(
[
"*test_2 PASSED*",
]
)


class TestmonCollect:
def test_failed_setup_phase(self, testdir):
testdir.makepyfile(
Expand Down
2 changes: 1 addition & 1 deletion testmon/pytest_testmon.py
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ def pytest_pycollect_makeitem(self, collector, name, obj):

def pytest_collection_modifyitems(self, session, config, items):
_, should_collect, should_select = config.testmon_config
if should_select or should_collect:
if should_collect and not session.testsfailed:
config.testmon_data.sync_db_fs_nodes(retain=set(self.raw_nodeids))

@pytest.hookimpl(hookwrapper=True)
Expand Down

0 comments on commit f08e250

Please sign in to comment.