Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Captured stdout in a subtest (pytest-subtests) is not displayed properly #750

Closed
yugokato opened this issue Oct 20, 2023 · 2 comments
Closed

Comments

@yugokato
Copy link

Captured stdout in a subtest is not displayed as expected. Tested with v3.2.0 and v4.0.2 - both have the issue in a different way. Also the issue behavior changes depending on the --capture option value. I tried fd (the default value) and tee-sys. Only v3.2.0 with --capture tee-sys seems to work as expected.

  • Test code to reproduce:
def test_something(subtests):
    print("main test")

    with subtests.test("subtest"):
        print("sub test")
  • Results:

    • v3.2.0

      1. command: pytest --html fd_3.2.0.html
        -> Captured stdout from the subtest is missing
      2. command: pytest --html tee-sys_3.2.0.html --capture tee-sys
        -> Looks good
    • v4.0.2

      1. command: pytest --html fd_4.0.2.html
        -> main test and subtest are displayed as separate tests. One has captured stdout from the main test, the other has captured stdout from the subtest
      2. command: pytest --html tee-sys_4.0.2.html --capture tee-sys
        -> main test and subtest are displayed as separate tests. One has no captured stdout, the other has captured stdout from both main test and subtest

fd_3 2 0
tee-sys_3 2 0
fd_4 0 2
tee-sys_4 0 2

@jeffwright13
Copy link
Collaborator

jeffwright13 commented Oct 22, 2023

Is this an issue with pytest-html, or with pytest in general? I haven't used pytest-subtests before, so maybe I don't know what to expect as its output, but I created a new subdirectory and new venv (3.10.5), copied your reproduction code in there into a file named test_subtests.py, and ran pytest -v -rA. I don't see any subtest output on console, only output from the main test. Should we expect to see a separate entry in the PASSES section?

(test-subtest) ~/coding/test_subtests pytest -v -rA
================================================= test session starts ==================================================
platform darwin -- Python 3.10.5, pytest-7.4.2, pluggy-1.3.0 -- /Users/jwr003/coding/test_subtests/venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/jwr003/coding/test_subtests
plugins: subtests-0.11.0
collected 1 item

test_subtests.py::test_subtests [subtest] SUBPASS                                                                [100%]
test_subtests.py::test_subtests PASSED                                                                           [100%]

======================================================== PASSES ========================================================
____________________________________________________ test_subtests _____________________________________________________
------------------------------------------------- Captured stdout call -------------------------------------------------
main test
=============================================== short test summary info ================================================
PASSED test_subtests.py::test_subtests
========================================= 1 passed, 1 subtests passed in 0.02s =========================================
(test-subtest) ~/coding/test_subtests

And indeed, when I specify the tee option, I see the subtest output, which is identical to your result, only just not using pytest-html:

(test-subtest) ~/coding/test_subtests pytest -v -rA --capture tee-sys
================================================= test session starts ==================================================
platform darwin -- Python 3.10.5, pytest-7.4.2, pluggy-1.3.0 -- /Users/jwr003/coding/test_subtests/venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/jwr003/coding/test_subtests
plugins: subtests-0.11.0
collected 1 item

test_subtests.py::test_subtests main test
sub test
[subtest] SUBPASS                                                                [100%]
test_subtests.py::test_subtests PASSED                                                                           [100%]

======================================================== PASSES ========================================================
____________________________________________________ test_subtests _____________________________________________________
------------------------------------------------- Captured stdout call -------------------------------------------------
main test
sub test
=============================================== short test summary info ================================================
PASSED test_subtests.py::test_subtests
========================================= 1 passed, 1 subtests passed in 0.04s =========================================

@yugokato
Copy link
Author

Thanks for pointing it out. I agree that the issue is in somewhere else (although there might be something to be looked at in pytest-html for how a sub test result is displayed differently between v3.2.0 v.s. v4.0.2)
I'll open an issue in pytest-subtests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants