Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to keep files after test suite passes #500

Open
al-rigazzi opened this issue Feb 26, 2024 · 1 comment
Open

Add support to keep files after test suite passes #500

al-rigazzi opened this issue Feb 26, 2024 · 1 comment
Labels
area: test Issues related to the test suite type: feature Issues that include feature request or feature idea

Comments

@al-rigazzi
Copy link
Collaborator

al-rigazzi commented Feb 26, 2024

Description

Add the support to keep files in test_output after the execution of pytest even if the tests pass.

Justification

Currently, all files generated at test time are stored in tests/test_output/ for the duration of the test run and deleted in case all tests pass, kept if at least one test failed. In some cases, anyhow, it could be useful to keep said files even when tests pass. For example, when developing one feature and implementing the corresponding tests, it can be useful to look at output files. The current workaround is to make a test fail intentionally, but I think developers may want a programmatic way of enabling such behavior.

Implementation Strategy

I would suggest to add an(other) environment variable to control whether test files should be kept, and check its value in the following function:

def pytest_sessionfinish(
    session: pytest.Session, exitstatus: int  # pylint: disable=unused-argument
) -> None:
    """
    Called after whole test run finished, right before
    returning the exit status to the system.
    """
    if exitstatus == 0 and not os.environ.get("SMARTSIM_TEST_KEEPFILES") is not None:
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        shutil.rmtree(test_output_root)
    else:
        # kill all spawned processes in case of error
        kill_all_test_spawned_processes()

where I underlined the suggest change.

@al-rigazzi al-rigazzi added the type: feature Issues that include feature request or feature idea label Feb 26, 2024
@mellis13 mellis13 added the area: test Issues related to the test suite label Apr 5, 2024
@mellis13
Copy link
Contributor

mellis13 commented Apr 5, 2024

An alternative would be to just keep test outputs by default and consider if we should have an option to remove test output after a run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: test Issues related to the test suite type: feature Issues that include feature request or feature idea
Projects
None yet
Development

No branches or pull requests

2 participants