Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up parameterized tests which use Git #761

Merged
merged 25 commits into from
Jan 1, 2025
Merged

Speed up parameterized tests which use Git #761

merged 25 commits into from
Jan 1, 2025

Conversation

akaihola
Copy link
Owner

@akaihola akaihola commented Oct 20, 2024

This is done by using the new module-scoped Git repository fixture from Darkgraylib.

To get aggregate timings for parameterized tests:

pip install pytest-reportlog
pytest --report-log=pytest.log.json
python analyze_pytest_log.py
analyze_pytest_log.py
import json
from collections import defaultdict


def aggregate_test_durations(log_contents):
    """Aggregate durations of parameterized tests from pytest-reportlog output.

    Args:
        log_contents (str): The contents of the pytest-reportlog file

    Returns:
        dict: A dictionary mapping test names to their total duration

    """
    # Dictionary to store total durations for each base test name
    test_durations = defaultdict(float)

    # Process each line
    for line in log_contents.strip().split("\n"):
        try:
            report = json.loads(line)

            # Extract the base test name by removing the parameter portion
            full_nodeid = report["nodeid"]
            base_name = full_nodeid.split("[")[0]

            # Add the duration to the total for this test
            test_durations[base_name] += report["duration"]

        except (json.JSONDecodeError, KeyError, IndexError) as e:
            print(f"Error processing line: {e} in {line}")
            continue

    return dict(test_durations)

def main():
    # Sample usage
    log_contents = open("pytest.log.json").read()

    results = aggregate_test_durations(log_contents)

    # Print results in a formatted way
    print("\nAggregated test durations:")
    print("-" * 50)
    by_duration = sorted(results.items(), key=lambda x: x[1], reverse=True)
    for test_name, duration in by_duration:
        print(f"{test_name:<40} {duration:.6f} seconds")

if __name__ == "__main__":
    main()

@akaihola akaihola added enhancement New feature or request CI labels Oct 20, 2024
@akaihola akaihola added this to the Darker 3.1.0 milestone Oct 20, 2024
@akaihola akaihola self-assigned this Oct 20, 2024
@akaihola akaihola force-pushed the git-test-cache branch 2 times, most recently from 85d9279 to f49c0dc Compare November 4, 2024 20:15
@akaihola akaihola force-pushed the git-test-cache branch 2 times, most recently from 50388df to 2211f88 Compare November 9, 2024 21:00
@akaihola akaihola force-pushed the git-test-cache branch 6 times, most recently from e6e4ade to 9db0a6a Compare November 17, 2024 21:01
This is done by using the new module-scoped Git repository fixture from
Darkgraylib.
@akaihola
Copy link
Owner Author

akaihola commented Jan 1, 2025

Hi @DeinAlptraum & @clintonsteiner, Happy New Year! Are you interested to review how the speed of Darker unit tests on Windows almost doubled?

The amount of changes is quite large, and unfortunately the readability of tests suffered a bit (in my opinion), but now it's making way fewer Git subprocess calls (which are very expensive on Windows).

Copy link
Collaborator

@DeinAlptraum DeinAlptraum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy new year!
LGTM, but can't guarantee I didn't miss anything among the large number of changes

@akaihola akaihola merged commit 6f5b4cf into master Jan 1, 2025
38 checks passed
@akaihola akaihola deleted the git-test-cache branch January 1, 2025 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI enhancement New feature or request
Projects
Development

Successfully merging this pull request may close these issues.

2 participants