Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create the app once per run, not once per test #3125

Merged
merged 1 commit into from
Sep 20, 2019
Merged

Create the app once per run, not once per test #3125

merged 1 commit into from
Sep 20, 2019

Conversation

quis
Copy link
Member

@quis quis commented Sep 17, 2019

Turns out our tests spent a lot of time recreating the app for each test case, which is quite intense.

This commit moves the app-creating code outside the fixture, so the fixture re-uses the same app for every test.

This cuts down the time taken to run the test suite to about 50 seconds.

It also makes the tests more parallelizable. Before this change going from 4 to 8 processes made the tests slower. Now it cuts them down from about 50 seconds to about 35 seconds1. So this commit also lets Pytest choose the best number of processes to run, since on my machine it chooses 8, which is the fastest.

Overall this means the tests take about 15% of the time they used to. Assuming each person on the teams runs the tests on average once a day, this will save 15 hours per person, per year.

Before

image

image

After

image

image


  1. Based on a 2.2GHz quad-core Intel Core i7 processor on a 2015 MacBook Pro

create_app(app)
app.test_client_class = TestClient
ctx = app.app_context()


class ElementNotFound(Exception):
pass


@pytest.fixture
Copy link
Contributor

@leohemsted leohemsted Sep 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can just change this to @pytest.fixture(scope='session') to get it to execute once, and save on module level code

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-wrote in 1a353d9

@servingUpAces
Copy link
Contributor

One small change for Admin, one giant savings for devs.
Yahoo!

Turns out our tests spent a lot of time recreating the app for each test
case, which is quite intense.

This commit makes the fixture sessions level, so the app is only created
once per test session, not once per test function.

This cuts down the time taken to run the test suite to about 50 seconds.

It also makes the tests more parallelizable. Before this change going
from 4 to 8 processes made the tests slower. Now it cuts them down from
about 50 seconds to about 35 seconds[1]. So this commit also lets Pytest
choose the best number of processes to run, since on my machine it
chooses 8, which is the fastest.

Overall this means the

1. With a 2.2GHz quad-core Intel Core i7 processor on a 2015 MacBook Pro
@quis quis merged commit bda2f41 into master Sep 20, 2019
@quis quis deleted the faster-tests branch September 20, 2019 09:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants