Add global.gc() afterAll and increase max-old-space-size #895
Conversation
| } = models; | ||
| const { SlackUser, GitHubUser, Installation } = models; | ||
|
|
||
| const jestTimeout = 10000; |
There was a problem hiding this comment.
What is the default value usually?
There was a problem hiding this comment.
5 seconds is the jest default. This PR doesn't change the value of our tests.
I just extracted the value from the tests into a constant.
| "test": "jest --coverage --runInBand --forceExit", | ||
| "test:watch": "jest --runInBand --forceExit --watch --notify", | ||
| "test": "node --expose-gc --max-old-space-size=4096 ./node_modules/.bin/jest --coverage --runInBand --logHeapUsage --forceExit", | ||
| "test:watch": "node --expose-gc --max-old-space-size=4096 ./node_modules/.bin/jest --runInBand --logHeapUsage --forceExit --watch --notify", |
There was a problem hiding this comment.
I believe node_modules/.bin are automatically part of the $PATH thanks to some node/package.json magic, so we might be able to drop that? Buuut I have never done anything this low level with node, so it's very possible that I'm very wrong here 😅
There was a problem hiding this comment.
You are correct about the PATH, but we don't want to invoke jest, because that would use it's 'shebang' line which is running node without any flags. So that is, what we used before.
The purpose of this change is to directly call the node executable and point it to the script it should run. It this case jest. So the PATH does only matter for the first command in the line (which is node). Hope that makes sense.
Jest now logs memory consumption, since we often run into GC pauses while executing tests. As a result they are unnecessarily slow and more unreliable. Tests are now run with `-expose-gc`, which allows us to call `global.gc()` between runs (beforeAll), which should make the memory consumption in `watch` mode more stable.
dennissivia
force-pushed
the
improve-gc-insights-for-tests
branch
from
a0d6923 to
dc4290f
Compare
While running tests locally I repeatedly ran into memory allocation errors:
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory.I enabled
logHeapUsageand noticed, that even test failures related totimeoutsand missing mock requests were GC related.There are open issues regarding jest like this, and when running jest in
watchmode, that total memory keeps growing. On my system, it consumed more than 4GB after 4-5 full test runs.In order to address these issues, I added the following changes to the test command invocation:
expose-gcto allow us manually runningglobal.gc()afterAlllogHeapUsagefor manual inspectionmax-old-space-size=4096to increase the old-gen size to 4 Gig instead of 1,5 Gig. This should make testmore reliable with regards to 'out of memory' and 'timeout' errors.jest, which is good to make sure, that everybody run tests with the same jest version and does not accidentally use a globally installed version.If your laptop does not support 4GB for the GC, just add
NODE_OPTIONS=--max-old-space-size=1024beforenpm testand the value will be passed to v8.See allowed v8 flags
This should make tests more reliable and limit the memory allocation in watch mode.
Finally, I extracted the custom
jestTimeoutinto a constant instead of having a magic number in many tests.