Heap out of memory in 5.3.1-rc but not 5.3.0-beta

[JoshuaKGoldberg]

🔎 Search Terms

heap out of memory ineffective mark-compacts near heap limit Allocation failed

🕗 Version & Regression Information

• This is a crash
• This changed between versions ______ and _______
• This changed in commit or PR _______
• This is the behavior in every version I tried, and I reviewed the FAQ for entries about _________
• I was unable to test this on prior versions because _______

⏯ Playground Link

https://github.com/typescript-eslint/typescript-eslint/tree/281ce307cf92dbaf69ab6aba1c942ef5d342278c

💻 Code

git clone https://github.com/JoshuaKGoldberg/typescript-eslint
cd typescript-eslint
git checkout 281ce307cf92dbaf69ab6aba1c942ef5d342278c
yarn # make sure this is yarn berry, not v1 / classic

🙁 Actual behavior

In packages/utils, tsc -b tsconfig.build.json (yarn build) should run successfully.

🙂 Expected behavior

$ yarn build
yarn run v1.22.19
$ tsc -b tsconfig.build.json

<--- Last few GCs --->

[68654:0x158008000]    46789 ms: Mark-Compact 4076.4 (4143.2) -> 4075.2 (4143.2) MB, 1144.83 / 0.00 ms  (average mu = 0.121, current mu = 0.000) allocation failure; scavenge might not succeed
[68654:0x158008000]    48030 ms: Mark-Compact 4076.4 (4143.2) -> 4075.3 (4143.2) MB, 1240.50 / 0.00 ms  (average mu = 0.064, current mu = 0.000) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x102c3e14c node::Abort() [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
 2: 0x102c3e334 node::ModifyCodeGenerationFromStrings(v8::Local<v8::Context>, v8::Local<v8::Value>, bool) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
 3: 0x102dbb4f4 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
 4: 0x102f8fb9c v8::internal::Heap::GarbageCollectionReasonToString(v8::internal::GarbageCollectionReason) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
 5: 0x102f93a50 v8::internal::Heap::CollectGarbageShared(v8::internal::LocalHeap*, v8::internal::GarbageCollectionReason) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
 6: 0x102f904b4 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::internal::GarbageCollectionReason, char const*) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
 7: 0x102f8e23c v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
 8: 0x102f84e90 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
 9: 0x102f856f0 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
10: 0x102f6a6f4 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
11: 0x103351fe0 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
12: 0x1036b0c44 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
13: 0x108c22f00 
14: 0x1089cbed0 
15: 0x1036c8b3c Builtins_ArrayForEach [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
16: 0x108c124d8 
17: 0x108c24080 
18: 0x108c0ca74 
19: 0x10896bde4 
20: 0x108bba9d0 
21: 0x108c0b1dc 
22: 0x10896bde4 
23: 0x108bba9d0 
24: 0x108c0c6fc 
25: 0x10896bde4 
26: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
27: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
28: 0x108c12828 
29: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
30: 0x108c0ca74 
31: 0x10896bde4 
32: 0x108b78acc 
33: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
34: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
35: 0x108c12828 
36: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
37: 0x108c0ca74 
38: 0x10896bde4 
39: 0x1089b70fc 
40: 0x1089b7520 
41: 0x1089b7814 
42: 0x108bbb4d0 
43: 0x108ad4a78 
44: 0x108acee88 
45: 0x108aaf64c 
46: 0x108ab2c70 
47: 0x108ba82a4 
48: 0x1089b4e60 
49: 0x108af3e88 
50: 0x108aaf64c 
51: 0x108ab2c70 
52: 0x108bac2f4 
53: 0x108ab73c8 
54: 0x108aeb2c8 
55: 0x1089a7958 
56: 0x108ba6b40 
57: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
58: 0x108ae714c 
59: 0x108afbcd0 
60: 0x108977fec 
61: 0x10897820c 
62: 0x10853663c 
63: 0x108c13788 
64: 0x108bb00f4 
65: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
66: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
67: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
68: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
69: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
70: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
71: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
72: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
73: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
74: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
75: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
76: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
77: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
78: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
79: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
80: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
81: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
82: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
83: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
84: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
85: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
86: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
87: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
88: 0x1036283e4 Builtins_InterpreterEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
89: 0x10362650c Builtins_JSEntryTrampoline [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
90: 0x1036261f4 Builtins_JSEntry [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
91: 0x102efd14c v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
92: 0x102efc598 v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
93: 0x102dd6e80 v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
94: 0x102c23088 node::builtins::BuiltinLoader::CompileAndCall(v8::Local<v8::Context>, char const*, node::Realm*) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
95: 0x102cabb7c node::Realm::ExecuteBootstrapper(char const*) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
96: 0x102c08004 node::StartExecution(node::Environment*, std::__1::function<v8::MaybeLocal<v8::Value> (node::StartExecutionCallbackInfo const&)>) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
97: 0x102b794a4 node::LoadEnvironment(node::Environment*, std::__1::function<v8::MaybeLocal<v8::Value> (node::StartExecutionCallbackInfo const&)>) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
98: 0x102c7c0f4 node::NodeMainInstance::Run() [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
99: 0x102c0a44c node::LoadSnapshotDataAndRun(node::SnapshotData const**, node::InitializationResultImpl const*) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
100: 0x102c0a76c node::Start(int, char**) [/Users/josh/.nvm/versions/node/v20.2.0/bin/node]
101: 0x184ad3e50 start [/usr/lib/dyld]
error Command failed with signal "SIGABRT".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Additional information about the issue

Building worked in the previous commits on 5.3.0-beta. I also tried reverting the runtime changes to createIsolatedProgram.ts and estree-to-ts-node-types.ts from ac4beddeda21a1a15e228471e1e293c270b25cff but that didn't seem to fix the build locally.

[jakebailey]

Is there any chance you could bisect? Maybe with https://www.npmjs.com/package/every-ts?

[JoshuaKGoldberg]

Didn't get time yet to try the per-commit bisecting with every-ts, but some quick stabs at replacing package.json and some files internally:

5.3.0-dev.20231027 - broken

5.3.0-dev.20231026 - broken

5.3.0-dev.20231025 - good

5.3.0-dev.20231020 - good

[JoshuaKGoldberg]

Ok I tried out every-ts and ... I think it got stuck?

joshgoldberg ~/repos/typescript-eslint $ every-ts bisect good
aa9b6953441b53f8b14072c047f0519b611150c4 is the first bad commit
commit aa9b6953441b53f8b14072c047f0519b611150c4
Author: TypeScript Bot <[email protected]>
Date:   Thu Oct 26 06:24:18 2023 +0000

    Update package-lock.json

 package-lock.json | 40 ++++++++++++++++++++--------------------
 1 file changed, 20 insertions(+), 20 deletions(-)
joshgoldberg ~/repos/typescript-eslint $ every-ts bisect good
aa9b6953441b53f8b14072c047f0519b611150c4 is the first bad commit
commit aa9b6953441b53f8b14072c047f0519b611150c4
Author: TypeScript Bot <[email protected]>
Date:   Thu Oct 26 06:24:18 2023 +0000

    Update package-lock.json

 package-lock.json | 40 ++++++++++++++++++++--------------------
 1 file changed, 20 insertions(+), 20 deletions(-)
joshgoldberg ~/repos/typescript-eslint $ 

Commit I'm running from: JoshuaKGoldberg/typescript-eslint@cd4eb1f

Full list of commands I ran:

every-ts bisect start
every-ts bisect good 5.3.0-dev.20231025
every-ts bisect bad 5.3.0-dev.20231026
yarn
yarn build
every-ts bisect good
yarn
yarn build
every-ts bisect good
yarn
yarn build
every-ts bisect good
every-ts bisect good
every-ts bisect good

[jakebailey]

It didn't get stuck; bisecting ended at a package update commit.

If that's all of the commands you ran, nothing in that actually links the package into your workspace. You should consider running every-ts tsc -b tsconfig.build.json (or use the subcommand to get a dir you can link). Otherwise you're not actually running the code it's building (hence, repeated "good" until a random commit).

[jakebailey]

$ cd packages/utils
$ every-ts bisect start 5.2 main
$ every-ts bisect run timeout 20 tsc -b tsconfig.build.json -f
...
55395f9e01e0b2ff4152981a52aa76487df17605 is the first bad commit
commit 55395f9e01e0b2ff4152981a52aa76487df17605
Author: Sheetal Nandi <[email protected]>
Date:   Wed Oct 25 15:05:47 2023 -0700

    When caching type references cache the tracked symbols so they can be emitted in the d.ts file (#51721)

 src/compiler/checker.ts                            |  25 +-
 src/compiler/types.ts                              |   3 +
 src/testRunner/unittests/tsc/incremental.ts        |  51 +++
 .../incremental/generates-typerefs-correctly.js    | 371 +++++++++++++++++++++
 4 files changed, 449 insertions(+), 1 deletion(-)
 create mode 100644 tests/baselines/reference/tsc/incremental/generates-typerefs-correctly.js

Bisect points to #51721. @sheetalkamat