domain: avoid circular memory references #25993
Conversation
Add a simple `WeakReference` utility that we can use until the language provides something on its own.
Avoid circular references that the JS engine cannot see through because it involves an `async id` ⇒ `domain` link. Using weak references is not a great solution, because it increases the domain module’s dependency on internals and the added calls into C++ may affect performance, but it seems like the least bad one. Fixes: nodejs#23862
nodejs-github-bot
added
c++
|
This approach looks fine to me, given the current situation. How does the memory profile for this look like? E.g., taken the example from #23862 as-is, and hammering it with GET requests, how high would the memory usage be with and without domains usage on the server side? |
ChALkeR
added
the
memory
|
@ChALkeR What I’m getting is about 8 kB per request with that example before this patch, and flat memory usage afterwards. |
That is what I was expecting, but I was curious about the exact memory usage profile with/without domains code on the server logic side (with this patch enabled) for the same requests pattern. Anyway, this change looks good to me in approach and code-wise. I have not tested this locally (yet?), but this should be good once CI passes. |
I don’t think a cctest makes sense, but I could write a regular test if you feel that it’s important. |
If we want to reuse this is other code path I think we should cover the API. |
@refack I think we want to avoid that as much as possible. I’ve added a test, though.
@vdeturckheim No, that’s not necessarily true. The problem here is specific to domains, because they (needlessly?) keep references to objects that they track. Ideally, we’d revert this change and remove |
|
(That’s doesn’t mean that it’s not easy to run into memory leaks when using async_hooks, but that’s a fundamental design flaw and a much bigger issue.) |
| @@ -53,20 +57,22 @@ const asyncHook = createHook({ | |||
| init(asyncId, type, triggerAsyncId, resource) { | |||
| if (process.domain !== null && process.domain !== undefined) { | |||
| // If this operation is created while in a domain, let's mark it | |||
| pairing.set(asyncId, process.domain); | |||
| pairing.set(asyncId, process.domain[kWeak]); | |||
There was a problem hiding this comment.
Is it conceivable that user code tampers with process.domain?
What would happen if I set process.domain = {}?
There was a problem hiding this comment.
I don’t think the domain code is all that tampering-proof to being with…
I’m also not sure what we could do in that case.
| static void Get(const FunctionCallbackInfo<Value>& args) { | ||
| WeakReference* weak_ref = Unwrap<WeakReference>(args.Holder()); | ||
| Isolate* isolate = args.GetIsolate(); | ||
| if (!weak_ref->target_.IsEmpty()) |
There was a problem hiding this comment.
Technically not necessary: Persistent<T>::Get() will return an empty handle, causing ReturnValue<T>::Set() to default to undefined.
There was a problem hiding this comment.
Thought about it, and I think I’d prefer an explicit variant here, so that it’s a bit more obvious what happens when the object has been GC’ed…
refack
added
the
author ready
Add a simple `WeakReference` utility that we can use until the language provides something on its own. PR-URL: #25993 Fixes: #23862 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Сковорода Никита Андреевич <[email protected]> Reviewed-By: Vladimir de Turckheim <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
Avoid circular references that the JS engine cannot see through because it involves an `async id` ⇒ `domain` link. Using weak references is not a great solution, because it increases the domain module’s dependency on internals and the added calls into C++ may affect performance, but it seems like the least bad one. PR-URL: #25993 Fixes: #23862 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Сковорода Никита Андреевич <[email protected]> Reviewed-By: Vladimir de Turckheim <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
|
Is this fix planned to be back-ported to the v10 release? |
|
Any update if this will be backported to LTS? |
Add a simple `WeakReference` utility that we can use until the language provides something on its own. PR-URL: nodejs#25993 Fixes: nodejs#23862 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Сковорода Никита Андреевич <[email protected]> Reviewed-By: Vladimir de Turckheim <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
Avoid circular references that the JS engine cannot see through because it involves an `async id` ⇒ `domain` link. Using weak references is not a great solution, because it increases the domain module’s dependency on internals and the added calls into C++ may affect performance, but it seems like the least bad one. PR-URL: nodejs#25993 Fixes: nodejs#23862 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Сковорода Никита Андреевич <[email protected]> Reviewed-By: Vladimir de Turckheim <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
|
Backport opened in #27749 |
Add a simple `WeakReference` utility that we can use until the language provides something on its own. Backport-PR-URL: #27749 PR-URL: #25993 Fixes: #23862 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Сковорода Никита Андреевич <[email protected]> Reviewed-By: Vladimir de Turckheim <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
Avoid circular references that the JS engine cannot see through because it involves an `async id` ⇒ `domain` link. Using weak references is not a great solution, because it increases the domain module’s dependency on internals and the added calls into C++ may affect performance, but it seems like the least bad one. Backport-PR-URL: #27749 PR-URL: #25993 Fixes: #23862 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Сковорода Никита Андреевич <[email protected]> Reviewed-By: Vladimir de Turckheim <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
src: add WeakReference utility
Add a simple
WeakReferenceutility that we can use until thelanguage provides something on its own.
domain: avoid circular memory references
Avoid circular references that the JS engine cannot see through
because it involves an
async id⇒domainlink.Using weak references is not a great solution, because it increases
the domain module’s dependency on internals and the added calls into
C++ may affect performance, but it seems like the least bad one.
Fixes: #23862
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passes