Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix leak in File System Watcher #12144

Merged
merged 2 commits into from
Feb 7, 2023

Conversation

tsmaeder
Copy link
Contributor

@tsmaeder tsmaeder commented Feb 3, 2023

Signed-off-by: Thomas Mäder [email protected]

What it does

The code was calling Promise.race() repeatedly while waiting for a file to come into existence. This added a PromiseReaction object each time, which never was collected.

Fixes #12001

Contributed on behalf of STMicroelectronics

How to test

Make sure file system watchers are still started correctly: in particular for

  1. Open editors
  2. Preferences, Launch configs, etc.
  3. Workspace folders

Since the fix concerns file that don't exist in the beginning, it makes sense cases like when I add a new file .theia/launch.json and expect the launch configs to show up and changes to the file to be tracked.

Review checklist

Reminder for reviewers

The code was calling Promise.race() repeatedly while waiting for a file
to come into existence. This added a `PromiseReaction` object each time,
which never was collected

Contributed on behalf of STMicroelectronics

Signed-off-by: Thomas Mäder <[email protected]>
@paul-marechal
Copy link
Member

paul-marechal commented Feb 3, 2023

The old code was very unpalatable and the new code looked much better, except it was missing the part where cancellation should reject the promise returned by start with WatcherDisposal that acts as some flag value.

I was also not understanding why the previous code caused a leak, and I still don't precisely understand the mechanism by which it happened. But I've found discussions about Promise.race causing leaks when used in certain patterns: nodejs/node#17469 (comment)

@tsmaeder
Copy link
Contributor Author

tsmaeder commented Feb 6, 2023

I was also not understanding why the previous code caused a leak, and I still don't precisely understand the mechanism by which it happened.

When you say promiseA.then(promiseB), you need a pointer from promiseA to promiseB, so that when promiseA settles, you can start evaluating promiseB. This seems to be implemented via a PromiseReaction object in node (at least, that's what shows up in the heap dump). The same applies to race(): there needs to be some bookkeeping about which promise depende on which other promises. That bookkeeping consumes heap memory.

@paul-marechal
Copy link
Member

My point is that some garbage collection should be happening and it's not. People on the Promise.race leak thread seem to suggest this is an issue with V8's Promise.race implementation. They went as far as to create a garbage-collector-friendly JS implementation...

What we need to remember when using Promise.race from now on is that nothing will get collected for as long as one of the promises that's part of the race hasn't completed. Even if we discard the race promise.

Copy link
Member

@paul-marechal paul-marechal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM, file watching still works for me and CI is not showing new failures.

@paul-marechal paul-marechal merged commit 6959b6a into eclipse-theia:master Feb 7, 2023
@paul-marechal paul-marechal added this to the 1.35.0 milestone Feb 7, 2023
@vince-fugnitto vince-fugnitto added the filesystem issues related to the filesystem label Feb 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
filesystem issues related to the filesystem
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Potential filewatcher memory leak
3 participants