Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Viewer instances don't release memory when cell output is cleared #6642

Closed
jleibs opened this issue Jun 25, 2024 · 5 comments
Closed

Viewer instances don't release memory when cell output is cleared #6642

jleibs opened this issue Jun 25, 2024 · 5 comments
Labels
🪳 bug Something isn't working notebook Jupyter notebooks etc 🕸️ web regarding running the viewer in a browser
Milestone

Comments

@jleibs
Copy link
Member

jleibs commented Jun 25, 2024

A given notebook session can eventually run out of memory at which point the only way to fix it appears to be reloading the page.

My expectation is that when the output of a cell is cleared, the memory should be freed.

To repro, repeatedly execute a cell that outputs a viewer widget until it stops outputting.

At this point the console output for the page will show:

WebAssembly module validated with warning: failed to allocate executable memory for module
Uncaught (in promise) InternalError: out of memory 

image

It would be nice if we could also catch this and show the error in the notebook. Instead all we get is an empty white canvas and the newly introduced timeout warning.
image

@jleibs jleibs added 🪳 bug Something isn't working 👀 needs triage This issue needs to be triaged by the Rerun team labels Jun 25, 2024
@jleibs jleibs added this to the 0.17 milestone Jun 25, 2024
@jleibs jleibs added 🕸️ web regarding running the viewer in a browser notebook Jupyter notebooks etc and removed 👀 needs triage This issue needs to be triaged by the Rerun team labels Jun 25, 2024
@jleibs
Copy link
Member Author

jleibs commented Jun 25, 2024

I noticed that every time I re-execute a cell, I another instance of WebAssembly.instantiate being tracked:
image

@emilk
Copy link
Member

emilk commented Jun 26, 2024

Is this a new problem, or a pre-existing one? Not that it matters a lot, seems very high-priority to fix it

@jleibs
Copy link
Member Author

jleibs commented Jun 26, 2024

It's new. The old notebook approach added an iframe to the page. When the previous iframe is removed from the DOM, it seems like the right cleanup happens.

@jleibs
Copy link
Member Author

jleibs commented Jun 26, 2024

Digging into the memory inspector and taking snapshots after each cell execution, you can see we leak approximately 100MB per execution. Something is holding a reference that keeps it from being GC'd.

image

@jleibs
Copy link
Member Author

jleibs commented Jun 26, 2024

I believe this is actually an upstream anywidget issue: manzt/anywidget#613

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🪳 bug Something isn't working notebook Jupyter notebooks etc 🕸️ web regarding running the viewer in a browser
Projects
None yet
Development

No branches or pull requests

2 participants