Add a section on memory management for extension
#2864
Conversation
guide/src/memory.md
Outdated
|
|
||
| ## On writing extension | ||
|
|
||
| Using `#[pyfunction]` and `#[pymethods]` makes `Python::with_gil` a no-op and |
There was a problem hiding this comment.
This is not really correct. Using Python::allow_threads implies that Python::with_gil is not a no-op.
I think, the main observation is that within #[pyfunction] and #[pymethod] is already held and hence Python objects owned references tied to the GIL lifetime will usually be released only when the #[pyfunction] or #[pymethod] returns.
(The above also suggests another correction: It isn't any Python objects, but only those owned by GIL-bound Rust references.)
I would also suggest that the text stresses that this is usually not an issue except when the #[pyfunction] or #[pymethod] are long-running and/or perform a lot of transient allocations. This does happen, but it is not the common case.
Finally, the section is highly redundant to the existing one on "GIL-bound memory" and I would prefer if the information was integrated into that section. Since that section already contains everything related to the behaviour within loops and how to apply a GILPool, I think the main additions this PR should make are a) how this interactions with #[pyfunction] and #[pymethod] and b) a link to #1056 for further information on future design directions.
There was a problem hiding this comment.
Got it.
I have made the section a small comment on the "GIL-bound memory" section, with a small explanation of the mechanism.
|
Build succeeded: |
To alievate the unbounded memory growth, we're replacing variable dereferencing with scoped `GILPool` as described in the pyo3 documentation recently updated. See: PyO3/pyo3#2864
To alievate the unbounded memory growth, we're replacing variable dereferencing with scoped `GILPool` as described in the pyo3 documentation recently updated. See: PyO3/pyo3#2864
Adding a special case of memory management when writing an extension.
This is a documentation of: #1056 and #2853