Skip to content

Fix __dict__ leak on Python 3.11 and 3.12#6234

Open
ngoldbaum wants to merge 6 commits into
PyO3:mainfrom
ngoldbaum:fix-dict-leak
Open

Fix __dict__ leak on Python 3.11 and 3.12#6234
ngoldbaum wants to merge 6 commits into
PyO3:mainfrom
ngoldbaum:fix-dict-leak

Conversation

@ngoldbaum

Copy link
Copy Markdown
Contributor

On CPython 3.11 and 3.12, every instance of a #[pyclass(dict)] class leaks one empty dict object at instance creation. This is separate from #5953 / #6198, which fixes the leak of the dict reference held in the slot at deallocation: with #6198 applied, #[pyclass(dict)] classes still leak one dict per instance on 3.11/3.12.

The other reference on 3.11 and 3.12 is due to Python eagerly instantiating __dict__ at class creation, which we clobber during pyclass setup. The eager instantiation was removed in Python 3.13.

Marking as a draft for now because the test requires #6198.

I found this with an AI model reviewing #6198.

Comment thread src/pyclass_init.rs Outdated
// zeroes (a valid empty slot value) or a valid owned pointer stored by the base
// `tp_new` through the type's `tp_dictoffset`.
new_contents.dict =
unsafe { core::ptr::read(&raw const (*(*contents).as_mut_ptr()).dict) };

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this will be easier to read?

        #[cfg(not(Py_3_13))]
        let new_contents = {
            let mut new_contents = new_contents;
            // SAFETY: `tp_alloc` zero-initializes the object, so the slot contains either
            // zeroes (a valid empty slot value) or a valid owned pointer stored by the base
            // `tp_new` through the type's `tp_dictoffset`.
-           new_contents.dict =
-               unsafe { core::ptr::read(&raw const (*(*contents).as_mut_ptr()).dict) };
+           let contents_ptr = (*contents).as_mut_ptr();
+           let dict_ptr = core::ptr::addr_of!((*contents_ptr).dict);
+           new_contents.dict = unsafe { core::ptr::read(dict_ptr) };
            new_contents
        };

@ngoldbaum
ngoldbaum marked this pull request as ready for review July 25, 2026 16:19
@ngoldbaum ngoldbaum mentioned this pull request Jul 25, 2026
11 tasks

@davidhewitt davidhewitt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, so many fixes for #[pyclass(dict)] all of a sudden!

@davidhewitt
davidhewitt added this pull request to the merge queue Jul 26, 2026
@davidhewitt
davidhewitt removed this pull request from the merge queue due to a manual request Jul 26, 2026

@davidhewitt davidhewitt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually just a couple of small comments, sorry.

Comment thread src/pyclass_init.rs Outdated
Comment thread src/pyclass_init.rs
Comment on lines +171 to +172
#[cfg(not(Py_3_13))]
let new_contents = {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For abi3 builds should we also consider doing a runtime gate? e.g. otherwise a 3.12 abi3 build still does this twiddling on 3.13+. It's probably harmless to have it not gated, but might be good to be thorough. (See bpo_45315_workaround as an example.)

Also I wonder if this can somehow be skipped for cases when T::Dict is PyClassDummySlot (i.e. no dict).

@ngoldbaum ngoldbaum Jul 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a eagerly_created_dict_possible helper that does the limited API runtime python version check and and the check for a PyClassDummySlot.

@davidhewitt davidhewitt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, one final suggestion particularly to cover the < 3.11 cases, and then let's merge.

Comment thread src/pyclass_init.rs
Comment on lines +208 to +218
#[cfg(Py_LIMITED_API)]
{
use crate::sync::PyOnceLock;
static IS_PYTHON_3_13: PyOnceLock<bool> = PyOnceLock::new();
!*IS_PYTHON_3_13.get_or_init(py, || py.version_info() >= (3, 13))
}
#[cfg(not(Py_LIMITED_API))]
{
let _ = py;
true
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slight preference to use cfg_select! here for newer conditional code. Also now that we're differentiating abi3 and non-abi3 pathways we can more properly account for the < 3.11 cases, maybe as the following:

Suggested change
#[cfg(Py_LIMITED_API)]
{
use crate::sync::PyOnceLock;
static IS_PYTHON_3_13: PyOnceLock<bool> = PyOnceLock::new();
!*IS_PYTHON_3_13.get_or_init(py, || py.version_info() >= (3, 13))
}
#[cfg(not(Py_LIMITED_API))]
{
let _ = py;
true
}
cfg_select! {
Py_LIMITED_API => {
use crate::sync::PyOnceLock;
static IS_PYTHON_3_11_OR_3_12: PyOnceLock<bool> = PyOnceLock::new();
IS_PYTHON_3_11_OR_3_12.get_or_init(py, || {
let version_info = py.version_info();
version_info >= (3, 11) && version_info < (3, 13)
})
},
not(Py_LIMITED_API) => {
let _ = py;
cfg!(Py_3_11)
}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants