Closed
Description
Feature or enhancement
Python's cyclic garbage collector relies on the global interpreter lock for thread-safety. There are a number of changes needed to make the GC thread-safe in --disable-gil
builds.
My intention is to implement these as a series of smaller changes.
- Garbage collection is guarded by
gcstate->collecting
. This need to be made thread-safe. - The
--disable-gil
builds should stop-the-world when finding garbage, but not when calling finalizers or destructors. (depends on Implement stop-the-world functionality (for--disable-gil
builds) #111964) - The
--disable-gil
builds should find GC-enabled objects by traversing the mimalloc heaps, because the_gc_next
/_gc_prev
lists are not thread-safe. (Note it's safe to use the lists during stop-the-world pauses; we just can't maintain them safely during normal Python execution). - The
--disable-gil
builds should probably use only a single generation to avoid frequent stop-the-world pauses - Eventually, we can get rid of the GC pre-header in
--disable-gil
builds to save memory. (This will require updating the trashcan mechanism.)
Remaining work:
- Fix scheduling of free-threaded GC (currently scheduled as if there is a young generation)
- Update Python docs
- Refactor out common code from
gc.c
andgc_free_threaded.c
See also #111964
Linked PRs
- gh-112529: Use atomic operations for
gcstate->collecting
#112533 - gh-112529: Track if debug allocator is used as underlying allocator #113747
- gh-112529: Use GC heaps for GC allocations in free-threaded builds #114157
- gh-112529: Implement GC for free-threaded builds #114262
- gh-112529: Remove PyGC_Head from object pre-header in free-threaded build #114564
- gh-112529: Simplify PyObject_GC_IsTracked and PyObject_GC_IsFinalized #114732
- gh-112529: Stop the world around gc.get_referents #114823
- gh-112529: Make the GC scheduling thread-safe #114880
- gh-112529: Use _PyThread_Id() in mimalloc (free-threaded build) #115488
- gh-112529: Revert "Use _PyThread_Id() in mimalloc in free-threaded build" #115524
- gh-112529: Don't untrack tuples or dicts with zero refcount #117370