Skip to content

Make the garbage collector thread-safe in --disable-gil builds #112529

Closed
@colesbury

Description

@colesbury

Feature or enhancement

Python's cyclic garbage collector relies on the global interpreter lock for thread-safety. There are a number of changes needed to make the GC thread-safe in --disable-gil builds.

My intention is to implement these as a series of smaller changes.

  • Garbage collection is guarded by gcstate->collecting. This need to be made thread-safe.
  • The --disable-gil builds should stop-the-world when finding garbage, but not when calling finalizers or destructors. (depends on Implement stop-the-world functionality (for --disable-gil builds) #111964)
  • The --disable-gil builds should find GC-enabled objects by traversing the mimalloc heaps, because the _gc_next/_gc_prev lists are not thread-safe. (Note it's safe to use the lists during stop-the-world pauses; we just can't maintain them safely during normal Python execution).
  • The --disable-gil builds should probably use only a single generation to avoid frequent stop-the-world pauses
  • Eventually, we can get rid of the GC pre-header in --disable-gil builds to save memory. (This will require updating the trashcan mechanism.)

Remaining work:

  • Fix scheduling of free-threaded GC (currently scheduled as if there is a young generation)
  • Update Python docs
  • Refactor out common code from gc.c and gc_free_threaded.c

See also #111964

Linked PRs

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions