Skip to content

Is there a nogil container? #947

Open
@impredicative

Description

@impredicative

I want a 3.13 nogil container, also with all optimizations, etc. I assume that the current 3.13 containers use the GIL.

Activity

tianon

tianon commented on Aug 12, 2024

@tianon
Member

We don't have a "nogil" image currently. For 3.13 we're using the default compilation settings, which AFAIK still results in the GIL. Unfortunately, compiling Python is pretty heavy, so expanding our current support matrix to include new "nogil" variants is also going to be a tough sell. 🙈

tianon

tianon commented on Aug 12, 2024

@tianon
Member

Are you saying you don't compile Python?

I'm not sure how you got there from what I said -- perhaps I can clarify: we do compile Python (currently ~42 times per architecture, in fact, and we support ~10 architectures where compilation is necessary), and it's very heavy to do so, and adding more variants would require us to do so even more times, so is not something we're willing to consider unless/until it is an officially recommended configuration (ideally the default configuration, but we'd be willing to consider an official statement along the lines of "everyone should try this out").

impredicative

impredicative commented on Aug 12, 2024

@impredicative
Author

nogil is the future of Python. It doesn't make sense to not support it. If this request seems strange, that may be so only because I am the first person to ask for it. Rest assured, there will be many more who will progressively ask for it. The pressure will build. This request is easy to reject now, but only now.

tianon

tianon commented on Aug 12, 2024

@tianon
Member

I agree that nogil is the future of Python, and there's nothing strange about the request. If I were completely opposed to the idea in general, I probably would've closed the issue already. It's more a matter of resourcing/prioritization, not desire/understanding. If the demand were actively higher, it would also be a lot easier to justify the added resources/maintenance spend -- the best way for folks to signal that is to react with 👍 on the top post in this thread.

To put that another way, my answer is more accurately "not yet", not "never".

If you'd like to help with the work, figuring out what changes are necessary to our current Dockerfiles would be a really useful first step.

ldeluigi

ldeluigi commented on Aug 19, 2024

@ldeluigi

An experimental JIT-enabled build would also be great to have

sekrause

sekrause commented on Aug 27, 2024

@sekrause

Instead of prodiving a whole second set of "nogil" Docker images the free-threaded executable could also be part of the normal images just like it is done with the official Windows and macOS installers:

CPython now has experimental support for running in a free-threaded mode, with the global interpreter lock (GIL) disabled. This is an experimental feature and therefore is not enabled by default. The free-threaded mode requires a different executable, usually called python3.13t or python3.13t.exe. Pre-built binaries marked as free-threaded can be installed as part of the official Windows and macOS installers, or CPython can be built from source with the --disable-gil option.

https://docs.python.org/3.13/whatsnew/3.13.html#free-threaded-cpython

So we would have /usr/local/bin/python3.13t in every image and call this to use the free-threaded mode.

Unfortunately that still means that all of Python has to be compiled twice.

impredicative

impredicative commented on Aug 27, 2024

@impredicative
Author

@sekrause But that would break a lot of tooling that expects Python to be at the prior file path.

sekrause

sekrause commented on Aug 27, 2024

@sekrause

But that would break a lot of tooling that expects Python to be at the prior file path.

The normal paths would still be there, but point to the normal GIL version. So if you don't adjust your tooling everything will be as before.

abebus

abebus commented on Sep 2, 2024

@abebus

If anyone seeking for some basic image to play with, I've made one: https://github.com/abebus/free-threaded-python-docker-image

martin-budbee

martin-budbee commented on Oct 18, 2024

@martin-budbee
SDAravind

SDAravind commented on Oct 21, 2024

@SDAravind

Python images without GIL and JIT enabled should be part of docker images. I have been using docker as a dev container and to experiment new versions of python. Thanks to all the work been done so far.

msingh0101

msingh0101 commented on Nov 22, 2024

@msingh0101
pitrou

pitrou commented on Dec 18, 2024

@pitrou

I'm a bit surprised that you're going out of your way to provide images for alpha versions of Python (if I'm reading https://hub.docker.com/_/python correctly, there are images for Python 3.14.0a2), but not for free-threaded builds which are an official feature of released Pythons. While these builds are still marked experimental, many Python packages have started using them to ensure they are free-threading compatible.

For example in Apache Arrow we have our own image for CI until the official Python Docker images support free-threaded builds:
https://github.com/apache/arrow/blob/main/ci/docker/python-free-threaded-wheel-manylinux-test-imports.dockerfile

Many Python open source projects are probably using similar home-grown recipes due to lack of official Docker image availability.

impredicative

impredicative commented on Dec 18, 2024

@impredicative
Author

I think four builds are needed:

  1. GIL and noJIT (current build)
  2. noGIL and noJIT (this issue)
  3. GIL and JIT
  4. noGIL and JIT

Is this correct?

2 remaining items

impredicative

impredicative commented on Dec 18, 2024

@impredicative
Author

Those are unrelated, perhaps open a separate issue for JIT builds?

It matters because the naming convention has to account for all possible variations. Also, it might help in understanding that bundling multiple versions in the same image is not a great idea.

(but I'd argue that JIT builds are unimportant for now, as they don't provide tangible benefits)

JIT may not matter too much now, but it's the sauce that could get Python closer to Julia-like speeds, so it may matter in the long term.

rinarakaki

rinarakaki commented on Feb 21, 2025

@rinarakaki

Any update?

tianon

tianon commented on Feb 21, 2025

@tianon
Member

#947 (comment) is still an accurate representation of the current maintainer position

I would be willing to consider something like docker-library/tomcat#299 to have a place where folks can see/test Dockerfiles for this, but we cannot reasonably commit to producing official built images for it at this time.

impredicative

impredicative commented on May 22, 2025

@impredicative
Author
angstwad

angstwad commented on May 22, 2025

@angstwad

While it's not exactly a fun time to roll your own image, most of the work is done by the team here. I built my own image pretty quickly to do some freetheading experiments and documented how I did it here: yaml/pyyaml#856 (comment)

You could also enable experimental JIT fairly quickly with yet another option (--experimental-jit-interpreter) as documented upstream

pitrou

pitrou commented on May 22, 2025

@pitrou

Rather than building Python yourself, a better option would be to use one of the builds from https://github.com/astral-sh/python-build-standalone/

impredicative

impredicative commented on May 22, 2025

@impredicative
Author

Rather than building Python yourself, a better option would be to use one of the builds from https://github.com/astral-sh/python-build-standalone/

I suppose this might be usable from uv setting up a project, and that the Dockerfile will have to use uv in this way, specifically by defining 3.13t or such (with the t suffix) as the Python requirement. I haven't tried it.

There are some open issues/bugs though wrt uv's support for t Python.


Various other ways to instal free-threaded Python are listed at https://py-free-threading.github.io/installing-cpython/, also via docker, such as perhaps via the quay.io/pypa/manylinux_2_28_x86_64 image.

hpkfft

hpkfft commented on Jun 15, 2025

@hpkfft

Given the significant improvements to the free-threaded mode in Python 3.14, I would suggest prioritizing #1052 over providing a container for free-threaded 3.13.

changed the title [-]Is there a 3.13 nogil container?[/-] [+]Is there a nogil container?[/+] on Jun 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @tianon@impredicative@sekrause@angstwad@pitrou

        Issue actions

          Is there a nogil container? · Issue #947 · docker-library/python