Default to building the C extension automatically when installing from source #1511
Assignees
Labels
build
contributions-welcome
We welcome contributions to fix this issue!
high priority
(first issues that will be worked on)
Comments
|
Hi @mgorny, I think this would be a very welcome addition. I'm happy to review PRs for this! |
matthewdouglas
added
contributions-welcome
mgorny
added a commit
to mgorny/bitsandbytes
that referenced
this issue
Feb 12, 2025
Call CMake and build the CPU extension when invoking the build via a PEP 517 backend, to ensure that at least some extension is built when users are building from source. This improves consistency with other Python packages, and reduces the risk of accidents. We are using `scikit-build-core` setuptools plugin to take care of CMake dependencies and call into CMake. However, we need to modify the `build_py` command to ensure that CMake is called prior to the setuptools command, as otherwise the newly built shared library won't be picked up by `build_py`. Since setuptools is still responsible for collecting the Python package, it also collects all other shared libraries that were built earlier, for example via manual CMake calls as done in the CI pipeline. Furthermore, if the user does not have `scikit-build-core` installed and calls `setup.py` directly, we output a warning but continue working as before. The logic can be further extended in the future, for example to detect the best COMPUTE_BACKEND default. Fixes bitsandbytes-foundation#1511
|
I've filed #1512 as my first attempt at the goal. |
|
I've got the same issue recently which bothered me a lot. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Feature request
Currently,
pip install git+https://github.com/bitsandbytes-foundation/bitsandbytes(or equivalent) does not build the C extension at all — the build system entirely depends on it being built prior to installing the Python package. While this makes sense for building the portable wheels for PyPI, it's a bit surprising for end users and downstream repackagers — to the point that they may actually package bitsandbytes without the C extension at all.I'd like to propose enhancing
setup.pyto automatically call CMake to build the C extension if it wasn't built yet. Ideally, we could go as far as to detect the correct kind of acceleration to use, but even building the CPU library would be a good start.Motivation
Making testing and repackaging easier. Reducing risk of mistakenly repackaging without the C extension at all. Making the "user-oriented" build process more consistent with regular Python packages — I think that would actually even make it possible to start publishing source distributions on PyPI, to provide wider platform support.
Your contribution
I'm ready to write the code and submit a pull request.
The text was updated successfully, but these errors were encountered: