Support for Gcov coverage in extensions

[jonathf]

What's the problem this feature will solve?

I have a Python package with a Pybind11 extension and I'd like to create coverage reports for both the Python and the C++ code. I can do this by doing the following running:

CFLAGS="-coverage" python setup.py build_ext --inplace
coverage run -m pytest ...
find src -type f -name '*.cpp' | xargs -I{} gcov -o build/temp*/src {}

This works because -coverage adds .gcno files in the build directory, which coverage recognizes and creates .gcda files. Finally gcov is used to create gcov files.

Describe the solution you'd like

Instead of using old not recommended python setup.py, I'd rather do something like:

CFLAGS="-coverage" pip install -e .

and have setup.cfg with:

[build_ext]
inplace=1

be enough to get a build directory with .gcno in it.

Alternative Solutions

Yes with old python setup.py

Additional context

Obviously this assumes gcov is installed on the system.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pfmoore]

This requires the ability to pass "configuration options" to the build backend. Pip currently doesn't support this, mostly because most build backends don't use config options.

To get this to work would need:

1. Setuptools (your build backend) to support setting the flags you need via some combinations of PEP 517 config_options values.
2. Pip to add a UI for users to pass config options to the backend.

A simple UI would be easy enough to add to pip, but it's not likely to get much priority until backends start supporting user configuration in the PEP 517 interface. So I'd suggest you initially explore how to make this work with the setuptools project, and then we can look at the front end UI in pip.

[pradyunsg]

Agreed, it would probably be a good idea to push this to setuptools, since they’d be the right folks to decide on the mechanism they wanna use to enable pip doing something like this.

[jonathf]

Okay.

Note that this is not about the copile flags per-say. Since this is a compiled component, I still need the setup.py, which I can configure to handle the CFLAGS. I added it to the example to make the example complete.

But irrespectively of how the compiler get the added flags, no gcno files are generated when compiling with pip, only through python setup.py build_ext, which as I understand is no longer recommended.

[SethMMorton]

I would like to ping this issue - as far as I can see, the only way to get gcov to successfully run for a C/C++ extension is to use python setup.py build_ext or python setup.py install, etc. However, setuptools is advising us to move away from this.

Is there some guidance on how to get equivalent behavior to python setup.py build_ext with pip install -e .?