build failure when enabling dtrace on FreeBSD

[swills]

BPO	29077
Nosy	@cburroughs, @ambv, @zware, @koobs, @swills, @0mp

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-12-26.19:19:02.376>
labels = ['3.8', 'build', '3.7', '3.9', '3.10']
title = 'build failure when enabling dtrace on FreeBSD'
updated_at = <Date 2021-08-10.09:36:19.945>
user = 'https://github.com/swills'

bugs.python.org fields:

activity = <Date 2021-08-10.09:36:19.945>
actor = '0mp'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build']
creation = <Date 2016-12-26.19:19:02.376>
creator = 'swills'
dependencies = []
files = []
hgrepos = []
issue_num = 29077
keywords = []
message_count = 4.0
messages = ['284038', '284209', '295239', '399309']
nosy_count = 6.0
nosy_names = ['cburroughs', 'lukasz.langa', 'zach.ware', 'koobs', 'swills', '0mp']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue29077'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

[swills]

When enabling dtrace support via the --with-dtrace configure flag, build fails on FreeBSD with this message:

/usr/sbin/dtrace -o Include/pydtrace_probes.h -h -s
dtrace: option requires an argument -- s

Looks like line 882 of Makefile.pre.in:

882         $(DTRACE) $(DFLAGS) -o $@ -h -s $<

changing this to:

882         $(DTRACE) $(DFLAGS) -o $@ -h -s $(srcdir)/Include/pydtrace.d

avoids that issue. Note that bmake is being used here.

After this however, another issue is encountered during linking:

Python/ceval.o: In function _PyEval_EvalFrameDefault': [Python/ceval.c](https://github.com/python/cpython/blob/main/Python/ceval.c):(.text+0xc94): undefined reference to __dtraceenabled_python___function__entry'
Python/ceval.c:(.text+0xcdd): undefined reference to __dtrace_python___function__entry' [Python/ceval.c](https://github.com/python/cpython/blob/main/Python/ceval.c):(.text+0xff0): undefined reference to __dtraceenabled_python___line'
Python/ceval.c:(.text+0x109f): undefined reference to __dtrace_python___line' [Python/ceval.c](https://github.com/python/cpython/blob/main/Python/ceval.c):(.text+0x125e): undefined reference to __dtraceenabled_python___line'
Python/ceval.c:(.text+0x12e5): undefined reference to __dtraceenabled_python___line' [Python/ceval.c](https://github.com/python/cpython/blob/main/Python/ceval.c):(.text+0x1358): undefined reference to __dtraceenabled_python___line'
Python/ceval.c:(.text+0x13f2): undefined reference to __dtraceenabled_python___line' [Python/ceval.c](https://github.com/python/cpython/blob/main/Python/ceval.c):(.text+0x146c): undefined reference to __dtraceenabled_python___line'
Python/ceval.o:Python/ceval.c:(.text+0x14de): more undefined references to __dtraceenabled_python___line' follow [Python/ceval.o](https://github.com/python/cpython/blob/main/Python/ceval.o): In function _PyEval_EvalFrameDefault':
Python/ceval.c:(.text+0x87f2): undefined reference to __dtraceenabled_python___function__return' [Python/ceval.c](https://github.com/python/cpython/blob/main/Python/ceval.c):(.text+0x8833): undefined reference to __dtrace_python___function__return'
Modules/gcmodule.o: In function collect': [Modules/gcmodule.c](https://github.com/python/cpython/blob/main/Modules/gcmodule.c):(.text+0x380): undefined reference to __dtraceenabled_python___gc__start'
Modules/gcmodule.c:(.text+0x38c): undefined reference to __dtrace_python___gc__start' [Modules/gcmodule.c](https://github.com/python/cpython/blob/main/Modules/gcmodule.c):(.text+0xe1b): undefined reference to __dtraceenabled_python___gc__done'
Modules/gcmodule.c:(.text+0xe2a): undefined reference to `__dtrace_python___gc__done'

which I believe is due to dtrace -G modifying Python/ceval.o and Modules/gcmodule.o.

Finally, note that all calls to dtrace -G or dtrace -h on FreeBSD 11.0 should have -xnolibs added to enable use without the DTrace kernel modules loaded. Otherwise dtrace processes the /usr/lib/dtrace scripts unnecessarily, and errors out when it tries to talk to the DTrace kernel code, causing build failure in a jail.

(I can provide patches, but suspect you have preferences about how the build system work that I'm not aware of, so wanted to explain before patching in ways that aren't satisfactory.)

[ambv]

I cannot repro as I don't have a FreeBSD environment handy. I will gladly accept a patch that makes it work with FreeBSD 11.0!

I admit I tested my implementation with OS X 10.11 and Linux (SystemTap with Ubuntu).

[koobs]

For what it's worth, the koobs-freebsd-current buildbot worker has DTrace enabled and can be used to build/test Python/dtrace bits

Zach and I spoke recently about getting custom builds hooked up to GitHub, which I believe is now already possible, so all that's left is to determine a way to pass the appropriate extra configure argument to the build without having to create another customer builder. CC Zach accordingly

[0mp]

Patch for the FreeBSD Ports to fix the build failure: https://reviews.freebsd.org/D31489

[emaste]

I tried building at 48ec61a on FreeBSD-current just now and ../configure --with-dtrace was successful. Build still failed at link, as above.

Aside, it seems that the tests are unaware of whether Python was built with dtrace support; default configure (without --with-dtrace) test_dtrace failed with dtrace: failed to compile script /tmp/cirrus-ci-build/Lib/test/dtracedata/call_stack.d: line 1: probe description python22930:::function-entry does not match any probes