Error when using regex in docs

[knyghty]

Describe the bug
When using the following code in my pyproject.toml, as documented:

exclude_also = ["(?s)\\A.*# pragma: exclude file.*\\Z"]

When running coverage report, I receive the following traceback:

Traceback (most recent call last):
  File "/opt/venv/bin/coverage", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/cmdline.py", line 970, in main
    status = CoverageScript().command_line(argv)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/cmdline.py", line 708, in command_line
    total = self.coverage.report(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/control.py", line 1090, in report
    return reporter.report(morfs, outfile=file)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/report.py", line 181, in report
    for fr, analysis in get_analysis_to_report(self.coverage, morfs):
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/report_core.py", line 100, in get_analysis_to_report
    analysis = coverage._analyze(morf)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/control.py", line 948, in _analyze
    return analysis_from_file_reporter(data, self.config.precision, file_reporter, filename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/results.py", line 31, in analysis_from_file_reporter
    statements = file_reporter.lines()
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/python.py", line 194, in lines
    return self.parser.statements
           ^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/python.py", line 189, in parser
    self._parser.parse_source()
  File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 273, in parse_source
    self._raw_parse()
  File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 134, in _raw_parse
    self.raw_excluded = self.lines_matching(self.exclude)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 112, in lines_matching
    regex_c = re.compile(regex, re.MULTILINE)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/__init__.py", line 228, in compile
    return _compile(pattern, flags)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/__init__.py", line 307, in _compile
    p = _compiler.compile(pattern, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_compiler.py", line 745, in compile
    p = _parser.parse(p, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_parser.py", line 979, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_parser.py", line 460, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_parser.py", line 862, in _parse
    p = _parse_sub(source, state, sub_verbose, nested + 1)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_parser.py", line 460, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_parser.py", line 840, in _parse
    raise source.error('global flags not at the start '
re.error: global flags not at the start of the expression at position 59

Removing the line from the configuration fixes things.

I didn't have time to investigate very thoroughly (sorry!), but If I had to make a random guess, I'm guessing this regex gets concatenated with other(s) and the (?s) is no longer at the beginning, which is broken in Python 3.11+.

To Reproduce

Unfortunately, this doesn't seem to be perfectly reproducible. While I can reproduce this now 100% of the time, when I first introduced the configuration, all seemed well. It was only (very) shortly after that this seemed to happen.

How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

1. What version of Python are you using? - 3.12.6
2. What version of coverage.py shows the problem? The output of coverage debug sys is helpful. - 7.6.1
3. What versions of what packages do you have installed? The output of pip freeze is helpful. - see below
4. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix. - difficult to determine if the code matters
5. What commands should we run to reproduce the problem? Be specific. Include everything, even git clone, pip install, and so on. Explain like we're five! - unsure what this depends on, but coverage run -m pytest followed by coverage report gives me the traceback. I run it like this to make sure pytest-cov or pytest-xdist are not the problem

Packages

anyio==4.6.0
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
asgiref==3.8.1
asttokens==2.4.1
boto3==1.35.24
botocore==1.35.24
brotli==1.1.0
certifi==2024.8.30
cffi==1.17.1
cfgv==3.4.0
charset-normalizer==3.3.2
click==8.1.7
colorama==0.4.6
coverage==7.6.1
cryptography==43.0.1
cssbeautifier==1.15.1
decorator==5.1.1
diff-match-patch==20230430
distlib==0.3.8
django==5.1.1
django-activity-stream==2.0.0
django-admin-notice==3.1.0
django-allauth==65.0.0
django-anymail==12.0
django-auto-prefetch==1.9.0
django-axes==6.5.2
django-browser-reload==1.15.0
django-countries==7.6.1
django-coverage-plugin==3.1.0
django-database-url==1.0.3
django-debug-toolbar==4.4.6
django-email-bandit==2.0
django-extensions==3.2.3
django-filter==24.3
django-formtools==2.5.1
django-hijack==3.6.0
django-htmx==1.19.0
django-import-export==4.1.1
django-ipware==7.0.1
django-localflavor==4.0
django-model-utils==5.0.0
django-object-actions==4.3.0
django-otp==1.5.4
django-permissions-policy==4.21.0
django-phonenumber-field==8.0.0
django-read-only==1.16.0
django-rq==2.10.2
django-rq-email-backend==2.0.0
django-sequences==3.0
django-storages==1.14.4
django-stubs==5.0.4
django-stubs-ext==5.0.4
django-template-partials==24.4
django-test-migrations==1.4.0
django-two-factor-auth==1.17.0
django-waffle==4.1.0
djlint==1.35.2
editorconfig==0.12.4
execnet==2.1.1
executing==2.1.0
factory-boy==3.3.1
faker==29.0.0
filelock==3.16.1
greenlet==3.0.3
gunicorn==23.0.0
h11==0.14.0
hiredis==3.0.0
html-tag-names==0.1.2
html-void-elements==0.1.0
httpcore==1.0.5
httpx==0.27.2
hubspot-api-client==9.0.0
identify==2.6.1
idna==3.10
iniconfig==2.0.0
ipdb==0.13.13
ipython==8.27.0
jedi==0.19.1
jmespath==1.0.1
jsbeautifier==1.15.1
json5==0.9.25
matplotlib-inline==0.1.7
messagebird==2.1.0
multidict==6.1.0
nodeenv==1.9.1
packaging==24.1
parso==0.8.4
pathspec==0.12.1
pexpect==4.9.0
phonenumbers==8.13.45
pillow==10.4.0
pip-lock==2.11.0
platformdirs==4.3.6
playwright==1.47.0
pluggy==1.5.0
pre-commit==3.8.0
prompt-toolkit==3.0.47
psycopg==3.2.2
psycopg-pool==3.2.3
ptyprocess==0.7.0
pure-eval==0.2.3
pycowsay==0.0.0.2
pycparser==2.22
pyee==12.0.0
pygments==2.18.0
pyjwt==2.9.0
pypng==0.20220715.0
pyright==1.1.381
pytest==8.3.3
pytest-base-url==2.1.0
pytest-cov==5.0.0
pytest-django==4.9.0
pytest-mock==3.14.0
pytest-playwright==0.5.2
pytest-recording==0.13.2
pytest-rerunfailures==14.0
pytest-sugar==1.0.0
pytest-xdist==3.6.1
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
python-ipware==3.0.0
python-slugify==8.0.4
python-stdnum==1.20
pywatchman==2.0.0
pyyaml==6.0.2
qrcode==7.4.2
redis==5.0.8
regex==2024.9.11
requests==2.32.3
rq==1.16.2
ruff==0.6.7
s3transfer==0.10.2
sentry-sdk==2.14.0
setuptools==75.1.0
six==1.16.0
sniffio==1.3.1
sqlparse==0.5.1
stack-data==0.6.3
tablib==3.5.0
tenacity==9.0.0
termcolor==2.4.0
text-unidecode==1.3
tqdm==4.66.5
traitlets==5.14.3
types-pyyaml==6.0.12.20240917
typing-extensions==4.12.2
urllib3==2.2.3
vcrpy==6.0.1
virtualenv==20.26.5
watchdog==5.0.2
wcwidth==0.2.13
whitenoise==6.7.0
wrapt==1.16.0
yarl==1.11.1

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Relevant configuration:

[tool.coverage.run]
branch = true
omit = [
    ".venv/*",
    "node_modules/*",
    "requirements*.txt",
    "runtime.txt",
    "staticfiles/*",
]
plugins = ["django_coverage_plugin"]
relative_files = true
source = ["."]

[tool.coverage.report]
exclude_also = ["(?s)\\A.*# pragma: exclude file.*\\Z"]
show_missing = true

[tool.pytest.ini_options]
DJANGO_SETTINGS_MODULE = "foo.settings"
markers = [
    "slow: marks tests as slow (deselect with '-m \"not slow\"')",
    "migration: marks tests as slow (deselect with '-m \"not migration\"')",
]
python_files = ["tests.py", "test_*.py"]
testpaths = ["foo"]

I saw that the code has changed a little in the master branch, so I tried that as well, but just got a slightly different traceback:

Traceback (most recent call last):
  File "/opt/venv/bin/coverage", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/cmdline.py", line 970, in main
    status = CoverageScript().command_line(argv)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/cmdline.py", line 708, in command_line
    total = self.coverage.report(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/control.py", line 1090, in report
    return reporter.report(morfs, outfile=file)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/report.py", line 181, in report
    for fr, analysis in get_analysis_to_report(self.coverage, morfs):
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/report_core.py", line 100, in get_analysis_to_report
    analysis = coverage._analyze(morf)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/control.py", line 948, in _analyze
    return analysis_from_file_reporter(data, self.config.precision, file_reporter, filename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/results.py", line 31, in analysis_from_file_reporter
    statements = file_reporter.lines()
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/python.py", line 194, in lines
    return self.parser.statements
           ^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/python.py", line 189, in parser
    self._parser.parse_source()
  File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 267, in parse_source
    self._raw_parse()
  File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 131, in _raw_parse
    self.raw_excluded = self.lines_matching(self.exclude)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 114, in lines_matching
    for match in re.finditer(regex, self.text, flags=re.MULTILINE):
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/__init__.py", line 224, in finditer
    return _compile(pattern, flags).finditer(string)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/__init__.py", line 307, in _compile
    p = _compiler.compile(pattern, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_compiler.py", line 745, in compile
    p = _parser.parse(p, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_parser.py", line 979, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_parser.py", line 460, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_parser.py", line 862, in _parse
    p = _parse_sub(source, state, sub_verbose, nested + 1)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_parser.py", line 460, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/re/_parser.py", line 840, in _parse
    raise source.error('global flags not at the start '
re.error: global flags not at the start of the expression at position 59

[nedbat]

Thanks, that's poor testing on my part. It will work if you change it to:

exclude_also = ["\\A(?s:.*# pragma: exclude file.*)\\Z"]

I'll update the docs.

[knyghty]

Indeed it does, thanks Ned!

[nedbat]

I've updated the docs in commit 3e15cbe.

[nedbat]

This is now released as part of coverage 7.6.2.