Error when using regex in docs #1863
Description
Describe the bug
When using the following code in my pyproject.toml, as documented:
exclude_also = ["(?s)\\A.*# pragma: exclude file.*\\Z"]When running coverage report, I receive the following traceback:
Traceback (most recent call last):
File "/opt/venv/bin/coverage", line 8, in <module>
sys.exit(main())
^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/cmdline.py", line 970, in main
status = CoverageScript().command_line(argv)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/cmdline.py", line 708, in command_line
total = self.coverage.report(
^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/control.py", line 1090, in report
return reporter.report(morfs, outfile=file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/report.py", line 181, in report
for fr, analysis in get_analysis_to_report(self.coverage, morfs):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/report_core.py", line 100, in get_analysis_to_report
analysis = coverage._analyze(morf)
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/control.py", line 948, in _analyze
return analysis_from_file_reporter(data, self.config.precision, file_reporter, filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/results.py", line 31, in analysis_from_file_reporter
statements = file_reporter.lines()
^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/python.py", line 194, in lines
return self.parser.statements
^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/python.py", line 189, in parser
self._parser.parse_source()
File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 273, in parse_source
self._raw_parse()
File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 134, in _raw_parse
self.raw_excluded = self.lines_matching(self.exclude)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 112, in lines_matching
regex_c = re.compile(regex, re.MULTILINE)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/__init__.py", line 228, in compile
return _compile(pattern, flags)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/__init__.py", line 307, in _compile
p = _compiler.compile(pattern, flags)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_compiler.py", line 745, in compile
p = _parser.parse(p, flags)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_parser.py", line 979, in parse
p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_parser.py", line 460, in _parse_sub
itemsappend(_parse(source, state, verbose, nested + 1,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_parser.py", line 862, in _parse
p = _parse_sub(source, state, sub_verbose, nested + 1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_parser.py", line 460, in _parse_sub
itemsappend(_parse(source, state, verbose, nested + 1,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_parser.py", line 840, in _parse
raise source.error('global flags not at the start '
re.error: global flags not at the start of the expression at position 59Removing the line from the configuration fixes things.
I didn't have time to investigate very thoroughly (sorry!), but If I had to make a random guess, I'm guessing this regex gets concatenated with other(s) and the (?s) is no longer at the beginning, which is broken in Python 3.11+.
To Reproduce
Unfortunately, this doesn't seem to be perfectly reproducible. While I can reproduce this now 100% of the time, when I first introduced the configuration, all seemed well. It was only (very) shortly after that this seemed to happen.
How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:
- What version of Python are you using? - 3.12.6
- What version of coverage.py shows the problem? The output of
coverage debug sysis helpful. - 7.6.1 - What versions of what packages do you have installed? The output of
pip freezeis helpful. - see below - What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix. - difficult to determine if the code matters
- What commands should we run to reproduce the problem? Be specific. Include everything, even
git clone,pip install, and so on. Explain like we're five! - unsure what this depends on, butcoverage run -m pytestfollowed bycoverage reportgives me the traceback. I run it like this to make sure pytest-cov or pytest-xdist are not the problem
Packages
anyio==4.6.0 argon2-cffi==23.1.0 argon2-cffi-bindings==21.2.0 asgiref==3.8.1 asttokens==2.4.1 boto3==1.35.24 botocore==1.35.24 brotli==1.1.0 certifi==2024.8.30 cffi==1.17.1 cfgv==3.4.0 charset-normalizer==3.3.2 click==8.1.7 colorama==0.4.6 coverage==7.6.1 cryptography==43.0.1 cssbeautifier==1.15.1 decorator==5.1.1 diff-match-patch==20230430 distlib==0.3.8 django==5.1.1 django-activity-stream==2.0.0 django-admin-notice==3.1.0 django-allauth==65.0.0 django-anymail==12.0 django-auto-prefetch==1.9.0 django-axes==6.5.2 django-browser-reload==1.15.0 django-countries==7.6.1 django-coverage-plugin==3.1.0 django-database-url==1.0.3 django-debug-toolbar==4.4.6 django-email-bandit==2.0 django-extensions==3.2.3 django-filter==24.3 django-formtools==2.5.1 django-hijack==3.6.0 django-htmx==1.19.0 django-import-export==4.1.1 django-ipware==7.0.1 django-localflavor==4.0 django-model-utils==5.0.0 django-object-actions==4.3.0 django-otp==1.5.4 django-permissions-policy==4.21.0 django-phonenumber-field==8.0.0 django-read-only==1.16.0 django-rq==2.10.2 django-rq-email-backend==2.0.0 django-sequences==3.0 django-storages==1.14.4 django-stubs==5.0.4 django-stubs-ext==5.0.4 django-template-partials==24.4 django-test-migrations==1.4.0 django-two-factor-auth==1.17.0 django-waffle==4.1.0 djlint==1.35.2 editorconfig==0.12.4 execnet==2.1.1 executing==2.1.0 factory-boy==3.3.1 faker==29.0.0 filelock==3.16.1 greenlet==3.0.3 gunicorn==23.0.0 h11==0.14.0 hiredis==3.0.0 html-tag-names==0.1.2 html-void-elements==0.1.0 httpcore==1.0.5 httpx==0.27.2 hubspot-api-client==9.0.0 identify==2.6.1 idna==3.10 iniconfig==2.0.0 ipdb==0.13.13 ipython==8.27.0 jedi==0.19.1 jmespath==1.0.1 jsbeautifier==1.15.1 json5==0.9.25 matplotlib-inline==0.1.7 messagebird==2.1.0 multidict==6.1.0 nodeenv==1.9.1 packaging==24.1 parso==0.8.4 pathspec==0.12.1 pexpect==4.9.0 phonenumbers==8.13.45 pillow==10.4.0 pip-lock==2.11.0 platformdirs==4.3.6 playwright==1.47.0 pluggy==1.5.0 pre-commit==3.8.0 prompt-toolkit==3.0.47 psycopg==3.2.2 psycopg-pool==3.2.3 ptyprocess==0.7.0 pure-eval==0.2.3 pycowsay==0.0.0.2 pycparser==2.22 pyee==12.0.0 pygments==2.18.0 pyjwt==2.9.0 pypng==0.20220715.0 pyright==1.1.381 pytest==8.3.3 pytest-base-url==2.1.0 pytest-cov==5.0.0 pytest-django==4.9.0 pytest-mock==3.14.0 pytest-playwright==0.5.2 pytest-recording==0.13.2 pytest-rerunfailures==14.0 pytest-sugar==1.0.0 pytest-xdist==3.6.1 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 python-ipware==3.0.0 python-slugify==8.0.4 python-stdnum==1.20 pywatchman==2.0.0 pyyaml==6.0.2 qrcode==7.4.2 redis==5.0.8 regex==2024.9.11 requests==2.32.3 rq==1.16.2 ruff==0.6.7 s3transfer==0.10.2 sentry-sdk==2.14.0 setuptools==75.1.0 six==1.16.0 sniffio==1.3.1 sqlparse==0.5.1 stack-data==0.6.3 tablib==3.5.0 tenacity==9.0.0 termcolor==2.4.0 text-unidecode==1.3 tqdm==4.66.5 traitlets==5.14.3 types-pyyaml==6.0.12.20240917 typing-extensions==4.12.2 urllib3==2.2.3 vcrpy==6.0.1 virtualenv==20.26.5 watchdog==5.0.2 wcwidth==0.2.13 whitenoise==6.7.0 wrapt==1.16.0 yarl==1.11.1
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Relevant configuration:
[tool.coverage.run]
branch = true
omit = [
".venv/*",
"node_modules/*",
"requirements*.txt",
"runtime.txt",
"staticfiles/*",
]
plugins = ["django_coverage_plugin"]
relative_files = true
source = ["."]
[tool.coverage.report]
exclude_also = ["(?s)\\A.*# pragma: exclude file.*\\Z"]
show_missing = true
[tool.pytest.ini_options]
DJANGO_SETTINGS_MODULE = "foo.settings"
markers = [
"slow: marks tests as slow (deselect with '-m \"not slow\"')",
"migration: marks tests as slow (deselect with '-m \"not migration\"')",
]
python_files = ["tests.py", "test_*.py"]
testpaths = ["foo"]I saw that the code has changed a little in the master branch, so I tried that as well, but just got a slightly different traceback:
Traceback (most recent call last):
File "/opt/venv/bin/coverage", line 8, in <module>
sys.exit(main())
^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/cmdline.py", line 970, in main
status = CoverageScript().command_line(argv)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/cmdline.py", line 708, in command_line
total = self.coverage.report(
^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/control.py", line 1090, in report
return reporter.report(morfs, outfile=file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/report.py", line 181, in report
for fr, analysis in get_analysis_to_report(self.coverage, morfs):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/report_core.py", line 100, in get_analysis_to_report
analysis = coverage._analyze(morf)
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/control.py", line 948, in _analyze
return analysis_from_file_reporter(data, self.config.precision, file_reporter, filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/results.py", line 31, in analysis_from_file_reporter
statements = file_reporter.lines()
^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/python.py", line 194, in lines
return self.parser.statements
^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/python.py", line 189, in parser
self._parser.parse_source()
File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 267, in parse_source
self._raw_parse()
File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 131, in _raw_parse
self.raw_excluded = self.lines_matching(self.exclude)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.12/site-packages/coverage/parser.py", line 114, in lines_matching
for match in re.finditer(regex, self.text, flags=re.MULTILINE):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/__init__.py", line 224, in finditer
return _compile(pattern, flags).finditer(string)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/__init__.py", line 307, in _compile
p = _compiler.compile(pattern, flags)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_compiler.py", line 745, in compile
p = _parser.parse(p, flags)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_parser.py", line 979, in parse
p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_parser.py", line 460, in _parse_sub
itemsappend(_parse(source, state, verbose, nested + 1,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_parser.py", line 862, in _parse
p = _parse_sub(source, state, sub_verbose, nested + 1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_parser.py", line 460, in _parse_sub
itemsappend(_parse(source, state, verbose, nested + 1,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/re/_parser.py", line 840, in _parse
raise source.error('global flags not at the start '
re.error: global flags not at the start of the expression at position 59