re.groupindex is available for modification and continues to work, having incorrect data inside it

[py-user]

BPO	14260
Nosy	@gvanrossum, @birkenfeld, @vstinner, @ezio-melotti, @merwok, @py-user, @ericsnowcurrently, @serhiy-storchaka
Files	re_groupindex_copy.patch re_groupindex_proxy.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-03-30.07:08:36.635>
created_at = <Date 2012-03-12.05:28:22.645>
labels = ['expert-regex', 'type-bug']
title = 're.groupindex is available for modification and continues to work, having incorrect data inside it'
updated_at = <Date 2015-03-30.07:08:36.634>
user = 'https://github.com/py-user'

bugs.python.org fields:

activity = <Date 2015-03-30.07:08:36.634>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-03-30.07:08:36.635>
closer = 'serhiy.storchaka'
components = ['Regular Expressions']
creation = <Date 2012-03-12.05:28:22.645>
creator = 'py.user'
dependencies = []
files = ['37098', '37099']
hgrepos = []
issue_num = 14260
keywords = ['patch', 'needs review']
message_count = 18.0
messages = ['155442', '155459', '155484', '155546', '155560', '155570', '155593', '155702', '155734', '175494', '175497', '175502', '175505', '230450', '234878', '239041', '239094', '239526']
nosy_count = 11.0
nosy_names = ['gvanrossum', 'georg.brandl', 'vstinner', 'ezio.melotti', 'eric.araujo', 'mrabarnett', 'py.user', 'python-dev', 'eric.snow', 'serhiy.storchaka', 'Abel.Farias']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14260'
versions = ['Python 3.5']

[py-user]

>>> import re
>>> p = re.compile(r'abc(?P<n>def)')
>>> p.sub(r'\g<n>', 'abcdef123abcdef')
'def123def'
>>> p.groupindex['n'] = 2
>>> p.sub(r'\g<n>', 'abcdef123abcdef')
'def123def'
>>> p.groupindex
{'n': 2}
>>>

[mrabarnett]

The re module creates the dict purely for the benefit of the user, and as it's a normal dict, it's mutable.

An alternative would to use an immutable dict or dict-like object, but Python doesn't have such a class, and it's probably not worth writing one just for this use-case.

[py-user]

Matthew Barnett wrote:

The re module creates the dict purely for the benefit of the user

this dict affects on regex.sub()

>>> import re
>>> p = re.compile(r'abc(?P<n>def)')
>>> p.groupindex
{'n': 1}
>>> p.groupindex['n'] = 2
>>> p.sub(r'\g<n>', 'abcdef')
Traceback (most recent call last):
  File "/usr/local/lib/python3.2/sre_parse.py", line 811, in expand_template
    literals[index] = s = g(group)
IndexError: no such group

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.2/re.py", line 286, in filter
    return sre_parse.expand_template(template, match)
  File "/usr/local/lib/python3.2/sre_parse.py", line 815, in expand_template
    raise error("invalid group reference")
sre_constants.error: invalid group reference
>>>

[mrabarnett]

It appears I was wrong. :-(

The simplest solution in that case is for it to return a _copy_ of the dict.

[merwok]

But regex.sub is affected only if you manually muck with the dict, right? If so, then it looks like a case of “it hurts when I do this” (the doctor’s reply: “Don’t do this.”)

[py-user]

the first message shows how it can work with a broken dict

Éric Araujo wrote:

But regex.sub is affected only if you manually muck with the dict, right?

I can get code from anywhere

[merwok]

I can get code from anywhere
I am afraid I don’t understand. Could you start again and explain what bug you ran into, i.e. what behavior does not match what the docs say? At present this report looks like it is saying “when I put random things in an internal data structures then bad things happen”, and I don‘t think Python promises to not break when people do random editions to internal data structures.

[py-user]

I take someone's code
make tests for its behavior
all tests say "the code is working"
I continue to write the code
make tests for its behavior
all tests say "the code is working"
I install it somewhere and it crashes

now it is depending on the cache, when this exception is raised

Éric Araujo wrote:

and I don‘t think Python promises to not break when people do random editions

when people do something wrong, python should raise an exception

[birkenfeld]

Looks like a case for a read-only dict/dictproxy :)

[serhiy-storchaka]

I fully agree with Éric. Just don't do this.

[skrah]

I'm not so sure. If dicts or classes are used for configuration
or informational purposes, I prefer them to be locked down.

An example of the first is the decimal context, where it was possible
to write context.emax = 9 instead of context.Emax = 9 without getting
an error. This is an easy mistake to make and can be hard to track
down in a large program.

The mistake here is maybe less likely, but I agree with Georg that
it's a case for a read-only dict/dictproxy.