Interning string constants with null character

[serhiy-storchaka]

BPO	28350
Nosy	@brettcannon, @birkenfeld, @rhettinger, @ncoghlan, @vstinner, @benjaminp, @serhiy-storchaka, @1st1
PRs	[Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
Files	all_name_chars.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-10-04.15:27:43.425>
created_at = <Date 2016-10-03.19:23:36.619>
labels = ['interpreter-core', 'type-bug', '3.7']
title = 'Interning string constants with null character'
updated_at = <Date 2017-03-31.16:36:09.847>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:09.847>
actor = 'dstufft'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-10-04.15:27:43.425>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2016-10-03.19:23:36.619>
creator = 'serhiy.storchaka'
dependencies = []
files = ['44945']
hgrepos = []
issue_num = 28350
keywords = ['patch']
message_count = 6.0
messages = ['277994', '277997', '278042', '278043', '278047', '278049']
nosy_count = 9.0
nosy_names = ['brett.cannon', 'georg.brandl', 'rhettinger', 'ncoghlan', 'vstinner', 'benjamin.peterson', 'python-dev', 'serhiy.storchaka', 'yselivanov']
pr_nums = ['552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28350'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7']

[serhiy-storchaka]

Currently string constants are interned if they consist of ASCII word characters ([0-9A-Za-z_]). But strings are tested only to the first null character. This is not problem for names, since they can't include null characters, but string constants that contains ASCII non-word characters after the null character passes this test.

Proposed simple patch fixes the testing function all_name_chars().

Other question: shouldn't PyUnicode_IsIdentifier() be used in 3.x?

[vstinner]

all_name_chars.patch LGTM.

[serhiy-storchaka]

Thanks Victor.

[vstinner]

I have no opinion on interning non-ASCII strings.

[python-dev]

New changeset 522adc2e082a by Serhiy Storchaka in branch '2.7':
Issue bpo-28350: String constants with null character no longer interned.
https://hg.python.org/cpython/rev/522adc2e082a

New changeset d7ab3241aef2 by Serhiy Storchaka in branch '3.5':
Issue bpo-28350: String constants with null character no longer interned.
https://hg.python.org/cpython/rev/d7ab3241aef2

New changeset 8585b4de4fc0 by Serhiy Storchaka in branch '3.6':
Issue bpo-28350: String constants with null character no longer interned.
https://hg.python.org/cpython/rev/8585b4de4fc0

New changeset 563d523036c6 by Serhiy Storchaka in branch 'default':
Issue bpo-28350: String constants with null character no longer interned.
https://hg.python.org/cpython/rev/563d523036c6

[serhiy-storchaka]

Added tests and refactored all_name_chars().