The ast module fails to reject certain malformed nodes

[brandtbucher]

BPO	40355
Nosy	@pablogsal, @miss-islington, @remilapeyre, @brandtbucher, @isidentical, @curtisbucher
PRs	bpo-40355: ast.literal_eval rejects malformed Dict nodes #19868 [3.8] bpo-40355: Improve error messages in ast.literal_eval with malformed Dict nodes (GH-19868) #19947

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/brandtbucher'
closed_at = <Date 2020-05-06.08:21:23.178>
created_at = <Date 2020-04-21.19:59:22.958>
labels = ['type-bug', 'library', '3.9']
title = 'The ast module fails to reject certain malformed nodes'
updated_at = <Date 2020-05-06.08:21:23.177>
user = 'https://github.com/brandtbucher'

bugs.python.org fields:

activity = <Date 2020-05-06.08:21:23.177>
actor = 'BTaskaya'
assignee = 'brandtbucher'
closed = True
closed_date = <Date 2020-05-06.08:21:23.178>
closer = 'BTaskaya'
components = ['Library (Lib)']
creation = <Date 2020-04-21.19:59:22.958>
creator = 'brandtbucher'
dependencies = []
files = []
hgrepos = []
issue_num = 40355
keywords = ['patch']
message_count = 4.0
messages = ['366938', '366941', '368200', '368202']
nosy_count = 6.0
nosy_names = ['pablogsal', 'miss-islington', 'remi.lapeyre', 'brandtbucher', 'BTaskaya', 'curtisbucher']
pr_nums = ['19868', '19947']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40355'
versions = ['Python 3.9']

[brandtbucher]

There are several places in the ast module where the use of zip is allowing malformed nodes to have unpaired children silently thrown away. A couple of short examples:

>>> from ast import Constant, Dict, literal_eval, unparse
>>> nasty_dict = Dict(keys=[Constant("I don't have a value!")], values=[])
>>> unparse(nasty_dict)
'{}'
>>> literal_eval(nasty_dict)
{}

I'm currently working on a patch to raise errors instead.

[isidentical]

See https://bugs.python.org/msg360767 for ast.unparse's point of view regarding malformed nodes.

[pablogsal]

New changeset c21c512 by Curtis Bucher in branch 'master':
bpo-40355: Improve error messages in ast.literal_eval with malformed Dict nodes (GH-19868)
c21c512

[miss-islington]

New changeset 2a3b876 by Miss Islington (bot) in branch '3.8':
bpo-40355: Improve error messages in ast.literal_eval with malformed Dict nodes (GH-19868)
2a3b876