Handle unpacking of */** arguments and rvalues in the compiler

[markshannon]

BPO	39320
Nosy	@vstinner, @markshannon, @bharel, @pablogsal, @brandtbucher
PRs	bpo-39320: Handle unpacking of *values in compiler #17984 bpo-39320: Handle unpacking of **values in compiler #18141 bpo-39320: Don't use an oparg for new DICT_* opcodes. #18243 bpo-39320: Fix changes in the evaluation logic of unpacking operations. #18264

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-08-30.09:13:18.070>
created_at = <Date 2020-01-13.12:28:10.197>
labels = ['interpreter-core', '3.9', 'performance']
title = 'Handle unpacking of */** arguments and rvalues in the compiler'
updated_at = <Date 2021-12-18.15:41:31.066>
user = 'https://github.com/markshannon'

bugs.python.org fields:

activity = <Date 2021-12-18.15:41:31.066>
actor = 'bar.harel'
assignee = 'none'
closed = True
closed_date = <Date 2021-08-30.09:13:18.070>
closer = 'Mark.Shannon'
components = ['Interpreter Core']
creation = <Date 2020-01-13.12:28:10.197>
creator = 'Mark.Shannon'
dependencies = []
files = []
hgrepos = []
issue_num = 39320
keywords = ['patch']
message_count = 5.0
messages = ['359901', '360548', '360749', '375260', '408857']
nosy_count = 5.0
nosy_names = ['vstinner', 'Mark.Shannon', 'bar.harel', 'pablogsal', 'brandtbucher']
pr_nums = ['17984', '18141', '18243', '18264']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue39320'
versions = ['Python 3.9']

[markshannon]

Currently the unpacking of starred values in arguments and the right hand side of assignments is handled in the interpreter without any help from the compiler.
The layout of arguments and values is visible to the compiler, so the compiler should do more of the work.

We can replace the complex bytecodes used in unpacking with simpler more focused ones.
Specifically the collection building operations
BUILD_LIST_UNPACK, BUILD_TUPLE_UNPACK, BUILD_SET_UNPACK and BUILD_TUPLE_UNPACK_WITH_CALL
can be replaced with simpler, and self-explanatory operations:
LIST_TO_TUPLE, LIST_EXTEND, SET_UPDATE

In addition, the mapping operations
BUILD_MAP_UNPACK and BUILD_MAP_UNPACK_WITH_CALL
can be replaced with DICT_UPDATE and DICT_MERGE.

DICT_MERGE is like DICT_UPDATE but raises an exception for duplicate keys.

This change would not have much of an effect of performance, as the bytecodes listed are relatively rarely used, but shrinking the interpreter is always beneficial.

[markshannon]

New changeset 13bc139 by Mark Shannon in branch 'master':
bpo-39320: Handle unpacking of *values in compiler (GH-17984)
13bc139

[markshannon]

New changeset 8a4cd70 by Mark Shannon in branch 'master':
bpo-39320: Handle unpacking of **values in compiler (GH-18141)
8a4cd70

[vstinner]

These changes introduced a regression: bpo-41531 "Python 3.9 regression: Literal dict with > 65535 items are one item shorter".

[bharel]

Does this count as a regression or as an unintended bugfix for evaluation order?

https://stackoverflow.com/a/70404659/1658617