Missing *-unpacking generalizations

[Yhg1s]

BPO	2292
Nosy	@gvanrossum, @Yhg1s, @birkenfeld, @terryjreedy, @pfmoore, @ncoghlan, @abalkin, @scoder, @larryhastings, @benjaminp, @ezio-melotti, @agbuckley, @bitdancer, @ethanfurman, @ericsnowcurrently, @berkerpeksag, @serhiy-storchaka, @1st1, @zooba, @phmc, @NeilGirdhar
Files	starunpack.diff starunpack2.diff starunpack3.diff starunpack4.diff starunpack5.diff starunpack6.diff starunpack6.diff starunpack7.diff starunpack8.diff starunpack9.diff starunpack10.diff starunpack11.diff starunpack12.diff starunpack13.diff starunpack14.diff starunpack15.diff starunpack16.diff starunpack17.diff starunpack18.diff starunpack19.diff starunpack20.diff starunpack21.diff starunpack22.diff starunpack23.diff starunpack24.diff starunpack25.diff starunpack26.diff starunpack27.diff starunpack28.diff starunpack29.diff starunpack30.diff starunpack31.diff starunpack32.diff starunpack33.diff starunpack34.diff starunpack35.diff starunpack36.diff starunpack37.diff starunpack38.diff starunpack39.diff starunpack40.diff starunpack41.diff starunpack42.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/benjaminp'
closed_at = <Date 2015-05-06.00:17:23.889>
created_at = <Date 2008-03-15.15:41:19.194>
labels = ['interpreter-core', 'type-feature', 'release-blocker']
title = 'Missing *-unpacking generalizations'
updated_at = <Date 2017-02-11.15:40:07.206>
user = 'https://github.com/Yhg1s'

bugs.python.org fields:

activity = <Date 2017-02-11.15:40:07.206>
actor = 'serhiy.storchaka'
assignee = 'benjamin.peterson'
closed = True
closed_date = <Date 2015-05-06.00:17:23.889>
closer = 'benjamin.peterson'
components = ['Interpreter Core']
creation = <Date 2008-03-15.15:41:19.194>
creator = 'twouters'
dependencies = []
files = ['9955', '32670', '37779', '37788', '37790', '37791', '37792', '37795', '37798', '37799', '37801', '37805', '37806', '37811', '37821', '37851', '37852', '37853', '37854', '37866', '37867', '37871', '37876', '37877', '37896', '37911', '37920', '37921', '37925', '37926', '37928', '38062', '38070', '38252', '38253', '38341', '38395', '38429', '38611', '38613', '38856', '39059', '39230']
hgrepos = []
issue_num = 2292
keywords = ['patch']
message_count = 172.0
messages = ['63548', '63550', '63551', '63552', '63553', '63554', '63557', '63563', '63859', '65012', '65018', '65023', '65079', '65089', '65095', '65096', '65100', '65109', '65110', '65111', '65116', '65117', '65125', '67465', '88533', '88537', '100293', '100295', '100296', '100304', '100305', '116416', '149588', '151143', '186099', '186101', '191754', '192984', '203191', '234332', '234342', '234366', '234368', '234370', '234372', '234377', '234378', '234380', '234384', '234385', '234386', '234393', '234394', '234397', '234404', '234405', '234406', '234408', '234409', '234411', '234413', '234414', '234415', '234416', '234418', '234419', '234420', '234421', '234422', '234432', '234436', '234445', '234457', '234458', '234459', '234460', '234462', '234463', '234464', '234465', '234466', '234467', '234468', '234469', '234483', '234489', '234493', '234516', '234518', '234519', '234520', '234521', '234522', '234528', '234529', '234530', '234531', '234537', '234538', '234539', '234540', '234541', '234543', '234544', '234663', '234668', '234669', '234670', '234681', '234711', '234754', '234755', '234757', '234758', '234759', '234760', '234764', '234766', '234767', '234771', '234772', '234785', '234800', '234914', '234916', '235009', '235018', '235029', '235030', '235031', '235038', '235040', '235041', '235058', '235281', '235292', '235297', '235302', '235310', '235368', '235641', '235646', '235647', '235648', '236557', '236702', '236703', '237257', '237688', '238751', '241113', '242210', '242212', '242438', '242442', '242446', '242624', '242625', '242626', '242629', '242631', '242666', '242669', '242721', '242723', '242724', '242739', '248019', '248025', '257139', '261617', '287606']
nosy_count = 25.0
nosy_names = ['gvanrossum', 'twouters', 'georg.brandl', 'terry.reedy', 'paul.moore', 'ncoghlan', 'belopolsky', 'scoder', 'larry', 'benjamin.peterson', 'ezio.melotti', 'andybuckley', 'r.david.murray', 'zbysz', 'ethan.furman', 'eric.snow', 'berker.peksag', 'Joshua.Landau', 'serhiy.storchaka', 'yselivanov', 'steve.dower', 'pconnell', 'NeilGirdhar', 'Jeff.Kaufman', 'SpaghettiToastBook']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue2292'
versions = ['Python 3.5']

[Yhg1s]

The attached patch adds the missing *-unpacking generalizations.
Specifically:

>> a, b, *c = range(5)

>>> *a, b, c = a, b, *c
>>> a, b, c
([0, 1, 2], 3, 4)
>>> [ *a, b, c ]
[0, 1, 2, 3, 4]
>>> L = [ a, (3, 4), {5}, {6: None}, (i for i in range(7, 10)) ]
>>> [ *item for item in L ]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Also, yielding everything from an iterator:

>>> def flatten(iterables):
...     for it in iterables:
...         yield *it
... 
>>> L = [ a, (3, 4), {5}, {6: None}, (i for i in range(7, 10)) ]
>>> flatten(L)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

[abalkin]

This was discussed years ago and never got enough support:

http://mail.python.org/pipermail/python-dev/2005-October/057177.html

[gvanrossum]

Didn't you say it does sets too? Does this work?
a = [1, 2, 3]
{1, *a, 0, 4} # {0, 1, 2, 3, 4}

How about dicts?
kwds = {'z': 0, 'w': 12}
{'x': 1, 'y': 2, **kwds} # {'x': 1, 'y': 2, 'z': 0, 'w': 12}

Also, now that we support

[*a, b, c]

shouldn't we also support

foo(*a, b, c)

?

[Yhg1s]

On Sat, Mar 15, 2008 at 9:12 AM, Guido van Rossum <report@bugs.python.org>
wrote:

Guido van Rossum <guido@python.org> added the comment:

Didn't you say it does sets too? Does this work?
a = [1, 2, 3]
{1, *a, 0, 4} # {0, 1, 2, 3, 4}

Yes.

How about dicts?
kwds = {'z': 0, 'w': 12}
{'x': 1, 'y': 2, **kwds} # {'x': 1, 'y': 2, 'z': 0, 'w': 12}

Not yet.

Also, now that we support

[*a, b, c]

shouldn't we also support

foo(*a, b, c)

Sure. (And also 'foo(*a, *b, *c)'?) But have you taken a look lately at the
function definition grammar? I need some time to sort it out :)

[gvanrossum]

Looking at the flatten() example I'm curious -- how come the output of

>> flatten(L)

is displayed as a list rather than as <generator at xxxxxx> ?

Also, do I understand correctly that

yield *(1, 2, 3)

is equivalent to

yield 1
yield 2
yield 3

? (That's really cool.)

[Yhg1s]

On Sat, Mar 15, 2008 at 9:18 AM, Guido van Rossum <report@bugs.python.org>
wrote:

Guido van Rossum <guido@python.org> added the comment:

Looking at the flatten() example I'm curious -- how come the output of

>>> flatten(L)

is displayed as a list rather than as <generator at xxxxxx> ?

It's a typo. It should've been list(flatten(L)) :-) (see the tests included
in the patch.)

[abalkin]

It looks like I completely missed PEP-3132. Sorry for a misleading
comment above.

At least I am not alone: "A little new feature in Python 3 that many
overviews don't tell you about: extended unpacking." http://pyside.blogspot.com/2007/10/new-in-python-3-extended-
unpacking.html

[birkenfeld]

Just a nit: the syntax error message for invalid starred expressions
should be changed from "can use starred expression only as assignment
target".

[gvanrossum]

This is fun, but needs more work (see python-3000 thread).

I'm setting the priority to low, since I won't hold up a release to get
this in (if there's even a rough consensus).

[Yhg1s]

Updated patch: reworked some internals, and added generalization of
functioncalls after talking with Guido. *args is now considered just
another positional argument, and can occur anywhere in the positional
argument section. It can also occur more than once. Keyword arguments
now have to appear after *args, and **kwargs can now occur multiple
times at any position in the keyword argument list. test_extcall has
some examples.

(The opcodes are largely unaffected; just the order of '*args' and
keyword arguments is changed. Behind the scenes, anything after the
first '*args' argument is collapsed into a single *args, and everything
after the first '**kwargs' is likewise collapsed. The common case
(meaning any currently valid syntax, barring the 2to3 fix to swap *args
and keyword arguments) does not change in meaning or codepath, just the
complex cases are handled differently.)

This is still Work In Progress. To do: implement the dict unpacking
syntax (the mechanics are already there for keyword arguments to
functioncalls), make sure the precendence of * is correct, get more
complete test coverage, iron out the cosmetic bugs in the 2to3 fixer.

Bzr branch for this patch is
http://code.python.org/python/users/twouters/starunpack . There is also
a branch with just the functioncall changes (although the starunpack
changes are a small sprinkling on top of that branch, as it uses the
same new mechanics): http://code.python.org/python/users/twouters/funcargs .