Proof of concept: implicit call syntax

[ncoghlan]

BPO	18788
Nosy	@gvanrossum, @ncoghlan, @pitrou, @ezio-melotti, @alex, @ethanfurman, @serhiy-storchaka
Files	implicit_call_statements.diff: Restore print statements as implicit call statements

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-08-20.14:06:16.644>
created_at = <Date 2013-08-20.04:59:08.796>
labels = []
title = 'Proof of concept: implicit call syntax'
updated_at = <Date 2013-08-21.01:50:17.779>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2013-08-21.01:50:17.779>
actor = 'ethan.furman'
assignee = 'none'
closed = True
closed_date = <Date 2013-08-20.14:06:16.644>
closer = 'ncoghlan'
components = []
creation = <Date 2013-08-20.04:59:08.796>
creator = 'ncoghlan'
dependencies = []
files = ['31381']
hgrepos = []
issue_num = 18788
keywords = ['patch']
message_count = 10.0
messages = ['195679', '195681', '195682', '195684', '195690', '195694', '195695', '195697', '195698', '195702']
nosy_count = 9.0
nosy_names = ['gvanrossum', 'ncoghlan', 'peter.otten', 'pitrou', 'ezio.melotti', 'alex', 'ethan.furman', 'Ramchandra Apte', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue18788'
versions = []

[ncoghlan]

After watching one of the presenters at PyTexas struggle painfully with the fact "print value" doesn't work in Python 3, I decided I had to at least *try* to see if a general "implicit call" syntax was feasible within the constraints of the CPython parser.

The attached patch shows that it is, indeed, possible to implement such a syntax by modifying the definition of expr_stmt to allow for an implicit trailing argument list (using the normal argument syntax).

The key benefit of such a feature is that the following would once again be legal Python syntax:

print "Hello world!"

Ambiguous cases (such as "expr * expr" and "expr ** expr") obey the rule that implicit calls are very *low* precedence, so you have to use parens to force the call interpretation. Similarly, a bare expression is not call - you must still append () to force a call without arguments.

In addition to being a pain at the interactive prompt, the change of print from a statement to a function has also been noted as one of the biggest problems with compatibility of third party user scripts for Linux distros changing the default system Python to Python 3. This change would mean most Python 2 print statements either do nothing (no arguments), produce an unexpected trailing newline (trailing comma) or work exactly as they do in Python 2. Only those that use the Python 2 redirection syntax continue to trigger a syntax error in Python 3.

While the *code* changes to achieve this behaviour turned out to be relatively small (most of the diff is in the autogenerated parser code), the documentation and general pedagogical impact could be substantially larger.

If there's even an outside chance this could be accepted, I'm happy to work up a full PEP for it.

[pitrou]

In addition to being a pain at the interactive prompt

Just type "pr<TAB>" and the "print(" call will be auto-completed.
(with TAB-completion being enabled by default in 3.4 ;-))

the change of print from a statement to a function has also been noted
as one of the biggest problems with compatibility of third party user
scripts for Linux distros changing the default system Python to Python 3

Well, I know 2to3 has had some bad press recently, but it should still make it a piece of cake to convert such small scripts.
(since you don't need to keep 2.x compatibility for them)

[peterotten]

a bare expression is not call

Wouldn't that silently swallow a lot of bare

print

statements?

[serhiy-storchaka]

Ambiguous cases (such as "expr * expr" and "expr ** expr") obey the rule that implicit calls are very *low* precedence, so you have to use parens to force the call interpretation.

Does "a + b c" mean "(a + b)(c)"? Does "a + b (c)" mean "(a + b)((c))"?

What does "a (b, c)" mean? "a.__call__(b, c)" or "a.__call__((b, c))"?

[ncoghlan]

As Serhiy's examples show, the ambiguities this introduces get confusing fast. A more constrained version that (for example) permitted only name references for the callable could resolve that, but it's probably still a bad idea. Still, interesting to know it is *technically* possible, if anyone decides to pursue it further :)

The "silently ignores bare print statements" aspect isn't new - Python 3 already behaves that way, since a print on its own line now just references the builtin without calling it.

[serhiy-storchaka]

I like a parens-less call syntax, but it is unpythonic and even anti-pythonic. It can be good in some other language (like Forth or Tcl, may be Perl or Ruby) but not in Python. It requires some other coordinated features (a syntax to obtaining a reference to a function/method, perhaps shorter syntax for lambdas, braced blocks, different scoping and binding rules) and different philosophy. It doesn't quack as Python.

[RamchandraApte]

I do not see much use of of implicit call syntax, a good editor will autocomplete the brackets.

[gvanrossum]

Well aren't there other languages (Ruby, Coffeescript) that have the same syntax? How do they do it? (Haskell does it too, but I don't think we can learn from it -- but in Ruby and IIRC Coffeescsript it is syntactic sugar for a regular call.)

Heck, even Python's predecessor, ABC, used "f x" to mean "f(x)". It solved "a b + c" by giving function call binding the *tightest* possible priority though, interpreting it as (a b) + c, and hence (a(b)) + c", which would be no good for Nick's hope that "print x+1, y" can be made to mean print(x+1, y) -- ABC would interpret it as "(print(x)) + 1, y".

Still, I'd like to hear about how Ruby/Coffee solve this.

[alex]

I suppose I'm one of the more qualified people to comment on how Ruby does it: a mess of hacks in the lexer/parser. Ruby's case is complicated by the fact that a bare foo can either be a local variable or a method call on self. Consider the case a +b, should that be parsed as a call to a with a unary + on b, or an addition? In Ruby this depends on whether a is already defined as a local. Basically I think this is a terrible idea, and would encourage as strongly as possible to not consider this.

[ncoghlan]

To clarify how my patch works: leaving out the parentheses is permitted
*only* when the call is a statement unto itself. That's how it avoids
conflicting with any existing syntax like "a *b" or "a **b": the parser
consumes those as part of the initial expression, so you *need* the parens
to control precedence and get it interpreted as a call instead. It was
surprisingly elegant at the implementation level - once I got the new
expr_stmt definition and parsing right, everything later in the chain "just
worked".

A real patch might want to look at the following enhancements:

• constraining the LHS further. This needs to happen at the AST
  construction stage, since the parser can't look far enough ahead to limit
  it. The patch currently uses the same rules as augmented assignment targets
• constraining the RHS. For example, disallowing generator expressions, so
  they need to use the parenthesised form.
• adding an AST node for clean source-to-source transformations
• allowing implicit calls in more places. For example, as an assignment
  value, or a return value.

Proving it was possible at all with pgen scratched my itch, so someone else
will need to run with it for it to turn into a concrete proposal that
addresses the readability issues.