Python 3.3.4: SyntaxError with correct source code encoding # -*- coding: latin-1 -*-

[OPiMedia]

BPO	20731
Nosy	@loewis, @birkenfeld, @vstinner, @larryhastings, @bitdancer, @serhiy-storchaka, @koobs, @OPiMedia
Files	test.py test1.py issue20731.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-03-01.15:34:34.358>
created_at = <Date 2014-02-22.19:20:15.566>
labels = ['interpreter-core', 'build', 'OS-windows', 'release-blocker']
title = 'Python 3.3.4: SyntaxError with correct source code encoding  # -*- coding: latin-1 -*-'
updated_at = <Date 2014-03-01.21:34:15.725>
user = 'https://github.com/OPiMedia'

bugs.python.org fields:

activity = <Date 2014-03-01.21:34:15.725>
actor = 'loewis'
assignee = 'none'
closed = True
closed_date = <Date 2014-03-01.15:34:34.358>
closer = 'python-dev'
components = ['Interpreter Core', 'Windows']
creation = <Date 2014-02-22.19:20:15.566>
creator = 'OPi'
dependencies = []
files = ['34190', '34210', '34211']
hgrepos = []
issue_num = 20731
keywords = ['patch']
message_count = 15.0
messages = ['211942', '211961', '212038', '212046', '212063', '212068', '212073', '212074', '212437', '212438', '212445', '212492', '212504', '212505', '212520']
nosy_count = 11.0
nosy_names = ['loewis', 'georg.brandl', 'vstinner', 'larry', 'Arfrever', 'r.david.murray', 'miwa', 'python-dev', 'serhiy.storchaka', 'koobs', 'OPi']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue20731'
versions = ['Python 3.3', 'Python 3.4']

[OPiMedia]

The test program:
#!/usr/bin/env python
# -- coding: latin-1 --

print('test')

is correct in Python 3.3.3, but cause this error with Python 3.3.4:
File "./test.py", line 3
-
^
SyntaxError: invalid syntax

I use Windows 7, with bash (of Cygwin) and
Python 3.3.4 (v3.3.4:7ff62415e426, Feb 10 2014, 18:13:51) [MSC v.1600 64 bit (AMD64)] on win32

[bitdancer]

For whatever it's worth, it works fine for me on 3.3 tip on unix. The fact that it says line 3 and shows just a - is suspicious.

[loewis]

I can reproduce the problem. I started debugging it; the issue is that, after reading the two comment lines (and detecting the source encoding), fileio_readinto gets called, which calls read(), and read() returns 17 (bytes read), with are

"-\n\nprint('test')"

So for some reason, the - is not consumed yet, even though it was previously read out of the file descriptor.

[bitdancer]

Oops, I missed the fact that it was Benjamin that added Serhiy...Benjamin can add himself if he wants :)

[miwa]

You can avoid this by changing end of line to CRLF. (I don't know why)

I attach a simpler test case.

[OPiMedia]

All works fine with UTF-8.
All works fine with latin-1 (or other) and Windows end of line.

Maybe this issue is important, "First line can be executed twice":
http://bugs.python.org/issue18960

[serhiy-storchaka]

I can't test on Windows but may be this patch fixes the issue.

[birkenfeld]

Just in time for 3.3.5...

[python-dev]

New changeset ade5e4922a54 by Martin v. Löwis in branch '3.3':
Issue bpo-20731: Properly position in source code files even if they
http://hg.python.org/cpython/rev/ade5e4922a54

[loewis]

Thanks for the patch, this is applied to 3.3 and 3.4.