Multiple "unknown option" errors when passing unknown arguments to the interpreter

[The-Compiler]

BPO	40527
Nosy	@vstinner, @serhiy-storchaka, @The-Compiler, @corona10, @miss-islington
PRs	bpo-40527: Fix command line argument parsing #19955 [3.8] bpo-40527: Fix command line argument parsing (GH-19955) #19956

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-05-06.14:41:54.922>
created_at = <Date 2020-05-06.09:32:05.332>
labels = ['interpreter-core', 'type-bug', '3.8', '3.9']
title = 'Multiple "unknown option" errors when passing unknown arguments to the interpreter'
updated_at = <Date 2020-05-06.16:17:31.263>
user = 'https://github.com/The-Compiler'

bugs.python.org fields:

activity = <Date 2020-05-06.16:17:31.263>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-05-06.14:41:54.922>
closer = 'corona10'
components = ['Interpreter Core']
creation = <Date 2020-05-06.09:32:05.332>
creator = 'The Compiler'
dependencies = []
files = []
hgrepos = []
issue_num = 40527
keywords = ['patch']
message_count = 8.0
messages = ['368231', '368244', '368246', '368249', '368253', '368259', '368265', '368275']
nosy_count = 7.0
nosy_names = ['vstinner', 'hieu.nguyen', 'serhiy.storchaka', 'tweksteen', 'The Compiler', 'corona10', 'miss-islington']
pr_nums = ['19955', '19956']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40527'
versions = ['Python 3.8', 'Python 3.9']

[The-Compiler]

A minor issue I just discovered today: When e.g. doing "python3 --foo", the output is:

unknown option --foo
unknown option --foo
unknown option --foo
usage: /usr/bin/python3 [option] ... [-c cmd | -m mod | file | -] [arg] ...

With more dashes in the options, it seems to get worse:

unknown option --foo-bar-baz
unknown option --foo-bar-baz
unknown option --foo-bar-baz
unknown option --foo-bar-baz
unknown option --foo-bar-baz
unknown option --foo-bar-baz
unknown option --foo-bar-baz
usage: /usr/bin/python3 [option] ... [-c cmd | -m mod | file | -] [arg] ...

This seems very similar to bpo-16306 (thus why I added people involved there to the nosy list), except that it happens with long rather than short arguments.

This only happens with Python 3.8/3.9, not 3.7 or older. I was able to bisect this to the following commit:

commit 6dcb542
Author: Victor Stinner <vstinner@redhat.com>
Date: Tue Mar 5 02:44:12 2019 +0100

bpo-36142: Add _PyPreConfig_ReadFromArgv() (GH-12173)

The new function is now responsible to parse -E and -I command line
arguments.

[vstinner]

Oh right, that's because Python parses the command line at least 3 times :-) The first 2 times, it is not supposed to log errors. It's a bug: attached PR 19955 fix it.

[vstinner]

I was able to bisect this to the following commit: (...)

Oh, well done! Yeah, it's all my fault ;-) It's a regression of the PEP-587 (PyConfig) implementation.

I chose to share code between Python/preconfig.c and Python/initconfig.c rather than duplicate code. The implementation of _PyPreConfig_Read() and PyConfig_Read() is quite complex.

[corona10]

New changeset 2668a9a by Victor Stinner in branch 'master':
bpo-40527: Fix command line argument parsing (GH-19955)
2668a9a

[miss-islington]

New changeset bce4dda by Miss Islington (bot) in branch '3.8':
bpo-40527: Fix command line argument parsing (GH-19955)
bce4dda

[corona10]

Thanks for the bug report Florian and for work Victor!

I am now closing this issue :)