argparse: assertion failure if optional argument has square/round brackets in metavar

[oxplot]

BPO	14046
Nosy	@ncoghlan, @miss-islington
PRs	[3.7] bpo-11874: fix assertion failure in argparse metavar handling (GH-1826) #7527 [3.6] bpo-11874: fix assertion failure in argparse metavar handling (GH-1826) #7528 [3.7] bpo-11874: fix assertion failure in argparse metavar handling (GH-1826) #7530 [2.7] bpo-11874: fix assertion failure in argparse metavar handling (GH-1826) #7553
Superseder	bpo-11874: argparse assertion failure with brackets in metavars
Files	argparse_wrap_test.py: Test which causes assertion failure.

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-01-17.06:20:12.682>
created_at = <Date 2012-02-18.05:21:48.914>
labels = ['type-bug', 'library']
title = 'argparse: assertion failure if optional argument has square/round brackets in metavar'
updated_at = <Date 2018-06-09.01:28:05.561>
user = 'https://bugs.python.org/oxplot'

bugs.python.org fields:

activity = <Date 2018-06-09.01:28:05.561>
actor = 'miss-islington'
assignee = 'none'
closed = True
closed_date = <Date 2016-01-17.06:20:12.682>
closer = 'martin.panter'
components = ['Library (Lib)']
creation = <Date 2012-02-18.05:21:48.914>
creator = 'oxplot'
dependencies = []
files = ['24554']
hgrepos = []
issue_num = 14046
keywords = []
message_count = 7.0
messages = ['153632', '154234', '245970', '319040', '319049', '319053', '319126']
nosy_count = 6.0
nosy_names = ['ncoghlan', 'bethard', 'paul.j3', 'oxplot', 'John Jones', 'miss-islington']
pr_nums = ['7527', '7528', '7530', '7553']
priority = 'normal'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = '11874'
type = 'behavior'
url = 'https://bugs.python.org/issue14046'
versions = ['Python 2.7']

[oxplot]

I have a web server written in python which takes an optional argument telling it what IP and port to bind to. Here's the definition:

parser.add_argument(
  '-b', '--bind',
  metavar="[ip]:port",
  type=unicode,
  help="IP and port to bind to."
)

There are several other arguments as well. When the usage is printed (due to parse failures or -h), if the command line is longer than terminal width, it is wrapped. Here is the (relevant) code in argparse.py:311 which deals with this scenario:

# wrap the usage parts if it's too long
text_width = self._width - self._current_indent 
if len(prefix) + len(usage) > text_width: 
 
  # break usage into wrappable parts 
  part_regexp = r'\(.*?\)+|\[.*?\]+|\S+' 
  opt_usage = format(optionals, groups) 
  pos_usage = format(positionals, groups) 
  opt_parts = _re.findall(part_regexp, opt_usage) 
  pos_parts = _re.findall(part_regexp, pos_usage) 
  assert ' '.join(opt_parts) == opt_usage
  assert ' '.join(pos_parts) == pos_usage

Note how part_regexp extracts words, and text surrounded with round/square brackets. Now the argument I defined above, when displayed in usage text, looks like this: [-b [ip]:port]

part_regexp however, will cut it into "[-b [ip]" and ":port]" and concatenated later with " ", it will have an extra space which causes the first assertion to fail.

I fiddled with part_regexp but that proved unreliable at best. I think the opt_parts should be obtained, not from the final formatted text, but from actual argument objects.

For a demonstration, see the attachment.

[bethard]

Yes, this is a known bug (bpo-11874). Patches welcome.

[JohnJones]

Im kind of surprised this bug has lasted for so many years :)

setting part_regexp to:
r'\(.*?\(.*?\).*?\)+|\[.*?\[.*?\].*?\]+|\S+'
fixes the issue for me, although its not very elegant

[ncoghlan]

New changeset 66f02aa by Nick Coghlan (wim glenn) in branch 'master':
bpo-11874: fix assertion failure in argparse metavar handling (GH-1826)
66f02aa

[miss-islington]

New changeset 376c272 by Miss Islington (bot) in branch '3.6':
bpo-11874: fix assertion failure in argparse metavar handling (GH-1826)
376c272

[miss-islington]

New changeset 842985f by Miss Islington (bot) in branch '3.7':
bpo-11874: fix assertion failure in argparse metavar handling (GH-1826)
842985f

[miss-islington]

New changeset 4e6bd24 by Miss Islington (bot) in branch '2.7':
bpo-11874: fix assertion failure in argparse metavar handling (GH-1826)
4e6bd24