argparse add_mutually_exclusive_group should accept existing arguments to register conflicts

[gotgenes]

BPO	10984
Files	args_in_multiple_mutually_exclusive_groups.patch: Allow arguments to be passed into add_mutually_exclusive_group multigroup_1.patch multigroup_4.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2011-01-22.23:01:55.200>
labels = ['type-feature', 'library']
title = 'argparse add_mutually_exclusive_group should accept existing arguments to register conflicts'
updated_at = <Date 2014-06-03.05:44:01.029>
user = 'https://bugs.python.org/gotgenes'

bugs.python.org fields:

activity = <Date 2014-06-03.05:44:01.029>
actor = 'paul.j3'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2011-01-22.23:01:55.200>
creator = 'gotgenes'
dependencies = []
files = ['30818', '30921', '30940']
hgrepos = []
issue_num = 10984
keywords = ['patch']
message_count = 13.0
messages = ['126862', '126879', '192444', '192765', '192844', '192954', '193071', '193184', '196771', '197960', '212458', '212482', '219660']
nosy_count = 5.0
nosy_names = ['bethard', 'gotgenes', 'micktwomey', 'paul.j3', 'jamadagni']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue10984'
versions = ['Python 3.3']

[gotgenes]

argparse supports registering conflicting arguments, however, it does so in a way that an argument may belong to at most one group of conflicting arguments. The inspiration for this bug is Stack Overflow question #4770576.
http://stackoverflow.com/questions/4770576/does-argparse-python-support-mutually-exclusive-groups-of-arguments

The most straightforward use case argparse can not accommodate is the following: the user has three flags, '-a', '-b', and '-c'. The flag '-b' is incompatible with both '-a' and with '-c', however, '-a' and '-c' are compatible with each other.

Current practice is to register a conflict by first defining a conflict group with parser.add_mutually_exclusive_group(), and then create new arguments within that group using group.add_argument(). Because the programmer is not allowed to create the argument prior to creating the group, an argument cannot be registered in two exclusive groups.

I feel it would be much more useful to be given the option to create exclusive groups after the programmer has defined and created the options, as is the design for ConflictsOptionParser
http://pypi.python.org/pypi/ConflictsOptionParser/

[bethard]

I'm definitely open to providing such functionality. I assume you're imagining something like:

parser = argparse.ArgumentParser()
a_action = parser.add_argument('-a')
b_action = parser.add_argument('-b')
c_action = parser.add_argument('-c')
d_action = parser.add_argument('-d')
parser.add_mutually_exclusive_group(a_action, c_action)
parser.add_mutually_exclusive_group(a_action, d_action)
...

If you can supply a patch, I'll take a look at it.

[micktwomey]

I've picked up on this as part of the EP 2013 sprints.

I've attached a patch which implements the behaviour described in the comments. It doesn't break any existing functionality but the help generated by argparse is definitely not quite right.

[paulj3]

This approach of simply adding the existing actions to the group's _group_actions works fine, at least when it comes catching the error.

It may be difficult to get a useful usage line. In usage, arguments appear in the order in which they were created, optionals first, positionals after. Group notation is added if the subset of the arguments appear in its list in the same order.

In the patch .rst,

usage: PROG [-h] (--foo | --bar) [--baz]

the foo,bar group is marked correctly; the foo,baz group is not contiguous and is omited.

In bethard's example neither group will be marked
(class TestMutuallyExclusiveGroupWithExistingArguments)

But the problem isn't just with adding existing arguments.

class TestMutuallyExclusiveOptionalsMixed illustrates this with a case where group and parser arguments overlap.

In class TestMutuallyExclusiveOptionalsAndPositionalsMixed, the mix of optionals and positionals makes group marking impossible.

If the groups are in order, but overlap, usage can be a confusing mix

Groups ab, bc, cd, produce:
[-a A | [-b B | [-c C | -d D]

But if created in a different order, the usage can be:
[-a A | [-b B | -c C] -d D]

So there are 2 issues

• if groups are not continuous or overlap, what is a meaningful usage?
• any choice is likely to require a major reworking of the formatting logic.

Since confusing group markings are worse than none, a first step might be to flag a group added via this patch as 'do not mark'. Also add a note to the documentation that user may need to write their own grouping instructions (in usage, description or epilog).

[paulj3]

One usage option is to make a subclass of HelpFormatter (the accepted way of customizing a formatter), and write a function that formats each group independently. For the example case, the resulting format might be:

usage: PROG [-h] [-b] [-a | -c] [-a | -d]

-h and -b are not part of any group. These are followed by the two groups. -a is repeated because it appears in both groups.

[paulj3]

While playing with some examples, I found that exclusive group formatting has another failure case. If the usage line is long enough to wrap, optionals and positionals are formatted separately, with positionals appearing on a separate line(s). That means that if a group includes a positional, it will not be marked.

So (shortening lines for convenience sake), instead of:

usage: [-h] ... (-a | -b | x)

we get

usage: [-h] ... [-a] [-b]
       x

This is true even if arguments are added to the group in the normal way.

[paulj3]

This patch adds a MultiGroupHelpFormatter that formats groups even if they share actions or the actions are not in the same order as in the parse._actions list. It sorts the groups so positional actions, if any appear in the correct order.

A long test case generates this help:

usage: PROG [-h] [-a A | -c C] [-a A | -d D] [-a A | -b B] [-b B | -d D]
            [-d D | x] foo [-b B | y]

positional arguments:
  x           x help
  foo         foo help        
  y           y help

optional arguments:
  -h, --help  show this help message and exit
  -a A        a help
  -b B        b help
  -c C        c help
  -d D        d help

In the 2nd usage line, the 2 groups, and action foo, are shown in the order in which x, foo, y were defined (and hence will be parsed), even though the groups were not defined in that order.

The default formatter could not format these groups, generating '[-h] [-a A] [-b B] ... x foo y' instead.

I have included the latest patch from http://bugs.python.org/issue11874. This splits the usage line generated by _format_actions_usage into parts that are groups or independent actions. The goal there is to correctly split long usage lines into multiple lines. Here it makes it easier to format groups and actions in new order.

If existing actions are added to new group as in the original patch for this issue, that group gets a no_usage = True attribute. The default formatter then will not attempt to format this group. The MultiGroupHelpFormatter ignores this attribute.

This patch needs better documentation. Test cases also need refinement, improving the names, and eliminating redundancies. Some of the new tests are copies of existing ones, but using the new formatter.

[paulj3]

This patch produces the same usage as before, but I have rewritten _format_actions_usage() for both HelpFormatter and MultiGroupFormater.

The original HelpFormatter._format_actions_usage() formats the actions, splices in group markings, cleans up the text, if needed, tries to break it down into parts. But this is fragile, as shown here and in issues 11874, 18349).

Now _format_group_usage() and _format_just_actions_usage() format groups and actions directly, without the splice and divide steps. _format_actions_usage() for both classes call these to build a list of usage parts.

This change also solves http://bugs.python.org/issue11874 and http://bugs.python.org/issue18349, since it does not have to break up a formatted text line (and in the process get confused by [] and ()).

[paulj3]

A possible further tweak is, in take_action(), test for conflicts before adding the action to 'seen_non_default_actions'

            if argument_values is not action.default:
                #seen_non_default_actions.add(action)
                for conflict_action in action_conflicts.get(action, []):
                    if conflict_action in seen_non_default_actions:
                       ...
                seen_non_default_actions.add(action)

This does not cause problems with any existing tests, but makes it possible to add an action twice to a group. Why do that? To prevent an action from occurring more than once. For some actions like 'count' and 'append' repeated use is expected, but for others it isn't expected, and may sometimes be a nuisance (the last occurrence is the one that sticks).

An example use would be:

    parser = argparse.ArgumentParser(prog="PROG",
        formatter_class=argparse.MultiGroupHelpFormatter)
    action = parser.add_argument('--arg', help='use this argument only once')
    group1 = parser.add_mutually_exclusive_group(action, action)
    args  = parser.parse_args()

calling this with:

python3 test_once.py --arg test --arg next

would produce this error message:

usage: PROG [-h] [--arg ARG | --arg ARG]
PROG: error: argument --arg: not allowed with argument --arg

The usage and error message aren't as clear as they might be if this feature was added 'from scratch'. But for a minor change like this, that may be an acceptable price.

[paulj3]

Another situation in which this MultiGroupHelpFormatter helps is when one or more of the groups includes an optional positional.

The regular formatter moves all the positionals to the end, after the optionals. This move could easily break up a mutually exclusive group, and make formatting it impossible. But the MultiGroupHelpFormatter gives the group order priority.

Thus for example:

    p=argparse.ArgumentParser()
    # (formatter_class=argparse.MultiGroupHelpFormatter)
    g=p.add_mutually_exclusive_group()
    g.add_argument('-f')
    g.add_argument('foo',nargs='?')
    g=p.add_mutually_exclusive_group()
    g.add_argument('-b')
    g.add_argument('-c')
    g.add_argument('bar',nargs='*',default='X')
    print(p.format_usage())

produces (positionals at end, no group markings)

usage: PROG [-h] [-f F] [-b B] [-c C] [foo] [bar [bar ...]]

But the MultiGroupHelpFormatter produces:

usage: PROG [-h] [-f F | foo] [-b B | -c C | bar [bar ...]]

In this last case, the positionals are listed with their respective groups, and the groups are ordered by the relative ordering of the positionals.

[jamadagni]

I also wish to see argparse allowing me to define a group of arguments that conflict with another argument or another group of arguments and FWIW I feel the help output should be like:

prog [ --conflicter | [ --opt1 ] [ --opt2 ] ]

where --conflicter conflicts with --opt1 and --opt2 but those two don't conflict with each other and all are optional.