Allow the easy creation of executable utilities with invoke.

[sophacles]

Description

Splitting this ticket from #170 at the request of @bitprophet

It would be nice to easily create scripts that I can place in my $PATH that internally use invoke, but are named according to script functionality. Some rational (copied from referenced ticket discussion):

I could probably train myself always to do:

inv -c /path/to/source/tree/utils/$TASK ....

But I kind of like having them:

1. as a script available in my $PATH
2. as tools named cognitively, just like when I'm using busybox... i just link (e.g) ls and cp rather than calling busybox ls ... or busybox cp file1 file2
3. in a consistent place (depending on what I'm doing, the source tree where the task files live can vary. It's in one place on my dev machine, and different places in deployment depending on what OS, nfs details, and so on. We're still working out our "one true deployment strategy" so this changes as stuff comes up).

Also noteworthy, several of my targets are most commonly called from places in the system that are not the directory where invoke lives.

Currently I do this via the code in this gist: https://gist.github.com/sophacles/c17ce33a14d582dfa268 and creating scripts entry points in a setup.py.

---

To-do list

[Note: this part of the ticket is being maintained by @bitprophet]

• Parameterization of version number by itself (not name+version as we currently do)
• Parameterization of tool/package name, defaulting to the name seen in argv[0]
  • Capitalized in --version output, e.g. Invoke 1.2
  • Allowance for arbitrary additional lines of version info, e.g. for Fabric wanting to display its + Paramiko's, and even e.g. other data like Python version?
  • Left as-is in --help output, e.g. inv[oke] --core-opts ...
  • Is there an automated way to determine inv vs invoke vs etc? We don't have access to setuptools' data at runtime, so we'll probably have to provide this manually. (Maybe stored in similar fashion as version - reused in both spots?)
• Ability to update global --help to include elements of --list, i.e. for tools distributing with built-in tasks as subcommands (which would be...most of them...Fabric is actually unusual in this regard).
  • I.e. probably sticking --list type output above the core flags, which is what most real subcommand based CLI apps tend to do (see git, pip etc)
• Fix up per-task help to honor the dynamic program name (it's hardcoded right now, as above)
• Ability to override which flags are presented and/or their default values

Cleanup:

• Check all the task modules in _support to remove any no longer in use (& consolidate some if possible)
• Check all methods in _utils.py for whether they've gone out of use
• Ensure docs build & the docstrings recently added look good rendered

[bitprophet]

As I think I mentioned in #170 this basically just translates to "make the existing cli.py even easier to reuse".

[sophacles]

Anyone looking do this right now, until it can be properly included in invoke, can do what I do at this gist: https://gist.github.com/sophacles/fc6bf504af1d4e3d4485

It uses some introspection magic to figure out where the dispatch_me() call is coming from, and re-works the command line before dispatching to invoke. It's not perfect, nor pretty, but it gets the job done for now.

[bitprophet]

I was kinda wanting this just the other day, in a one-off fashion (i.e. a single task as a 'binary' - for now I just run invoke directly but it'd be nice to distribute as a single binstub) though obviously having a full collection available as per earlier comments here is equally valid.

Suspect the real driver for this will be my ongoing fabric 2 work (providing a fab binary that adds/changes core flags as well as being renamed), so might happen sooner instead of later.

[bitprophet]

I don't see a note here about it, but at some point we did do a TINY bit of work towards this by allowing users to override --version output,

invoke/invoke/cli.py

Lines 268 to 271 in 6aeffbd

	if version:
	print(version)
	else:
	print("Invoke {0}".format(__version__))

- I think it was coderanger who requested it.

Whatever we do for this (which will be Soon now because I need it, again) should tickle that too, or replace it.

[bitprophet]

Also noticed there's a bug of sorts, doing core help (inv --help) obtains a dynamic program name based on argv[0], which is nice, as it means simply calling invoke.cli.main from another package's CLI entrypoint works correctly. However, it a) ignores the "name + version" bit passed in to dispatch, and b) isn't reflected in per-task help, despite the 'header' of the two help outputs being similar otherwise.

Updated ticket desc with a checklist & more thoughts.

[bitprophet]

Realized as I started writing docs for this that the truly common case isn't Fabric, which still cares about exposing the idea of arbitrary tasks to be run, but a program where the distributed task tree are the ONLY tasks ever to be run, because they're effectively subcommands.

In that scenario, the default split between --help and --list is unhelpful at best and confusing at worst. We really want the ability for --help to show both the task tree and core options, think pip / git help output.

Having updates to help output was already required here but this is a more involved version of it. Hopefully not too hard, though.

[bitprophet]

One open question, do we refer to "baked-in" tasks as "commands" or "subcommands"? Most programs seem to use the terminology "commands", but for some reason my instinct was to refer to them as "subcommands" (since the program itself is "the command").

[bitprophet]

Have been doing a metric shitton of implementation/refactoring towards Program these past few days, it's shaping up pretty nicely and as a side effect has cleaned up the relevant existing tests at least a little bit (sometimes moderately). Most of the way there.

[sophacles]

Looking at the branch for this, I'm pretty excited about it landing. Fantastic work as usual @bitprophet!

As for the open question, I'm +1 on subcommands being the terminology, as that is how my documentation refers to them where I use my hack mentioned above.

[bitprophet]

Thanks for the feedback @sophacles :) glad you're watching since I was going to ask you your opinion on the API once I had it working well enough.

Currently bedeviled by the old "do I write the class as a series of bundled functional subroutines passing data around (i.e. classes as code reuse only), or do I shovel all that data into self.xyz (i.e. classes as state keepers too)" question. I've gone with the latter because it simplifies a lot of niggly bits in the implementation, but it does make targeted unit-style testing harder & is less "appropriate" given Program's intended API is very short-lived - having one around for more than a few lines is unlikely.

Then again, I waffle a lot on just how unit-y to make the tests for this module, since much of the behavior is only interesting at a high level ("did giving X argv result in triggering Y behavior just prior to exit?") and unit testing specific methods doesn't ensure they're even called by run.

So...having to stick with "call Program().run() and examine result" for some tests is not a huge loss really.

[bitprophet]

cli.py is dead, long live cli.py \o/ all tests pass again. Back to the checklist...