[WIP] Add automatic CLI generation from GW tasks

[kotfic]

This PR relies on girder/girder_worker_utils#30 and provides a proof of concept implementation that relies on a terrible argument metadata API for auto-generation of a girder worker CLI. Consider the following girder worker task:

import click
from girder_worker.app import app
from girder_worker_utils.decorators import parameter


@parameter('a', cli_opts={'type': click.INT})
@parameter('b', cli_opts={'type': click.INT})
@parameter('zero',
           cli_args=('-z', '--zero'),
           cli_opts={'type': click.BOOL,
                     'help': 'return "undefined" if dividing by zero'})
@app.task(bind=True)
def divide(self, a, b, zero=False):
    if zero and b == 0:
        return 'undefined'

    return a / b

This implements a gwrun command that automatically discovers installed tasks and provides a click based application. E.g.:

⇒ gwrun --help
Usage: gwrun [OPTIONS] COMMAND [ARGS]...

Options:
  -o, --output [json|stdout]
  --help                      Show this message and exit.

Commands:
  divide    Proxy that evaluates object once.

And

 ⇒ gwrun divide --help
Usage: gwrun divide [OPTIONS] A B

  Proxy that evaluates object once.

  :class:`Proxy` will evaluate the object each time, while the promise will
  only evaluate it once.

Options:
  -z, --zero BOOLEAN  return "undefined" if dividing by zero
  --help              Show this message and exit.

Finally

⇒ gwrun divide 2 2     
1.0

The cli_args and cli_opts implementation (which is obviously ugly and confusing) are designed to be the low-level API on which a more intuitive and "intelligent" API may be built. For instance:

@parameter('a', type=float)
@parameter('b', type=float)
@app.task
def add(a, b):
    return a + b

represents a much nicer API and a mapping could be used to negotiate the difference between type=float and cli_opts={'type': click.FLOAT} behind the scenes. This serves the purpose of:

1. Improving the readability of the API for the majority usecase
2. Supporting a similar abstraction for handling automatic generation of a web UI

Note that this also attempts to implement basic output handling whereby the return value of the function may be directly printed to stdout (e.g. __repr__) or serialized to json before being printed.

[kotfic]

To test this I recommend the following:

mkdir gwruntest; cd gwruntest

# Make a virtualenv
mkvirtualenv -a . gwruntest

# Clone the repos
git clone -b gwrun  [email protected]:girder/girder_worker.git
git clone -b gwrun_argspec [email protected]:girder/girder_worker_utils.git
git clone https://github.com/kotfic/gw_math_tasks.git

# Install the repos
pip install -e girder_worker_utils
pip install -e girder_worker
pip install -e gw_math_tasks

# Test the command
gwrun add 2 2
gwrun --help

Note there is an implementation of a toy plugin at kotfic/gw_math_tasks

[jbeezley]

Why not loop over description.positional_args and description.keyword_args?

[jbeezley]

I get the convenience of just using the task names, but I do think it will be necessary to at least give the user's a signal about what plugin added the task.

[jbeezley]

From a high level, this looks pretty nice. I think I want to try something non-trivial with it before final judgement though.