File: THOUGHTS.rst

package info (click to toggle)
python-invoke 1.4.1%2Bds-0.1~bpo10%2B1
  • links: PTS, VCS
  • area: main
  • in suites: buster-backports
  • size: 1,704 kB
  • sloc: python: 11,377; makefile: 18; sh: 12
file content (202 lines) | stat: -rw-r--r-- 6,621 bytes parent folder | download | duplicates (3)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
==============================================
Random thoughts unsuitable for public docs yet
==============================================

CLI type mapping
================

Some loose thoughts on bridging the "shell is strings, Python wants
lists/dicts/integers/bools/etc" problem.

Methodologies
-------------

* Explicit mapping, as with ``argparse``: this particular flag turns into a
  list/boolean/int/whatever. Because we're specifically mapping to function
  keyword arguments, a little of that complexity can be removed, but generally
  it'll look very similar. E.g.::

    @args(foo=int)
    def mytask(foo):
        ...

  would turn this::

    $ invoke mytask --foo 7

  into ``7``, not ``"7"``.
* Introspection-based mapping, i.e. introspecting the default values of a
  function signature and automatically transforming the CLI input. E.g.::

    def mytask(foo=5):
        ...

  invoked as::

    $ invoke mytask --foo 7

  results in the Python value ``7`` instead of ``"7"``, just as with the
  explicit example above.
* Formatting-based mapping, i.e. having (optional) conventions in the string
  format of an incoming flag argument that cause transformations to occur.
  E.g. we could say that commas in an argument automatically trigger
  transformation into a list of strings; thus the invocation::

    $ invoke mytask --items a,b,c

  would on the Python end turn into a call like this::

    mytask(items=['a', 'b', 'c'])

What to do?
~~~~~~~~~~~

We haven't decided exactly how many of these to use -- we may end up using all
three of them as appropriate, with some useful/sensible default and the option
to enable/disable things for power users. The trick is to balance
power/features with becoming overly complicated to understand or utilize.

Other types
-----------

Those examples cover integers/numbers, and lists/iterables. Strings are
obviously easy/the default. What else is there?

* Booleans: these are relatively simple too, either a flag exists (``True``) or
  is omitted (``False``).
  
    * Could also work in a ``--foo`` vs ``--no-foo`` convention to help with
      the inverse, i.e. values which should default to ``True`` and then need
      to be turned "off" on the command line. E.g.::

        def mytask(option=True):
            ...

      could result in having a flag called ``--no-option`` instead of
      ``--option``. (Or possibly both.)

* Dicts: these are tougher, but we could potentially use something like::

    $ invoke mytask --dictopt key1=val1,key2=val2

  resulting in::

    mytask(dictopt={'key1': 'val1', 'key2': 'val2'})


Parameterizing tasks
====================

Old "previous example" (at time the below was split out of live docs, the
actual previous example had been changed a lot and no longer applied)::

    $ invoke test --module=foo test --module=bar
    Cleaning
    Testing foo
    Cleaning
    Testing bar

The previous example had a bit of duplication in how it was invoked; an
intermediate use case is to bundle up that sort of parameterization into a
"meta" task that itself invokes other tasks in a parameterized fashion.

TK: API for this? at CLI level would have to be unorthodox invocation, e.g.::

    @task
    def foo(bar):
        print(bar)

    $ invoke --parameterize foo --param bar --values 1 2 3 4
    1
    2
    3
    4

Note how there's no "real" invocation of ``foo`` in the normal sense. How to
handle partial application (e.g. runtime selection of other non-parameterized
arguments)? E.g.::

    @task
    def foo(bar, biz):
        print("%s %s" % (bar, biz))

    $ invoke --parameterize foo --param bar --values 1 2 3 4 --biz "And a"
    And a 1
    And a 2
    And a 3
    And a 4

That's pretty clunky and foregoes any multi-task invocation. But how could we
handle multiple tasks here? If we gave each individual task flags for this,
like so::

    $ invoke foo --biz "And a" --param foo --values 1 2 3 4

We could do multiple tasks, but then we're stomping on tasks' argument
namespaces (we've taken over ``param`` and ``values``). Really hate that.

**IDEALLY** we'd still limit parameterization to library use since it's an
advanced-ish feature and frequently the parameterization vector is dynamic (aka
not the sort of thing you'd give at CLI anyway)

Probably best to leave that in the intermediate docs and keep it lib level;
it's mostly there for Fabric and advanced users, not something the average
Invoke-only user would care about. Not worth the effort to make it work on CLI
at this point.

::

    @task
    def stuff(var):
        print(var)

    # NOTE: may need to be part of base executor since Collection has to know
    # to pass the parameterization option/values into Executor().execute()?
    class ParameterizedExecutor(Executor):
        # NOTE: assumes single dimension of parameterization.
        # Realistically would want e.g. {'name': [values], ...} structure and
        # then do cross product or something
        def execute(self, task, args, kwargs, parameter=None, values=None):
            # Would be nice to generalize this?
            if parameter:
                # TODO: handle non-None parameter w/ None values (error)
                # NOTE: this is where parallelization would occur; probably
                # need to move into sub-method
                for value in values:
                    my_kwargs = dict(kwargs)
                    my_kwargs[parameter] = value
                    super(self, ParameterizedExecutor).execute(task, kwargs=my_kwargs)
            else:
                super(self, ParameterizedExecutor).execute(task, args, kwargs)


Getting hairy: one task, with one pre-task, parameterized
=========================================================

::

    @task
    def setup():
        print("Yay")

    @task(pre=[setup])
    def build():
        print("Woo")

    class OhGodExecutor(Executor):
        def execute(self, task, args, kwargs, parameter, values):
            # assume always parameterized meh
            # Run pretasks once only, instead of once per parameter value
            for pre in task.pre:
                self.execute(self.collection[pre])
            for value in values:
                my_kwargs = dict(kwargs)
                my_kwargs[parameter] = value
                super(self, OhGodExecutor).execute(task, kwargs=my_kwargs)


Still hairy: one task, with a pre-task that itself has a pre-task
=================================================================

All the things: two tasks, each with pre-tasks, both parameterized
==================================================================