1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195
|
# asyncinject
[](https://pypi.org/project/asyncinject/)
[](https://github.com/simonw/asyncinject/releases)
[](https://github.com/simonw/asyncinject/blob/main/LICENSE)
Run async workflows using pytest-fixtures-style dependency injection
## Installation
Install this library using `pip`:
$ pip install asyncinject
## Usage
This library is inspired by [pytest fixtures](https://docs.pytest.org/en/6.2.x/fixture.html).
The idea is to simplify executing parallel `asyncio` operations by allowing them to be defined using a collection of functions, where the function arguments represent dependent functions that need to be executed first.
The library can then create and execute a plan for executing the required functions in parallel in the most efficient sequence possible.
Here's an example, using the [httpx](https://www.python-httpx.org/) HTTP library.
```python
from asyncinject import Registry
import httpx
async def get(url):
async with httpx.AsyncClient() as client:
return (await client.get(url)).text
async def example():
return await get("http://www.example.com/")
async def simonwillison():
return await get("https://simonwillison.net/search/?tag=empty")
async def both(example, simonwillison):
return example + "\n\n" + simonwillison
registry = Registry(example, simonwillison, both)
combined = await registry.resolve(both)
print(combined)
```
If you run this in `ipython` or `python -m asyncio` (to enable top-level await in the console) you will see output that combines HTML from both of those pages.
The HTTP requests to `www.example.com` and `simonwillison.net` will be performed in parallel.
The library notices that `both()` takes two arguments which are the names of other registered `async def` functions, and will construct an execution plan that executes those two functions in parallel, then passes their results to the `both()` method.
### Registry.from_dict()
Passing a list of functions to the `Registry` constructor will register each function under their introspected function name, using `fn.__name__`.
You can set explicit names instead using a dictionary:
```python
registry = Registry.from_dict({
"example": example,
"simonwillison": simonwillison,
"both": both
})
```
Those string names will be used to match parameters, so each function will need to accept parameters named after the keys used in that dictionary.
### Registering additional functions
Functions that are registered can be regular functions or `async def` functions.
In addition to registering functions by passing them to the constructor, you can also add them to a registry using the `.register()` method:
```python
async def another():
return "another"
registry.register(another)
```
To register them with a name other than the name of the function, pass the `name=` argument:
```python
async def another():
return "another 2"
registry.register(another, name="another_2")
```
### Resolving an unregistered function
You don't need to register the final function that you pass to `.resolve()` - if you pass an unregistered function, the library will introspect the function's parameters and resolve them directly.
This works with both regular and async functions:
```python
async def one():
return 1
async def two():
return 2
registry = Registry(one, two)
# async def works here too:
def three(one, two):
return one + two
print(await registry.resolve(three))
# Prints 3
```
### Parameters are passed through
Your dependent functions can require keyword arguments which have been passed to the `.resolve()` call:
```python
async def get_param_1(param1):
return await get(param1)
async def get_param_2(param2):
return await get(param2)
async def both(get_param_1, get_param_2):
return get_param_1 + "\n\n" + get_param_2
combined = await Registry(get_param_1, get_param_2, both).resolve(
both,
param1 = "http://www.example.com/",
param2 = "https://simonwillison.net/search/?tag=empty"
)
print(combined)
```
### Parameters with default values are ignored
You can opt a parameter out of the dependency injection mechanism by assigning it a default value:
```python
async def go(calc1, x=5):
return calc1 + x
async def calc1():
return 5
print(await Registry(calc1, go).resolve(go))
# Prints 10
```
### Tracking with a timer
You can pass a `timer=` callable to the `Registry` constructor to gather timing information about executed tasks.. Your function should take three positional arguments:
- `name` - the name of the function that is being timed
- `start` - the time that it started executing, using `time.perf_counter()` ([perf_counter() docs](https://docs.python.org/3/library/time.html#time.perf_counter))
- `end` - the time that it finished executing
You can use `print` here too:
```python
combined = await Registry(
get_param_1, get_param_2, both, timer=print
).resolve(
both,
param1 = "http://www.example.com/",
param2 = "https://simonwillison.net/search/?tag=empty"
)
```
This will output:
```
get_param_1 436633.584580685 436633.797921747
get_param_2 436633.641832699 436634.196364347
both 436634.196570217 436634.196575639
```
### Turning off parallel execution
By default, functions that can run in parallel according to the execution plan will run in parallel using `asyncio.gather()`.
You can disable this parallel exection by passing `parallel=False` to the `Registry` constructor, or by setting `registry.parallel = False` after the registry object has been created.
This is mainly useful for benchmarking the difference between parallel and serial execution for your project.
## Development
To contribute to this library, first checkout the code. Then create a new virtual environment:
cd asyncinject
python -m venv venv
source venv/bin/activate
Now install the dependencies and test dependencies:
pip install -e '.[test]'
To run the tests:
pytest
|