File: README.md

package info (click to toggle)
python-blockbuster 1.5.26-1
  • links: PTS, VCS
  • area: main
  • in suites: forky, sid
  • size: 232 kB
  • sloc: python: 888; makefile: 8
file content (276 lines) | stat: -rw-r--r-- 9,219 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
# Blockbuster

Blockbuster is a Python package designed to detect and prevent blocking calls within an asynchronous event loop.
It is particularly useful when executing tests to ensure that your asynchronous code does not inadvertently call blocking operations, 
which can lead to performance bottlenecks and unpredictable behavior.

In Python, the asynchronous event loop allows for concurrent execution of tasks without the need for multiple threads or processes.
This is achieved by running tasks cooperatively, where tasks yield control back to the event loop when they are waiting for I/O operations or other long-running tasks to complete.

However, blocking calls, such as file I/O operations or certain networking operations, can halt the entire event loop, preventing other tasks from running.
This can lead to increased latency and reduced performance, defeating the purpose of using asynchronous programming.

The difficulty with blocking calls is that they are not always obvious, especially when working with third-party libraries or legacy code.
This is where Blockbuster comes in: it helps you identify and eliminate blocking calls in your codebase during testing, ensuring that your asynchronous code runs smoothly and efficiently.
It does this by wrapping common blocking functions and raising an exception when they are called within an asynchronous context.

Notes:
- Blockbuster currently only detects `asyncio` event loops.
- Blockbuster is tested only with CPython. It may work with other Python implementations if it's possible to monkey-patch the functions with `setattr`.

## Installation

The package is named `blockbuster`.
For instance with `pip`:

```bash
pip install blockbuster
```

It is recommended to constrain the version of Blockbuster.
Blockbuster doesn't strictly follow semver. Breaking changes such as new rules added may be introduced between minor versions, but not between patch versions.
So it is recommended to constrain the Blockbuster version on the minor version.
For instance, with `uv`:

```bash
uv add "blockbuster>=1.5.5,<1.6"
```

## Using BlockBuster

### Manually

To activate BlockBuster manually, create an instance of the `BlockBuster` class and call the `activate()` method:

```python
from blockbuster import BlockBuster

blockbuster = BlockBuster()
blockbuster.activate()
```

Once activated, BlockBuster will raise a `BlockingError` exception whenever a blocking call is detected within an `asyncio` event loop.

To deactivate BlockBuster, call the `deactivate()` method:

```python
blockbuster.deactivate()
```

### Using the context manager

BlockBuster can also be activated using a context manager, which automatically activates and deactivates the checks within the `with` block:

```python
from blockbuster import blockbuster_ctx

with blockbuster_ctx():
    # Your test code here
```

### Usage with Pytest

Blockbuster is intended to be used with testing frameworks like `pytest` to catch blocking calls. 
Here's how you can integrate Blockbuster into your `pytest` test suite:

```python
import pytest
import time
from blockbuster import BlockBuster, BlockingError, blockbuster_ctx
from typing import Iterator

@pytest.fixture(autouse=True)
def blockbuster() -> Iterator[BlockBuster]:
    with blockbuster_ctx() as bb:
        yield bb

async def test_time_sleep() -> None:
    with pytest.raises(BlockingError, match="sleep"):
        time.sleep(1)  # This should raise a BlockingError
```

By using the `blockbuster_ctx` context manager, Blockbuster is automatically activated for every test, and blocking calls will raise a `BlockingError`.

## How it works

Blockbuster works by wrapping common blocking functions from various modules (e.g., `os`, `socket`, `time`) and replacing them with versions that check if they are being called from within an `asyncio` event loop.
If such a call is detected, Blockbuster raises a `BlockingError` to indicate that a blocking operation is being performed inappropriately.

Blockbuster supports by default the following functions and modules:

- **Time Functions**:
  - `time.sleep`
- **OS Functions**:
  - `os.getcwd`
  - `os.statvfs`
  - `os.sendfile`
  - `os.rename`
  - `os.remove`
  - `os.unlink`
  - `os.mkdir`
  - `os.rmdir`
  - `os.link`
  - `os.symlink`
  - `os.readlink`
  - `os.listdir`
  - `os.scandir`
  - `os.access`
  - `os.stat`
  - `os.replace`
  - `os.read`
  - `os.write`
- **OS path Functions**:
  - `os.path.ismount`
  - `os.path.samestat`
  - `os.path.sameopenfile`
  - `os.path.islink`
  - `os.path.abspath`
- **IO Functions**:
  - `io.BufferedReader.read`
  - `io.BufferedWriter.write`
  - `io.BufferedRandom.read`
  - `io.BufferedRandom.write`
  - `io.TextIOWrapper.read`
  - `io.TextIOWrapper.write`
- **Socket Functions**:
  - `socket.socket.connect`
  - `socket.socket.accept`
  - `socket.socket.send`
  - `socket.socket.sendall`
  - `socket.socket.sendto`
  - `socket.socket.recv`
  - `socket.socket.recv_into`
  - `socket.socket.recvfrom`
  - `socket.socket.recvfrom_into`
  - `socket.socket.recvmsg`
  - `ssl.SSLSocket.write`
  - `ssl.SSLSocket.send`
  - `ssl.SSLSocket.read`
  - `ssl.SSLSocket.recv`
- **SQLite Functions**:
  - `sqlite3.Cursor.execute`
  - `sqlite3.Cursor.executemany`
  - `sqlite3.Cursor.executescript`
  - `sqlite3.Cursor.fetchone`
  - `sqlite3.Cursor.fetchmany`
  - `sqlite3.Cursor.fetchall`
  - `sqlite3.Connection.execute`
  - `sqlite3.Connection.executemany`
  - `sqlite3.Connection.executescript`
  - `sqlite3.Connection.commit`
  - `sqlite3.Connection.rollback`
- **Thread lock Functions**:
  - `threading.Lock.acquire`
  - `threading.Lock.acquire_lock`
- **Built-in Functions**:
  - `input`

Some exceptions to the rules are already in place:
- Importing modules does blocking calls as it interacts with the file system. Since this operation is cached and very hard to avoid, it is excluded from the detection.
- Blocking calls done by the `pydevd` debugger.
- Blocking calls done by the `pytest` framework.

## Customizing Blockbuster

### Adding custom rules

Blockbuster is not a silver bullet and may not catch all blocking calls.
In particular, it will not catch blocking calls that are done by third-party libraries that do blocking calls in C extensions.
For these third-party libraries, you can declare your own custom rules to Blockbuster to catch these blocking calls.

Eg.:
```python
from blockbuster import BlockBuster, BlockBusterFunction
import mymodule

blockbuster = BlockBuster()
blockbuster.functions["my_module.my_function"] = BlockBusterFunction(my_module, "my_function")
blockbuster.activate()
```

Note: if blockbuster has already been activated, you will need to activate the custom rule yourself.

```python
from blockbuster import blockbuster_ctx, BlockBusterFunction
import mymodule

with blockbuster_ctx() as blockbuster:
    blockbuster.functions["my_module.my_function"] = BlockBusterFunction(my_module, "my_function")
    blockbuster.functions["my_module.my_function"].activate()
```

### Allowing blocking calls in specific contexts

You can customize Blockbuster to allow blocking calls in specific functions by using the `can_block_in` method of the `BlockBusterFunction` class.
This method allows you to specify exceptions for particular files and functions where blocking calls are allowed.

```python
from blockbuster import BlockBuster

blockbuster = BlockBuster()
blockbuster.activate()
blockbuster.functions["os.stat"].can_block_in("specific_file.py", {"allowed_function"})
```

### Deactivating specific checks

If you need to deactivate specific checks, you can directly call the `deactivate` method on the corresponding `BlockBusterFunction` instance:

```python
from blockbuster import BlockBuster

blockbuster = BlockBuster()
blockbuster.activate()
blockbuster.functions["socket.socket.connect"].deactivate()
```

## Contributing

Contributions are welcome! If you encounter any issues or have suggestions for improvements, please open an issue on the GitHub repository.

### Development Setup

Blockbuster uses `uv` to manage its development environment.
See the [uv documentation](https://docs.astral.sh/uv/getting-started/installation/) for more informationand how to install it.

To install the required dependencies, run the following command:

```bash
uv sync
```

### Running Tests

Tests are written using `pytest`.
To run the tests, use the following command:

```bash
uv run pytest
```

### Code Formatting

Code formatting is done using `ruff`. To format the code, run the following command:

```bash
uv run ruff format
```

### Code Linting

Code linting is done using `ruff`.
To lint the code, fixing any issues that can be automatically fixed, run the following command:

```bash
uv run ruff check --fix
```

## License

This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.

## Acknowledgments

Blockbuster uses the [forbiddenfruit](https://clarete.li/forbiddenfruit/) library to monkey-patch CPython immutable builtin functions and methods.

Blockbuster was greatly inspired by the [BlockHound](https://github.com/reactor/BlockHound) library for Java, which serves a similar purpose of detecting blocking calls in JVM reactive applications.