1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
|
Metadata-Version: 2.1
Name: iterable-io
Version: 1.0.0
Summary: Adapt generators and other iterables to a file-like interface
Home-page: https://github.com/pR0Ps/iterable-io
License: LGPLv3
Project-URL: Source, https://github.com/pR0Ps/iterable-io
Project-URL: Changelog, https://github.com/pR0Ps/iterable-io/blob/master/CHANGELOG.md
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
Requires-Python: >=3.5
Description-Content-Type: text/markdown
iterable-io
===========
[](https://github.com/pR0Ps/iterable-io/actions/workflows/tests.yml)
[](https://pypi.org/project/iterable-io/)
`iterable-io` is a small Python library that provides an adapter so that it's possible to read from
[iterable](https://docs.python.org/3/glossary.html#term-iterable) objects in the same way as
[file-like](https://docs.python.org/3/glossary.html#term-file-object) objects.
It is primarily useful as "glue" between two incompatible interfaces. As an example, in the case
where one interface expects a file-like object to call `.read()` on, and the other only provides a
generator of bytes.
One way to solve this issue would be to write all the bytes in the generator to a temporary file,
then provide that file instead, but if the generator produces a large amount of data then this is
both slow to start, and resource-intensive.
This library allows streaming data between these two incompatible interfaces so as data is requested
by `.read()`, it's pulled from the iterable. This keeps resource usage low and removes the startup
delay.
Installation
------------
```
pip install iterable-io
```
Documentation
-------------
The functionality of this library is accessed via a single function: `open_iterable()`.
`open_iterable()` is designed to work the same was as the builtin `open()`, except that it takes an
iterable to "open" instead of a file. For example, it can open the iterable in binary or text mode,
has options for buffering, encoding, etc. See the docstring of `open_iterable` for more detailed
documentation.
Simple examples
---------------
The following examples should be enough to understand in which cases `open_iterable()` would be
useful and get a high-level understanding of how to use it:
Read bytes from a generator of bytes:
```python
gen = generate_bytes()
# adapt the generator to a file-like object in binary mode
# (fp.read() will return bytes)
fp = open_iterable(gen, "rb")
while chunk := fp.read(4096):
process_chunk(chunk)
```
Read lines of text from a generator of bytes:
```python
gen = generate_bytes()
# adapt the generator to a file-like object in text mode
# (fp.read() will return a string, fp.readline is also available)
fp = open_iterable(gen, "rt", encoding="utf-8")
for line in fp:
process_line_of_text(line)
```
Tests
-----
This package contains extensive tests. To run them, install `pytest` (`pip install pytest`) and run
`py.test` in the project directory.
License
-------
Licensed under the [GNU LGPLv3](https://www.gnu.org/licenses/lgpl-3.0.html).
|
