1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
|
In-Memory Files
===============
Other sections of this documentation have explained how Rasterio can access
data stored in existing files on disk written by other programs or write files
to be used by other GIS programs. Filenames have been the typical inputs and
files on disk have been the typical outputs.
.. code-block:: python
with rasterio.open('example.tif') as dataset:
data_array = dataset.read()
There are different options for Python programs that have streams of bytes,
e.g., from a network socket, as their input or output instead of filenames.
One is the use of a temporary file on disk.
.. code-block:: python
import tempfile
with tempfile.NamedTemporaryFile() as tmpfile:
tmpfile.write(data)
with rasterio.open(tmpfile.name) as dataset:
data_array = dataset.read()
Another is Rasterio's :class:`.MemoryFile`, an abstraction for objects in GDAL's
in-memory filesystem.
MemoryFile: BytesIO meets NamedTemporaryFile
--------------------------------------------
The :class:`.MemoryFile` class behaves a bit like :class:`~io.BytesIO` and
:func:`~tempfile.NamedTemporaryFile`. A GeoTIFF file in a sequence of ``data`` bytes can be
opened in memory as shown below.
.. code-block:: python
from rasterio.io import MemoryFile
with MemoryFile(data) as memfile:
with memfile.open() as dataset:
data_array = dataset.read()
This code can be several times faster than the code using
:func:`~tempfile.NamedTemporaryFile` at roughly double the price in memory.
Writing MemoryFiles
-------------------
Incremental writes to an empty :class:`.MemoryFile` are also possible.
.. code-block:: python
with MemoryFile() as memfile:
while True:
data = f.read(8192) # ``f`` is an input stream.
if not data:
break
memfile.write(data)
with memfile.open() as dataset:
data_array = dataset.read()
These two modes are incompatible: a :class:`.MemoryFile` initialized with a sequence
of bytes cannot be extended.
An empty :class:`.MemoryFile` can also be written to using dataset API methods.
.. code-block:: python
with MemoryFile() as memfile:
with memfile.open(driver='GTiff', count=3, ...) as dataset:
dataset.write(data_array)
Reading MemoryFiles
-------------------
Like :class:`~io.BytesIO`, :class:`.MemoryFile` implements the Python file protocol and
provides :meth:`~.MemoryFile.read`, :meth:`~.MemoryFile.seek`, and :meth:`~.MemoryFile.tell`
methods. Instances are thus suitable as arguments for methods like
`requests.post() <https://requests.readthedocs.io/en/latest/api/#requests.post>`__.
.. code-block:: python
with MemoryFile() as memfile:
with memfile.open(driver='GTiff', count=3, ...) as dataset:
dataset.write(data_array)
requests.post('https://example.com/upload', data=memfile)
|
