1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
|
# xphyle: extraordinarily simple file handling
[](https://pypi.python.org/pypi/xphyle)
[](https://travis-ci.org/jdidion/xphyle)
[](https://coveralls.io/github/jdidion/xphyle?branch=master)
[](https://www.codacy.com/app/jdidion/xphyle?utm_source=github.com&utm_medium=referral&utm_content=jdidion/xphyle&utm_campaign=Badge_Grade)
[](http://xphyle.readthedocs.io/en/latest/?badge=latest)
[](https://zenodo.org/badge/latestdoi/71260678)
[](http://joss.theoj.org/papers/10.21105/joss.00255)
<img src="https://github.com/jdidion/xphyle/blob/master/docs/logo.png?raw=true"
alt="logo" width="200" height="200">
xphyle is a small python library that makes it easy to open compressed
files. Most importantly, xphyle will use the appropriate program (e.g. 'gzip') to compress/decompress a file if it is available on your system; this is almost always faster than using the corresponding python library. xphyle also provides methods that simplify common file I/O operations.
Recent version of xphyle (4.0.0+) require python 3.6. Older versions of xphyle support python 3.4+.
Please note that xphyle may work on Windows, but it is not tested.
# Installation
```
pip install xphyle
```
# Building from source
Clone this repository and run
```
make
```
# Example usages:
```python
from xphyle import *
from xphyle.paths import STDIN, STDOUT
# Open a compressed file...
myfile = xopen('infile.gz')
# ...or a compressed stream
# e.g. gzip -c afile | python my_program.py
stdin = xopen(STDIN)
# Easily write to the stdin of a subprocess
with open_('|cat', 'wt') as process:
process.write('foo')
# We have to tell xopen what kind of compression
# to use when writing to stdout
stdout = xopen(STDOUT, compression='gz')
# The `open_` method ensures that the file is usable with the `with` keyword.
# Print all lines in a compressed file...
with open_('infile.gz') as myfile:
for line in myfile:
print(line)
# ... or a compressed URL
with open_('http://foo.com/myfile.gz') as myfile:
for line in myfile:
print(line)
# Transparently handle paths and file objects
def dostuff(path_or_file):
with open_(path_or_file) as myfile:
for line in myfile:
print(line)
# Read all lines in a compressed file into a list
from xphyle.utils import read_lines
lines = list(read_lines('infile.gz'))
# Sum the rows in a compressed file where each line is an integer value
total = sum(read_lines('infile.gz', convert=int))
```
See the [Documentation](https://xphyle.readthedocs.io/en/latest/) for full usage information.
# Supported compression formats
* `gzip` (uses `igzip` or `pigz` if available)
* `bgzip`
* `bzip2` (uses `pbzip2` if available)
* `lzma`
* `zstd`
# Issues
Please report bugs and request enhancements using the [issue tracker](https://github.com/jdidion/xphyle).
# Roadmap
Future releases are mapped out using [GitHub Projects](https://github.com/jdidion/xphyle/projects).
# Citing xphyle
[Didion, JP (2017) xphyle: Extraordinarily simple file handling. Journal of Open Source Software; doi:10.21105/joss.00255](https://joss.theoj.org/papers/10.21105/joss.00255#)
# Acknowledgements
* [Dependencies scanned by PyUp.io](http://pyup.io/)
* Thanks to [@ctb](https://github.com/ctb) for reviewing the xphyle paper
|
