File: README.rst

package info (click to toggle)
lz4tools 1.3.1.1-1.1
  • links: PTS, VCS
  • area: main
  • in suites: bookworm, bullseye, sid
  • size: 380 kB
  • sloc: ansic: 3,988; python: 377; makefile: 8
file content (153 lines) | stat: -rw-r--r-- 5,953 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
==================
Lz4tools and Lz4f
==================

.. image:: https://travis-ci.org/darkdragn/lz4tools.svg?branch=master

Overview
--------
This package consists of three parts:

  1. lz4f - C-Module containing python bindings for all lz4frame functions.
  2. lz4tools - a zipfile-like file wrapper class and tarfile-like class for lz4 compressed files. 
  3. lz4toolsCli - a quick cli for using lz4tools static functions.

Before going any further, I recommend reading up on lz4 at: 
https://code.google.com/p/lz4/

It is an awesome compression algorithm and I can't thank Yann Collet enough for
putting together the C implementation and lz4frame.

Usage
-----
Cli:
    New addition, there is now a simple Cli for anyone wishing these
    capabilities would exist directly on the command line.
    
.. code:: bash

    $lz4toolsCli
    usage: lz4toolsCli [-h] [-f] [-t] [-d] input [output]
    
    positional arguments:
      input       The targeted input.
      output      Optional output target.
    
    optional arguments:
      -h, --help  show this help message and exit
      -f          Compress file. Default action if input is a file.
      -t          Compress directory to .tar.lz4. Default action if input is a
                  directory
      -d          Decompress. Default action if the file ends in .lz4.

C-Module / Bindings:
    I would recommend against using lz4f directly except in debug/testing
    situations. If necessary, a compress or decompress operation first needs a
    context that will be used with all lz4f functions:

    **lz4f compression the hard way:**

    .. code:: python

        >>> import lz4f
        >>> inputFile = open('fileIn', 'rb')
        >>> cCtx = lz4f.createCompressionContext
        >>> header = lz4f.compressBegin(cCtx)
        >>> data = lz4f.compressUpdate(inputFile.read(), cCtx)
        >>> end = lz4f.compressEnd(cCtx)
        >>> with open('output.lz4', 'wb') as out:
        ...     out.write(header)
        ...     out.write(data)
        ...     out.write(end)
        ...     out.flush()
        ...     out.close()
        >>> lz4f.freeCompContext(cCtx)
        >>> inputFile.close()
        >>> del header, data, end
    
    **lz4f compression the easy way:**

    .. code:: python

        >>> import lz4f
        >>> with open('output.lz4', 'wb') as out:
        ...     with open('fileIn', 'rb') as inFile:
        ...         out.write(lz4f.compressFrame(inFile.read())
        ...     out.flush()
        ...     out.close()
    
    **Advantages and disadvantages:**
      The easy way takes more ram. It reads the
      contents of the file into a buffer, passes it and compresses it all in one
      go. With the hard way you can have it read as little or as much as you
      like. For instance, you can break up the input into 64kiB chunks.
      Each chunk could be read, compressed and dropped to disk to conserve ram.

----

Lz4Tools Module:
  The lz4file class is currently read only. Right now it is a bit rough around
  the edges, however over the next couple of weeks, I will finish adding some
  docstrings, and such to make it more user friendly. As soon as I get a chance
  I will make it write capable. The easiest way to use it is with either the
  open or openTar methods. That's right! There is a lz4Tar class in the module
  that is a subclass of tarfile. 

  **lz4tools tar example:**

  .. code:: python

        >>> import lz4tools
        >>> lz4tools.compressTarDefault('src')
        >>> testTar = lz4tools.openTar('src.tar.lz4')
        >>> testTar.list()
        -rwxr-xr-x darkdragn/darkdragn          0 2014-10-02 23:06:09 src/
        -rw-r--r-- darkdragn/darkdragn      29905 2014-09-16 18:29:45 src/lz4hc.c
        -rw-r--r-- darkdragn/darkdragn       6781 2014-09-16 18:29:45 src/  xxhash.h
        -rw-r--r-- darkdragn/darkdragn      25662 2014-09-16 18:29:45 src/  xxhash.c
        -rw-rw-r-- darkdragn/darkdragn      13894 2014-10-02 20:22:09 src/lz4frame.h
        -rw-rw-r-- darkdragn/darkdragn      46241 2014-10-02 20:22:09 src/lz4.c
        -rw-r--r-- darkdragn/darkdragn       8832 2014-09-16 18:29:45 src/lz4hc.h
        -rw-rw-r-- darkdragn/darkdragn      11734 2014-10-02 23:06:08 src/python-lz4f.c
        -rw-rw-r-- darkdragn/darkdragn       2554 2014-10-02 20:22:09 src/python-lz4f.h
        -rw-r--r-- darkdragn/darkdragn      14882 2014-09-18 01:28:06 src/lz4.h
        -rw-rw-r-- darkdragn/darkdragn      50141 2014-10-02 23:04:05 src/lz4frame.c
    

  **lz4tools file example:**

  .. code:: python

        >>> import lz4tools
        >>> lz4tools.compressFileDefault('setup.py')
        >>> testFile = lz4tools.open('setup.py.lz4')
        >>> testFile.blkDict
        {0: {'decomp_e': 1445, 'compressed_begin': 7, 'blkSize': 923}}
        >>> testFile.seek(1002)
        >>> print testFile.read()
            test_suite = "nose.collector",
            keywords = ['lz4', 'lz4frame', 'lz4file', 'lz4tar'],
            classifiers=[
                'Development Status :: 5 - Production/Stable',
                'License :: OSI Approved :: BSD License',
                'Intended Audience :: Developers',
                'Programming Language :: C',
                'Programming Language :: Python',
                'Programming Language :: Python :: 2.6',
                'Programming Language :: Python :: 2.7',
            ],
        )

And thus ends the brief tutorial.

Notes
_____
  Version: 
    The first two digits of the version will always correspond with the version
    of lz4 that is included. Current version is r123, thus 1.2. The next digit
    corresponds to milestone improvements. Example: Once lz4file supports
    write. The last digit will be slight improvements. Usually some contextual
    error, or syntax error. Perhaps even a quick fix for python3.4, since I 
    don't use it often, if an issue is brought to my attention, I will provide
    a quick fix as quickly as possible.