1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
|
==================================
Missing and Unimplemented Features
==================================
This document attempts to capture features of the zstd C API that are currently
not exposed to Python.
If there is a feature on this page that would be beneficial to you, please
open a GitHub issue requesting its implementation. If an existing GitHub
issue exists, please leave a comment on the issue to amplify the importance of
the request (this project's author doesn't monitor the emoji counts).
Missing Constants
=================
* ``ZSTD_CLEVEL_DEFAULT``
* ``ZSTD_SRCSIZEHINT_MIN``
* ``ZSTD_SRCSIZEHINT_MAX``
* ``ZSTD_BLOCKSIZE_MAX_MIN``
* ``ZSTD_DECOMPRESSION_MARGIN``
Compression and Decompression Parameters
========================================
* ``ZSTD_p_forceAttachDict``
* ``ZSTD_dictForceLoad``
* ``ZSTD_c_targetCBlockSize``
* ``ZSTD_c_literalCompressionMode``
* ``ZSTD_c_srcSizeHint``
* ``ZSTD_d_stableOutBuffer``
* ``ZSTD_c_enableDedicatedDictSearch``
* ``ZSTD_c_stableInBuffer``
* ``ZSTD_c_stableOutBuffer``
* ``ZSTD_c_blockDelimiters``
* ``ZSTD_c_validateSequences``
* ``ZSTD_c_useBlockSplitter``
* ``ZSTD_c_useRowMatchFinder``
* ``ZSTD_d_forceIgnoreChecksum``
* ``ZSTD_d_refMultipleDDicts``
* ``ZSTD_refMultipleDDicts_e``
* ``ZSTD_c_prefetchCDictTables``
* ``ZSTD_c_enableSeqProducerFallback``
* ``ZSTD_c_maxBlockSize``
* ``ZSTD_c_searchForExternalRepcodes``
* ``ZSTD_d_disableHuffmanAssembly``
* ``ZSTD_d_stableOutBuffer``
Missing Functions
=================
* ``ZSTDMT_toFlushNow()``
* ``ZSTD_minCLevel()``
* ``ZSTD_cParam_getBounds()``
* ``ZSTD_dParam_getBounds()``
* ``ZSTD_generateSequences()``
* ``ZSTD_mergeBlockDelimiters()``
* ``ZSTD_compressSequences()``
* ``ZSTD_writeSkippableFrame()``
* ``ZSTD_decompressionMargin()``
* ``ZSTD_sequenceBound()``
Missing Features
================
* ``ZSTD_getFrameProgression()`` isn't exposed everywhere it could be.
* Compression parameters cannot be modified mid operation.
* ``ZSTD_Sequence`` and related ``ZSTD_getSequences()`` not exposed.
* ``ZSTD_threadPool`` not exposed.
* ``ZSTD_sequenceProducer_F`` and ``ZSTD_registerSequenceProducer()`` not
exposed.
* ``ZSTD_CCtx_getParameter()``, ``ZSTD_CCtxParam_getParameter()``, and
``ZSTD_DCtx_getParameter()`` could be leveraged for parameter retrieval.
* ``ZSTD_CCtx_setCParams()`` could potentially be utilized.
* ``ZSTD_error_*`` constants / error codes not exposed.
Potential 1.0 Release Features
==============================
The following track features and changes we would nominally perform
before a 1.0 release.
* Properly handle non-blocking I/O and partial writes for objects implementing
``io.RawIOBase``.
* Consider making reads across frames configurable behavior.
* Overall API design review.
* Use Python allocator where possible.
* Figure out what to do about experimental APIs not implemented by CFFI.
* APIs for auto adjusting compression parameters based on input size. e.g.
clamping the window log so it isn't too large for input.
* Consider allowing compressor and decompressor instances to be thread safe,
support concurrent operations. Or track when an operation is in progress and
refuse to let concurrent operations use the same instance.
* Support for magic-less frames for all decompression operations (``decompress()``
doesn't work due to sniffing the content size and the lack of a ZSTD API to
sniff magic-less frames - this should be fixed in 1.3.5.).
* Audit for complete flushing when ending compression streams.
* Deprecate legacy APIs.
* Audit for ability to control read/write sizes on all APIs.
* Detect memory leaks via bench.py.
* Remove low-level compression parameters from ``ZstdCompressor.__init__`` and
require use of ``ZstdCompressionParameters``.
* Consider a ``chunker()`` API for decompression.
* Consider stats for ``chunker()`` API, including finding the last consumed
offset of input data.
* Consider controls over resetting compression contexts (session only, parameters,
or session and parameters).
* Utilize ``ZSTD_getDictID_fromCDict()``?
* Stop relying on private libzstd headers and symbols (namely ``pool.h``).
And other non-essential features:
* Support for block compression APIs.
* API for ensuring max memory ceiling isn't exceeded.
* Move off nose for testing.
|
