1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
|
## Sixel band decoder
__Features__:
- written in C
- allocation free (single instance static memory)
- crafted for web assembly / emscripten
- also embeddable in native code
- quite fast (decoding throughput >150 MB/s)
- small wasm binary (~11 kB)
- small memory footprint (<<1 MB, depending on `MAX_WIDTH`)
### Note on WASM features
Currently wasm engines differ alot in supported wasm features.
To still support a wide variety of engines, this module is coded in vanilla C and
only uses the `bulk-memory` feature (optionally).
### Note on native usage
The decoder can be used from other native C/C++ projects by simply including `decoder.cpp`.
Still for native embedding there are a few things to consider first:
- single instance / static memory
The parser state is a single statically allocated struct. While this is a perfect fit
for isolated wasm module instances, it is prolly not what you want in a C or C++ project,
unless for a simple cmdline converter tool. Without code modifications decoding is limited
to one parser state / one image at a time. (This is easy fixable by introducing
state indirections at the function interfaces, but beyond the scope here.)
- code optimizations
The decoder loop is a highly optimized byte-by-byte loop in vanilla C.
While this also runs fast natively, things like SIMD extensions would give
another quite remarkable boost, but do not work reliable in wasm yet.
### Future optimization ideas
- wasm-SIMD, currently engines have no to lousy support with bad performance
(see dysfunctional proof of concept in `decoder-simd.cpp`)
- shared memory, currently shifting too much in browsers
- wasm-threading, still not fully landed in wasm, depends on stable shared memory interfaces
### Interface
To use the decoder from other non javascript wasm environments or natively,
you gonna need to hook into the following API.
For reference usage and how to build a full image decoder from the sixel band handling,
see the implementation for javascript under `src/WasmDecoder.ts`.
Important compile time settings (see build.sh to adjust):
- `CHUNK_SIZE` - max amount of chunk bytes
- `PALETTE_SIZE` - max colors in the palette
- `MAX_WIDTH` - max band width (excess pixels to the right are truncated)
Exported symbols:
- `void* get_state_address()`
Void pointer to the static `ParserState` struct in WASM memory.\
Properties of interest (indexed in 32bit):
- 1: fill color as ABGR32 (as given by `init`)
- 2: width+4 in M2, else 0
- 3: height in M2, else 0
- 4: raster numerator (unmodified) in M2, else 0
- 5: raster denominator (unmodified) in M2, else 0
- 6: raster width (unmodified) in M2, else 0
- 7: raster height (unmodified)in M2, else 0
- 8: truncate (as given by `init`)
- 9: image level (L0 - undecided, L1 - level 1, L2 - level 2)
- 10: operation mode (M0 - undecided, M1 - level 1/2 !truncate, M2 - level 2 truncating)
- 11: palette length
- `void* get_chunk_address()`
Void pointer to `ParserState.chunk` byte array (max size of `CHUNK_SIZE`).
Used to load image data to be processed by `decode`.
- `void* get_p0_address()`
Void pointer to first pixel line p0 the band. The other pixel lines p1 - p5
start at `get_p0_address() + (MAX_WIDTH + 4) * line_idx`.
Used to grab pixel data when a band was finished.
- `void* get_palette_address()`
Void pointer to `ParserState.palette` ABGR32 array (max size of `PALETTE_SIZE`).
Used to read/write palette colors.
- `void init(int fill_color, unsigned int palette_limit, int truncate)`
Initialize decoder for new image. Must be called before any decoding happens.
- `void decode(int start, int end)`
Decode data loaded into `ParserState.chunk[start .. end]` (right exclusive).
- `int current_width()`
Return the cursor advance of the current band in M1 mode, or width in M2 mode.
This is needed to properly construct the full image at the end of decoding,
in case the data did not finish with LF.
- `int current_width()`
M1 mode only - return the current lowermost pixel position touched by a sixel
of the current band. This is needed to properly construct the full image with
proper height for the current band, in case the data did not finish with LF.
Needed callbacks:
- `int mode_parsed(int mode)`
Called once early during decoding when the parser settled the operation mode.
The operation mode will be settled by the decoder on reading a valid raster
attributes command or any other sixel command or sixel data bytes.
Used to announce the operation mode and potential raster attributes for
further preparation steps.
Return 0 to continue, 1 to abort further processing.
- `int handle_band(int width)`
Called upon finishing decoding of a sixel band (on LF). `width` is either the
raster width (M2 mode), or the max cursor advance (both clamped to `maxWidth`).
Used to copy pixel in p0 .. p5, before continuing with the next band.
Return 0 to continue, 1 to abort further processing.
### Note on SIXEL handling
- The data to be digested by this decoder should only be the "Picture Definition" part of a SIXEL
escape sequence, as denoted by the spec. Any other data beyond that is likely to screw up
the image creation. In particular this means, that the decoder should run behind
a terminal escape sequence, that is capable of proper DEC style DCS parsing
(also handling spurious ESC, SUB and C1 codes).
- Raster or pixel ratio definitions are not dealt with beside width and height (if `truncate` is set).
Raster attributes are exposed unmodified in 0 .. 2^31-1 (as read from data),
and can be used to postprocess pixel data as needed.
- With `truncate` set in `init`, the decoder will truncate the image to given raster dimensions.
This does not apply to level 1 images without any raster attributes. While truncation to
raster dimensions on decoder side is not spec-conform, it is the expected data format created by
a spec-conform encoder.
- If `truncate` is not set, the width will be derived from cursor advance to the right, clamped
to `MAX_WIDTH-4`. In this mode, `current_height` is reported as lowermost pixel position touched by sixels.
Furthermore in this mode the height is not limited by any means, thus decoding may run forever.
Use some sort of accounting during `handle_band` to spot malformed data or excessive memory usage,
especially when dealing with data streams.
- Palette colors are applied immediately to sixels (printer mode), there is no terminal-like indexed mode.
While this is in line with the spec, it does not allow to mimick the palette behavior of older terminals
(e.g. palette animations are not possible).
- The decoder unconditionally strips the 8th bit, mapping all data bytes in 7-bit space.
While the spec defines this only as error recovery strategy for GR codes, the decoder also does this
for C1, which might lead to sixel command interpretation from spurious C1 codes. Note that C1
never should appear in sixel data, if used behind a proper escape sequence parser.
- Other than stated in the spec, the decoder does not error on low C0 codes, instead silently ignores them.
Again a proper escape sequence parser will filter / act upon those.
- The repeat count gets not limited/clamped to 32767. The digits are parsed in signed int32 for
performance reasons, and converted to unsigned before used as repeat count. While the counter
will show weird behavior above 2^31-1 (counting backwards), it should not be possible to overflow
the pixel arrays with malicious data (separately tested against `MAX_WIDTH` before any painting).
There are probably more deviations from the SIXEL spec not listed here.
### Status
Beta. The code is tested with unit/regression tests from the JS integration.
|
