1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114
|
#include "streamvbyte.h"
#if defined(_MSC_VER)
/* Microsoft C/C++-compatible compiler */
#include <intrin.h>
#elif defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))
/* GCC-compatible compiler, targeting x86/x86-64 */
#include <x86intrin.h>
#elif defined(__GNUC__) && defined(__ARM_NEON__)
/* GCC-compatible compiler, targeting ARM with NEON */
#include <arm_neon.h>
#elif defined(__GNUC__) && defined(__IWMMXT__)
/* GCC-compatible compiler, targeting ARM with WMMX */
#include <mmintrin.h>
#elif (defined(__GNUC__) || defined(__xlC__)) && \
(defined(__VEC__) || defined(__ALTIVEC__))
/* XLC or GCC-compatible compiler, targeting PowerPC with VMX/VSX */
#include <altivec.h>
#elif defined(__GNUC__) && defined(__SPE__)
/* GCC-compatible compiler, targeting PowerPC with SPE */
#include <spe.h>
#endif
#include <string.h> // for memcpy
#include "streamvbyte_shuffle_tables_encode.h"
#ifdef __AVX__
#include "streamvbyte_x64_encode.c"
#else
static uint8_t _encode_data(uint32_t val, uint8_t *__restrict__ *dataPtrPtr) {
uint8_t *dataPtr = *dataPtrPtr;
uint8_t code;
if (val < (1 << 8)) { // 1 byte
*dataPtr = (uint8_t)(val);
*dataPtrPtr += 1;
code = 0;
} else if (val < (1 << 16)) { // 2 bytes
memcpy(dataPtr, &val, 2); // assumes little endian
*dataPtrPtr += 2;
code = 1;
} else if (val < (1 << 24)) { // 3 bytes
memcpy(dataPtr, &val, 3); // assumes little endian
*dataPtrPtr += 3;
code = 2;
} else { // 4 bytes
memcpy(dataPtr, &val, sizeof(uint32_t));
*dataPtrPtr += sizeof(uint32_t);
code = 3;
}
return code;
}
static uint8_t *svb_encode_scalar(const uint32_t *in,
uint8_t *__restrict__ keyPtr,
uint8_t *__restrict__ dataPtr,
uint32_t count) {
if (count == 0)
return dataPtr; // exit immediately if no data
uint8_t shift = 0; // cycles 0, 2, 4, 6, 0, 2, 4, 6, ...
uint8_t key = 0;
for (uint32_t c = 0; c < count; c++) {
if (shift == 8) {
shift = 0;
*keyPtr++ = key;
key = 0;
}
uint32_t val = in[c];
uint8_t code = _encode_data(val, &dataPtr);
key |= code << shift;
shift += 2;
}
*keyPtr = key; // write last key (no increment needed)
return dataPtr; // pointer to first unused data byte
}
#endif
#ifdef __ARM_NEON__
#include "streamvbyte_arm_encode.c"
#endif
// Encode an array of a given length read from in to bout in streamvbyte format.
// Returns the number of bytes written.
size_t streamvbyte_encode(const uint32_t *in, uint32_t count, uint8_t *out) {
#ifdef __AVX__
return streamvbyte_encode_SSSE3(in,count,out);
#else
uint8_t *keyPtr = out;
uint32_t keyLen = (count + 3) / 4; // 2-bits rounded to full byte
uint8_t *dataPtr = keyPtr + keyLen; // variable byte data after all keys
#if defined(__ARM_NEON__)
uint32_t count_quads = count / 4;
count -= 4 * count_quads;
for (uint32_t c = 0; c < count_quads; c++) {
dataPtr += streamvbyte_encode_quad(in, dataPtr, keyPtr);
keyPtr++;
in += 4;
}
#endif
return svb_encode_scalar(in, keyPtr, dataPtr, count) - out;
#endif// no AVX
}
|