1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374
|
# Lua Circular Buffer Module
## Overview
A circular buffer library for an in-memory sliding window time series data
store.
## Module
### Example Usage
```lua
require "circular_buffer"
local cb = circular_buffer.new(1440, 1, 60)
local ERRORS = cb:set_header(1, "Errors")
cb:add(1e9, ERRORS, 1)
cb:add(1e9, ERRORS, 7)
local val = cb:get(1e9, ERRORS)
-- val == 8
```
### Functions
#### new
```lua
require "circular_buffer"
local cb = circular_buffer.new(1440, 1, 60)
```
Import the Lua _circular_buffer_ via the Lua 'require' function. The module is
globally registered and returned by the require function.
*Arguments*
- rows (unsigned) The number of rows in the buffer (must be > 1)
- columns (unsigned) The number of columns in the buffer
(must be > 0 and <= 256)
- seconds_per_row (unsigned) The number of seconds each row represents
(must be > 0).
*Return*
- circular_buffer userdata object.
#### version
```lua
local v = circular_buffer.version()
-- v == "1.0.0"
```
Returns a string with the running version of circular_buffer.
*Arguments*
- none
*Return*
- Semantic version string
### Methods
**Note:** All column arguments are 1 based. If the column is out of range for
the configured circular buffer a fatal error is generated.
#### add
```lua
d = cb:add(1e9, 1, 1)
-- d == 1
d = cb:add(1e9, 1, 99)
-- d == 100
```
Adds a value to the specified row/column in the circular buffer.
*Arguments*
- nanosecond (unsigned) The number of nanosecond since the UNIX epoch. The value
is used to determine which row is being operated on.
- column (unsigned) The column within the specified row to perform an add
operation on.
- value (double) The value to be added to the specified row/column.
*Return*
- The value of the updated row/column or nil if the time was outside the range
of the buffer.
#### set
```lua
d = cb:set(1e9, 1, 1)
-- d == 1
d = cb:set(1e9, 1, 99)
-- d == 99
```
Overwrites the value at a specific row/column in the circular buffer.
*Arguments*
- nanosecond (unsigned) The number of nanosecond since the UNIX epoch. The value
is used to determine which row is being operated on.
- column (unsigned) The column within the specified row to perform a set
operation on.
- value (double) The value to be overwritten at the specified row/column.
For aggregation methods "min" and "max" the value is only overwritten if it is
smaller/larger than the current value.
*Return*
- The resulting value of the row/column or nil if the time was outside the range
of the buffer.
#### get
```lua
d = cb:get(1e9, 1)
-- d == 99
```
Fetches the value at a specific row/column in the circular buffer.
*Arguments*
- nanosecond (unsigned) The number of nanosecond since the UNIX epoch. The value
is used to determine which row is being operated on.
- column (unsigned) The column within the specified row to retrieve the data
from.
*Return*
- The value at the specifed row/column or nil if the time was outside the range
of the buffer.
#### get_delta
```lua
d = cb:get_delta(1e9, 1)
-- d == 99
```
Fetches the delta value at a specific row/column in the circular buffer since
the last reset/output.
*Arguments*
- nanosecond (unsigned) The number of nanosecond since the UNIX epoch. The value
is used to determine which row is being operated on.
- column (unsigned) The column within the specified row to retrieve the data
from.
*Return*
- The delta value at the specifed row/column or nil if the time was outside the
range of the buffer.
#### get_range
```lua
local stats = circular_buffer.new(5, 1, 1)
stats:set(1e9, 1, 1)
stats:set(2e9, 1, 2)
stats:set(3e9, 1, 3)
stats:set(4e9, 1, 4)
stats:set(5e9, 1, 5)
local a = stats:get_range(1, 3e9, 4e9)
-- a = {3, 4}
```
Returns an array of column values spanning the specificed time range.
*Arguments*
- column (unsigned) The column that the computation is performed against.
- start (unsigned _optional_) The number of nanosecond since the UNIX epoch.
Sets the start time of the computation range; if nil the buffer's start time
is used.
- end (unsigned _optional_) The number of nanosecond since the UNIX epoch. Sets
the end time of the computation range (inclusive); if nil the buffer's end
time is used. The end time must be greater than or equal to the start time.
*Returns*
- Array of column values or nil if the range fell entirely outside of the
buffer.
#### get_range_delta
```lua
local stats = circular_buffer.new(5, 1, 1)
stats:set(1e9, 1, 1)
stats:set(2e9, 1, 2)
stats:set(3e9, 1, 3)
stats:set(4e9, 1, 4)
stats:set(5e9, 1, 5)
local a = stats:get_range(1, 3e9, 4e9)
-- a = {3, 4}
```
Returns an array of column delta values spanning the specificed time range since
the last reset/output.
*Arguments*
- column (unsigned) The column that the computation is performed against.
- start (unsigned _optional_) The number of nanosecond since the UNIX epoch.
Sets the start time of the computation range; if nil the buffer's start time
is used.
- end (unsigned _optional_) The number of nanosecond since the UNIX epoch. Sets
the end time of the computation range (inclusive); if nil the buffer's end
time is used. The end time must be greater than or equal to the start time.
*Returns*
- Array of column delta values or nil if the range fell entirely outside of the
buffer.
#### get_configuration
```lua
rows, columns, seconds_per_row = cb:get_configuration()
-- rows == 1440
-- columns = 1
-- seconds_per_row = 60
```
Returns the configuration options passed to _new_.
*Arguments*
- none
*Return*
- The circular buffer dimension values specified in the constructor.
- rows
- columns
- seconds_per_row
#### current_time
```lua
t = cb:current_time()
-- t == 86340000000000
```
Returns the timestamp of the newest row.
*Arguments*
- none
*Return*
- The time of the most current row in the circular buffer (nanoseconds).
#### set_header
```lua
column = cb:set_header(1, "Errors")
-- column == 1
```
Sets the header metadata for the specifed column.
*Arguments*
- column (unsigned) The column number where the header information is applied.
- name (string) Descriptive name of the column (maximum 15 characters). Any non
alpha numeric characters will be converted to underscores. (default: Column_N)
- unit (string _optional_) The unit of measure (maximum 7 characters). Alph
numeric, '/', and '*' characters are allowed everything else will be converted
to underscores. i.e. KiB, Hz, m/s (default: count)
- aggregation_method (string _optional_) Controls how the column data is
aggregated when combining multiple circular buffers.
- **sum** The total is computed for the time/column (default).
- **min** The smallest value is retained for the time/column.
- **max** The largest value is retained for the time/column.
- **none** No aggregation will be performed the column.
*Return*
- The column number passed into the function.
#### get_header
```lua
name, unit, aggregation_method = cb:get_header(1)
-- name == "Errors"
-- unit == "count"
-- aggregation_method == "sum"
```
Retrieves the header metadata for the specified column.
*Arguments*
- column (unsigned) The column number of the header information to be retrieved.
*Return*
- The current values of specified header column.
- name
- unit
- aggregation_method
#### reset_delta
```lua
-- only available when using the non lua_sandbox (the sandbox output manages the
-- reset in that case)
cb:reset_delta(")
```
Resets the delta counts back to initialized.
*Arguments*
- none
*Return*
- none
#### annotate
```lua
-- only available when using the lua_sandbox
cb:annotate(1e9, 1, "alert", "Anonmaly detected rate of change exceeded 2 standard deviations")
```
Creates/Overwrites the annotation at a specific row/column in the circular
buffer.
*Arguments*
- nanosecond (unsigned) The number of nanosecond since the UNIX epoch. The value
is used to determine which row is being operated on.
- column (unsigned) The column within the specified row to perform a set
operation on.
- type (string) - info|alert
- annotation (string) - full text of the annotation
- delta (bool _optional_) - include the change in the cbufd ouput (default true)
*Return*
- none
#### format
```lua
-- only available when using the lua_sandbox
cb:format("cbufd")
```
Sets an internal flag to control the output format of the circular buffer data
structure.
*Arguments*
- format (string)
- **cbuf** The circular buffer full data set format.
- **cbufd** The circular buffer delta data set format.
*Return*
- The circular buffer object.
### Output
```lua
-- only available when using the lua_sandbox todo: add tostring support
cb:format("cbuf")
output(cb) -- serializes the full buffer
cb:format("cbufd")
output(cb) -- serializes the delta of the buffer since the last output
```
The circular buffer can be passed to the lua_sandbox output() function. The
output format can be selected using the format() function.
The cbuf (full data set) output format consists of newline delimited rows
starting with a json header row followed by the data rows with tab delimited
columns. The time in the header corresponds to the time of the first data row,
the time for the other rows is calculated using the seconds_per_row header
value.
{json header}
row1_col1\trow1_col2\n
.
.
.
rowN_col1\trowN_col2\n
The cbufd (delta) output format consists of newline delimited rows starting with
a json header row followed by the data rows with tab delimited columns. The
first column is the timestamp for the row (time_t). The cbufd output will only
contain the rows that have changed and the corresponding delta values for each
column.
{json header}
row14_timestamp\trow14_col1\trow14_col2\n
row10_timestamp\trow10_col1\trow10_col2\n
Sample Cbuf Output
------------------
{"time":2,"rows":3,"columns":3,"seconds_per_row":60,"column_info":[{"name":"HTTP_200","unit":"count","aggregation":"sum"},{"name":"HTTP_400","unit":"count","aggregation":"sum"},{"name":"HTTP_500","unit":"count","aggregation":"sum"}], "annotations":[]}
10002 0 0
11323 0 0
10685 0 0
|