File: data-format.rst

package info (click to toggle)
flent 2.2.0-2
  • links: PTS, VCS
  • area: main
  • in suites: forky, sid, trixie
  • size: 5,156 kB
  • sloc: python: 11,662; ansic: 626; sh: 257; makefile: 225; javascript: 72; xml: 58
file content (161 lines) | stat: -rw-r--r-- 5,508 bytes parent folder | download | duplicates (5)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
The Data File Format
====================
The aggregated test data is saved in a file called
:file:`<test_name>-<date>.<title>.flent.gz` (the title part is omitted if no title is
specified by the :option:`-t` parameter). This file contains the data points
generated during the test, as well as some metadata.


The top-level object keys
-------------------------

.. envvar:: version

            The file format version as an integer.

.. envvar:: x_values

            An array of the x values for the test data (typically the time
            values for timeseries data).

.. envvar:: results

            A JSON object containing the result data series. The keys are the
            data series names; the value for each key is an array of y values
            for that data series. The data array has the same length as the
            :envvar:`x_values` array, but there may be missing data points
            (signified by null values).

.. envvar:: metadata

            An object containing various data points about the test run. The
            metadata values are read in as configuration parameters when the
            data set is loaded in for further processing. Not all tests use all
            the parameters, but they are saved anyway.

.. envvar:: raw_values

            An array of objects for each data series. Each element of the array
            contains the raw values as parsed from the test tool corresponding
            to that data series.


Metadata keys
-------------

.. envvar:: NAME

	    The test name.

.. envvar:: TITLE

	    Any extra title specified by the :option:`--title-extra` parameter
            when the test was run.

.. envvar:: HOSTS

	    List of the server hostnames connected to during the test.

.. envvar:: LOCAL_HOST

	    The hostname of the machine that ran the test.

.. envvar:: LENGTH

	    Test length in seconds, as specified by the :option:`--length` parameter.

.. envvar:: TOTAL_LENGTH

            Actual data series length, after the test has added time to the
            :envvar:`LENGTH`.

.. envvar:: STEP_SIZE

	    Time step size granularity.

.. envvar:: TIME

            ISO timestamp of the time the test was initiated.

.. envvar:: NOTE

	    Arbitrary text as entered with the :option:`--note` switch when the
            test was run.

.. envvar:: FLENT_VERSION

            Version of Flent that generated the data file.

.. envvar:: IP_VERSION

            IP version used to run test (as specified by command line
	    parameters, or auto-detected from :c:func:`getaddrinfo()` if unspecified).

.. envvar:: KERNEL_NAME

	    The kernel name as reported by :command:`uname -s`.

.. envvar:: KERNEL_RELEASE

            The kernel release as reported by :command:`uname -r`.

.. envvar:: MODULE_VERSIONS

            The sha1sum of certain interesting Linux kernel modules, if
            available. Can be used to match test data to specific code versions,
            if the kernel build is instrumented to, e.g., set the build ID to a
            git revision.

.. envvar:: SYSCTLS

            The values of several networking-related sysctls on the host (if
            available; Linux only).

.. envvar:: EGRESS_INFO

	    Interface name, qdisc, offload, driver and BQL configuration of the
            interface used to reach the test target. This requires that the
            :command:`ip` binary is present on Linux, but can be extracted from
            :command:`route` on BSD. Qdisc information requires the
            :command:`tc` binary to be present, and offload information requires
            :command:`ethtool`.

            If the :option:`--remote-metadata` is used, the extended metadata
            info is gathered for each of the hostnames specified. This is
            gathered under the :envvar:`REMOTE_METADATA` key in the metadata
            object, keyed by the hostname values passed to
            :option:`--remote-metadata`. Additionally, the
            :envvar:`REMOTE_METADATA` object will contain an object called
            :envvar:`INGRESS_INFO` which is a duplicate of
            :envvar:`EGRESS_INFO`, but with the destination IP exchanged for the
            source address of the host running flent. The assumption here is
            that :option:`--remote-metadata` is used to capture metadata of a
            router known to be in the test path, in which case
            :envvar:`INGRESS_INFO` will contain information about the reverse
            path from the router (which is ingress from the point of view of the
            host running flent). If the host being queried for remote metadata
            is off the path, the contents of :envvar:`INGRESS_INFO` will
            probably be the same as that of :envvar:`EGRESS_INFO` .

Extended metadata
-----------------

If the :option:`--extended-metadata` switch is turned on, the following
additional values are collected and stored (to the extent they are available
from the platform):

.. envvar:: IP_ADDRS

	    IP addresses assigned to the machine running flent.

.. envvar:: GATEWAYS

	    IP addresses of all detected default gateways on the system, and the
            interfaces they are reachable through. Only available if the
            :command:`netstat` binary is present on the system.

.. envvar:: EGRESS_INFO

            In the :envvar:`EGRESS_INFO` key, the IP address of the next-hop
            router and the interface MAC address are added if extended metadata
            is enabled.