1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
|
See how to configure and compile pmacct for MySQL/MariaDB use in the
"Configuring pmacct for compilation and installing" chapter of QUICKSTART.
To create the database and grant default permission to pmacctd you have to execute
the two scripts below, in the same order (assuming mysql command is in the path):
- To create v1 tables:
* mysql -u root -p < pmacct-create-db_v1.mysql
* mysql -u root -p < pmacct-grant-db.mysql
- To use v1 tables:
* data will be available in 'acct' table of 'pmacct' DB.
* Add 'sql_table_version: 1' line to your configuration.
Similarly, v2 to v9 tables:
- To create v2 tables:
* mysql -u root -p < pmacct-create-db_v2.mysql
* mysql -u root -p < pmacct-grant-db.mysql
- To use v2 tables:
* data will be available in 'acct_v2' table of 'pmacct' DB.
* Add 'sql_table_version: 2' line to your configuration.
[ ... ]
- To create v9 tables:
* mysql -u root -p < pmacct-create-db_v9.mysql
* mysql -u root -p < pmacct-grant-db.mysql
- To use v9 tables:
* data will be available in 'acct_v9' table of 'pmacct' DB.
* Add 'sql_table_version: 9' line to your configuration.
Similarly, BGP tables:
- To create BGP v1 tables:
* mysql -u root -p < pmacct-create-db_bgp_v1.mysql
* mysql -u root -p < pmacct-grant-db.mysql
- To use BGP v1 tables:
* data will be available in 'acct_bgp' table of 'pmacct' DB.
* Add 'sql_table_version: 1' line to your configuration.
* Add 'sql_table_type: bgp' line to your configuration.
- To understand difference between the various table versions:
* Do you need any of the BGP primitives ? Then look the next section.
* Do you need tags for traffic tagging ? Then you have to use v9.
* Do you need TCP flags ? Then you have to use v7.
* Do you need both IP addresses and AS numbers in the same table ? Then you have to use v6.
* Do you need packet classification ? Then you have to use v5.
* Do you need flows (other than packets) accounting ? Then you have to use v4.
* Do you need ToS/DSCP field (QoS) accounting ? Then you have to use v3.
* Do you need VLAN traffic accounting ? Then you have to use v2.
* If all of the above points sound useless, then use v1.
* v8 changes field names so to bring all supported databases to the same naming convention.
- To understand difference between the various BGP table versions:
* Only BGP table v1 is currently available.
- Aggregation primitives to SQL schema mapping. Although default schemas
come all with "NOT NULL", this is optional and depending on the scenario:
for example, if mixed L2 (containing L2 only info) and L3 (containing L2
and L3 info) flows are collected, maybe L3-related fields like src_host
or dst_host are best defined without the "NOT NULL" constraint.
Aggregation primitive => SQL table field:
* tag => agent_id (INT(4) UNSIGNED NOT NULL)
- or tag => tag (INT(4) UNSIGNED NOT NULL, if sql_table_version >= 9)
* tag2 => tag2 (INT(4) UNSIGNED NOT NULL, see README.tag2)
* label => label (VARCHAR(255) NOT NULL, see README.label)
* src_as => as_src (INT(4) UNSIGNED NOT NULL)
- or src_as => ip_src (INT(4) UNSIGNED NOT NULL), if sql_table_version < 6)
* dst_as => as_dst (INT(4) UNSIGNED NOT NULL)
- or dst_as => ip_dst (INT(4) UNSIGNED NOT NULL), if sql_table_version < 6)
* peer_src_as => peer_as_src (INT(4) UNSIGNED NOT NULL)
* peer_dst_as => peer_as_dst (INT(4) UNSIGNED NOT NULL)
* peer_src_ip => peer_ip_src (CHAR(45) NOT NULL)
- or (VARBINARY(16) NOT NULL, if sql_num_hosts: true)
* peer_dst_ip => peer_ip_dst (CHAR(45) NOT NULL)
- or (VARBINARY(16) NOT NULL, if sql_num_hosts: true)
* mpls_vpn_rd => mpls_vpn_rd (CHAR(18) NOT NULL)
* std_comm => comms (CHAR(24) NOT NULL)
* ext_comm => ecomms (CHAR(24) NOT NULL)
* lrg_comm => lcomms (CHAR(24) NOT NULL)
* as_path => as_path (CHAR(21) NOT NULL)
* local_pref => local_pref (INT(4) UNSIGNED NOT NULL)
* med => med (INT(4) UNSIGNED NOT NULL)
* dst_roa => roa_dst (CHAR(1) NULL)
* src_std_comm => comms_src (CHAR(24) NOT NULL)
* src_ext_comm => ecomms_src (CHAR(24) NOT NULL)
* src_lrg_comm => lcomms_src (CHAR(24) NOT NULL)
* src_as_path => as_path_src (CHAR(21) NOT NULL)
* src_local_pref => local_pref_src (INT(4) UNSIGNED NOT NULL)
* src_med => med_src (INT(4) UNSIGNED NOT NULL)
* src_roa => roa_src (CHAR(1) NULL)
* in_iface => iface_in (INT(4) UNSIGNED NOT NULL, see README.iface)
* out_iface => iface_out (INT(4) UNSIGNED NOT NULL, see README.iface)
* src_mask => mask_src (INT(1) UNSIGNED NOT NULL, see README.mask)
* dst_mask => mask_dst (INT(1) UNSIGNED NOT NULL, see README.mask)
* cos => cos (INT(1) UNSIGNED NOT NULL, see README.cos)
* etype => etype (CHAR(5) NOT NULL, see README.etype)
* src_host_country => country_ip_src (CHAR(2) NOT NULL, see README.GeoIP)
* dst_host_country => country_ip_dst (CHAR(2) NOT NULL, see README.GeoIP)
* src_host_pocode => pocode_ip_src (CHAR(12) NOT NULL, see README.GeoIP)
* dst_host_pocode => pocode_ip_dst (CHAR(12) NOT NULL, see README.GeoIP)
* src_host_coords => lat_ip_src (FLOAT NOT NULL, see README.GeoIP)
* src_host_coords => lon_ip_src (FLOAT NOT NULL, see README.GeoIP)
* dst_host_coords => lat_ip_dst (FLOAT NOT NULL, see README.GeoIP)
* dst_host_coords => lon_ip_dst (FLOAT NOT NULL, see README.GeoIP)
* sampling_rate => sampling_rate (INT(4) UNSIGNED NOT NULL, see README.sampling)
* sampling_direction => sampling_direction (CHAR(1) NOT NULL, see README.sampling)
* class => class_id (CHAR(16) NOT NULL)
* src_mac => mac_src (CHAR(17) NOT NULL)
* dst_mac => mac_dst (CHAR(17) NOT NULL)
* vlan => vlan (INT(2) UNSIGNED NOT NULL)
* in_vlan => vlan_in (INT(2) UNSIGNED NOT NULL)
* out_vlan => vlan_out (INT(2) UNSIGNED NOT NULL)
* src_as => as_src (INT(4) UNSIGNED NOT NULL)
* dst_as => as_dst (INT(4) UNSIGNED NOT NULL)
* src_host => ip_src (CHAR(45) NOT NULL)
- or (VARBINARY(16) NOT NULL, if sql_num_hosts: true)
* dst_host => ip_dst (CHAR(45) NOT NULL)
- or (VARBINARY(16) NOT NULL, if sql_num_hosts: true)
* src_net => net_src (CHAR(45) NOT NULL)
- or (VARBINARY(16) NOT NULL, if sql_num_hosts: true)
* dst_net => net_dst (CHAR(45) NOT NULL)
- or (VARBINARY(16) NOT NULL, if sql_num_hosts: true)
* src_port => src_port (INT(2) UNSIGNED NOT NULL)
- or src_port => port_src (INT(2) UNSIGNED NOT NULL, if sql_table_version >= 8)
* dst_port => dst_port (INT(2) UNSIGNED NOT NULL)
- or dst_port => port_dst (INT(2) UNSIGNED NOT NULL, if sql_table_version >= 8)
* tcpflags => tcp_flags (INT(2) UNSIGNED NOT NULL)
* proto => ip_proto (CHAR(6) NOT NULL)
- or (INT(1) UNSIGNED NOT NULL, if sql_num_protos: true)
* tos => tos (INT(4) UNSIGNED NOT NULL)
* post_nat_src_host => post_nat_ip_src (CHAR(45) NOT NULL)
- or (VARBINARY(16) NOT NULL, if sql_num_hosts: true)
* post_nat_dst_host => post_nat_ip_dst (CHAR(45) NOT NULL)
- or (VARBINARY(16) NOT NULL, if sql_num_hosts: true)
* post_nat_src_port => post_nat_port_src (INT(2) UNSIGNED NOT NULL)
* post_nat_dst_port => post_nat_port_dst (INT(2) UNSIGNED NOT NULL)
* nat_event => nat_event (INT(1) UNSIGNED NOT NULL)
* fwd_status => fwd_status (INT(1) UNSIGNED NOT NULL)
- or (VARCHAR(50) NOT NULL, if fwd_status_encode_as_string: true)
* mpls_label_top => mpls_label_top (INT(4) UNSIGNED NOT NULL)
* mpls_label_bottom => mpls_label_bottom (INT(4) UNSIGNED NOT NULL)
* mpls_label_stack => mpls_label_stack (VARCHAR(255) NOT NULL)
* tunnel_src_mac => tunnel_mac_src (CHAR(17) NOT NULL)
* tunnel_dst_mac => tunnel_mac_dst (CHAR(17) NOT NULL)
* tunnel_src_host => tunnel_ip_src (CHAR(45) NOT NULL)
- or (VARBINARY(16) NOT NULL, if sql_num_hosts: true)
* tunnel_dst_host => tunnel_ip_dst (CHAR(45) NOT NULL)
- or (VARBINARY(16) NOT NULL, if sql_num_hosts: true)
* tunnel_proto => tunnel_ip_proto (CHAR(6) NOT NULL)
- or (INT(1) UNSIGNED NOT NULL, if sql_num_protos: true)
* tunnel_tos => tunnel_tos (INT(4) UNSIGNED NOT NULL)
* tunnel_src_port => tunnel_port_src (INT(2) UNSIGNED NOT NULL)
* tunnel_dst_port => tunnel_port_dst (INT(2) UNSIGNED NOT NULL)
* tunnel_tcpflags => tunnel_tcp_flags (INT(2) UNSIGNED NOT NULL)
* timestamp_start => timestamp_start, timestamp_start_residual:
- timestamp_start DATETIME NOT NULL, see README.timestamp)
- timestamp_start_residual INT NOT NULL, see README.timestamp)
* timestamp_end => timestamp_end, timestamp_end_residual:
- timestamp_end DATETIME NOT NULL, see README.timestamp)
- timestamp_end_residual INT NOT NULL, see README.timestamp)
* timestamp_arrival => timestamp_arrival, timestamp_arrival_residual:
- timestamp_arrival DATETIME NOT NULL, see README.timestamp)
- timestamp_arrival_residual INT NOT NULL, see README.timestamp)
* timestamp_min => timestamp_min, timestamp_min_residual:
- timestamp_min DATETIME NOT NULL, see README.timestamp)
- timestamp_min_residual INT NOT NULL, see README.timestamp)
* timestamp_max => timestamp_max, timestamp_max_residual:
- timestamp_max DATETIME NOT NULL, see README.timestamp)
- timestamp_max_residual INT NOT NULL, see README.timestamp)
* export_proto_seqno => export_proto_seqno (INT(4) UNSIGNED NOT NULL, see README.export_proto)
* export_proto_version => export_proto_version (INT(2) UNSIGNED NOT NULL, see README.export_proto)
* export_proto_sysid => export_proto_sysid (INT(4) UNSIGNED NOT NULL, see README.export_proto)
- 'packets' and 'bytes' counters need to be defined as part of the SQL schema
whenever traffic flows are being accounted for; they are not required, and
are zeroed, if accounting for events, ie. using Cisco NEL; 'stamp_inserted'
and 'stamp_updated' time references are mandatory only if temporal aggregation
(sql_history) is enabled:
* flows (INT UNSIGNED NOT NULL)
- or (flows BIGINT UNSIGNED NOT NULL)
* packets (INT UNSIGNED NOT NULL)
- or (packets BIGINT UNSIGNED NOT NULL)
* bytes (BIGINT UNSIGNED NOT NULL)
* stamp_inserted (DATETIME NOT NULL)
- or (stamp_inserted BIGINT UNSIGNED NOT NULL, if timestamps_since_epoch: true)
* stamp_updated (DATETIME)
- or (stamp_updated BIGINT UNSIGNED, if timestamps_since_epoch: true)
- For custom-defined primitives refer to the README.custom_primitives doc.
NOTE: certain primitives, ie. BGP attributtes like AS-PATH and communities
(as_path, std_comm, etc.), can get arbitrarily long if not properly scoped
(ie. bgp_aspath_radius, bgp_stdcomm_pattern, etc.) and hence not fit in
default field definitions (ie. CHAR(21) or CHAR(24)). It is possible to
define these as arbitrarily-long variable-length strings using VARCHAR or
TEXT data types. Consult latest MySQL/MariaDB docs for examples and notes
(charset choices, etc.).
NOTE: mind to specify EVERYTIME which SQL table version you
intend to adhere to by using the following config directives:
* sql_table_version: [ 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 ]
* sql_table_type: [ bgp ]
NOTE: specifying a non-documented SQL table profile will result
in an non-determined behaviour. Unless this will create crashes
to the application, such situations will not be supported.
|
