1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
|
.. SPDX-License-Identifier: BSD-3-Clause
Copyright 2018 The DPDK contributors
DPDK Release 19.02
==================
New Features
------------
* **Added support for freeing hugepages exactly as originally allocated.**
Some applications using memory event callbacks (especially for managing
RDMA memory regions) require that memory be freed back to the system
exactly as it was originally allocated. These applications typically
also require that a malloc allocation not span across two separate
hugepage allocations. A new ``--match-allocations`` EAL init flag has
been added to fulfill both of these requirements.
* **Added API to register external memory in DPDK.**
A new ``rte_extmem_register``/``rte_extmem_unregister`` API was added to allow
chunks of external memory to be registered with DPDK without adding them to
the malloc heap.
* **Added support for using virtio-user without hugepages.**
The ``--no-huge`` mode was augmented to use memfd-backed memory (on systems
that support memfd), to allow using virtio-user-based NICs without
hugepages.
* **Release of the ENA PMD v2.0.0.**
Version 2.0.0 of the ENA PMD was added with the following additions:
* Added Low Latency Queue v2 (LLQv2). This feature reduces the latency
of the packets by pushing the header directly through the PCI to the
device. This allows the NIC to start handle packets right after the doorbell
without waiting for DMA.
* Added independent configuration of HW Tx and Rx ring depths.
* Added support for up to 8k Rx descriptors per ring.
* Added additional doorbell check on Tx, to handle Tx more efficiently for big
bursts of packets.
* Added per queue statistics.
* Added extended statistics using xstats DPDK API.
* The reset routine was aligned with the DPDK API, so now it can be
handled as in other PMDs.
* Fixed out of order (OOO) completion.
* Fixed memory leaks due to port stops and starts in the middle of
traffic.
* Updated documentation and features list of the PMD.
* **Updated mlx5 driver.**
Updated the mlx5 driver including the following changes:
* Fixed ``imissed`` counter to be reported through ``rte_eth_stats`` instead
of ``rte_eth_xstats``.
* Added packet header modification through Direct Verbs flow driver.
* Added ConnectX-6 PCI device ID to be proved by ``mlx5`` driver.
* Added flow counter support to Direct Verbs flow driver though DevX.
* Renamed build options for the glue layer to
``CONFIG_RTE_IBVERBS_LINK_DLOPEN`` for make and ``ibverbs_link`` for meson.
* Added static linkage of ``mlx`` dependency.
* Improved stability of E-Switch flow driver.
* Added new make build configuration to set the cacheline size for BlueField
correctly - ``arm64-bluefield-linux-gcc``.
* **Updated the enic driver.**
* Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
* Added a handler to get the firmware version string.
* Added support for multicast filtering.
* **Added dynamic queues allocation support for i40e VF.**
Previously, the available VF queues were reserved by PF at initialization
stage. Now both DPDK PF and Kernel PF (>=2.1.14) will support dynamic queue
allocation. At runtime, when VF requests for more queue exceed the initial
reserved amount, the PF can allocate up to 16 queues as the request after a
VF reset.
* **Added ICE net PMD.**
Added the new ``ice`` net driver for Intel(R) Ethernet Network Adapters E810.
See the :doc:`../nics/ice` NIC guide for more details on this new driver.
* **Added support for SW-assisted VDPA live migration.**
This SW-assisted VDPA live migration facility helps VDPA devices without
logging capability to perform live migration, a mediated SW relay can help
devices to track dirty pages caused by DMA. the IFC driver has enabled this
SW-assisted live migration mode.
* **Added security checks to the cryptodev symmetric session operations.**
Added a set of security checks to the access cryptodev symmetric session.
The checks include the session's user data read/write check and the
session private data referencing status check while freeing a session.
* **Updated the AESNI-MB PMD.**
* Added support for intel-ipsec-mb version 0.52.
* Added AES-GMAC algorithm support.
* Added Plain SHA1, SHA224, SHA256, SHA384, and SHA512 algorithms support.
* **Added IPsec Library.**
Added an experimental library ``librte_ipsec`` to provide ESP tunnel and
transport support for IPv4 and IPv6 packets.
The library provides support for AES-CBC ciphering and AES-CBC with HMAC-SHA1
algorithm-chaining, and AES-GCM and NULL algorithms only at present. It is
planned to add more algorithms in future releases.
See :doc:`../prog_guide/ipsec_lib` for more information.
* **Updated the ipsec-secgw sample application.**
The ``ipsec-secgw`` sample application has been updated to use the new
``librte_ipsec`` library, which has also been added in this release.
The original functionality of ipsec-secgw is retained, a new command line
parameter ``-l`` has been added to ipsec-secgw to use the IPsec library,
instead of the existing IPsec code in the application.
The IPsec library does not support all the functionality of the existing
ipsec-secgw application. It is planned to add the outstanding functionality
in future releases.
See :doc:`../sample_app_ug/ipsec_secgw` for more information.
* **Enabled checksum support in the ISA-L compressdev driver.**
Added support for both adler and crc32 checksums in the ISA-L PMD.
This aids data integrity across both compression and decompression.
* **Added a compression performance test tool.**
Added a new performance test tool to test the compressdev PMD. The tool tests
compression ratio and compression throughput.
* **Added intel_pstate support to Power Management library.**
Previously, using the power management library required the
disabling of the intel_pstate kernel driver, and the enabling of the
acpi_cpufreq kernel driver. This is no longer the case, as the use of
the intel_pstate kernel driver is now supported, and automatically
detected by the library.
API Changes
-----------
* eal: Function ``rte_bsf64`` in ``rte_bitmap.h`` has been renamed to
``rte_bsf64_safe`` and moved to ``rte_common.h``. A new ``rte_bsf64``
function has been added in ``rte_common.h`` that follows the convention set
by the existing ``rte_bsf32`` function.
* eal: Segment fd API on Linux now sets error code to ``ENOTSUP`` in more cases
where segment the fd API is not expected to be supported:
- On attempt to get a segment fd for an externally allocated memory segment
- In cases where memfd support would have been required to provide segment
fds (such as in-memory or no-huge mode)
* eal: Functions ``rte_malloc_dump_stats()``, ``rte_malloc_dump_heaps()`` and
``rte_malloc_get_socket_stats()`` are no longer safe to call concurrently with
``rte_malloc_heap_create()`` or ``rte_malloc_heap_destroy()`` function calls.
* mbuf: ``RTE_MBUF_INDIRECT()``, which was deprecated in 18.05, was replaced
with ``RTE_MBUF_CLONED()`` and removed in 19.02.
* sched: As result of the new format of the mbuf sched field, the
functions ``rte_sched_port_pkt_write()`` and
``rte_sched_port_pkt_read_tree_path()`` got an additional parameter of
type ``struct rte_sched_port``.
* pdump: The ``rte_pdump_set_socket_dir()``, the parameter ``path`` of
``rte_pdump_init()`` and enum ``rte_pdump_socktype`` were deprecated
since 18.05 and are removed in this release.
* cryptodev: The parameter ``session_pool`` in the function
``rte_cryptodev_queue_pair_setup()`` is removed.
* cryptodev: a new function ``rte_cryptodev_sym_session_pool_create()`` has been
introduced. This function is now mandatory when creating symmetric session
header mempool. Please note all crypto applications are required to use this
function from now on. Failed to do so will cause a
``rte_cryptodev_sym_session_create()`` function call return error.
ABI Changes
-----------
* mbuf: The format of the sched field of ``rte_mbuf`` has been changed
to include the following fields: ``queue ID``, ``traffic class``, ``color``.
* cryptodev: as shown in the 18.11 deprecation notice, the structure
``rte_cryptodev_qp_conf`` has added two parameters for symmetric session
mempool and symmetric session private data mempool.
* cryptodev: as shown in the 18.11 deprecation notice, the structure
``rte_cryptodev_sym_session`` has been updated to contain more information
to ensure safely accessing the session and session private data.
* security: A new field ``uint64_t opaque_data`` has been added to
``rte_security_session`` structure. That would allow upper layer to easily
associate/de-associate some user defined data with the security session.
Shared Library Versions
-----------------------
The libraries prepended with a plus sign were incremented in this version.
.. code-block:: diff
librte_acl.so.2
librte_bbdev.so.1
librte_bitratestats.so.2
librte_bpf.so.1
librte_bus_dpaa.so.2
librte_bus_fslmc.so.2
librte_bus_ifpga.so.2
librte_bus_pci.so.2
librte_bus_vdev.so.2
librte_bus_vmbus.so.2
librte_cfgfile.so.2
librte_cmdline.so.2
librte_compressdev.so.1
+ librte_cryptodev.so.6
librte_distributor.so.1
librte_eal.so.9
librte_efd.so.1
librte_ethdev.so.11
librte_eventdev.so.6
librte_flow_classify.so.1
librte_gro.so.1
librte_gso.so.1
librte_hash.so.2
librte_ip_frag.so.1
librte_jobstats.so.1
librte_kni.so.2
librte_kvargs.so.1
librte_latencystats.so.1
librte_lpm.so.2
+ librte_mbuf.so.5
librte_member.so.1
librte_mempool.so.5
librte_meter.so.2
librte_metrics.so.1
librte_net.so.1
librte_pci.so.1
+ librte_pdump.so.3
librte_pipeline.so.3
librte_pmd_bnxt.so.2
librte_pmd_bond.so.2
librte_pmd_i40e.so.2
librte_pmd_ixgbe.so.2
librte_pmd_dpaa2_qdma.so.1
librte_pmd_ring.so.2
librte_pmd_softnic.so.1
librte_pmd_vhost.so.2
librte_port.so.3
librte_power.so.1
librte_rawdev.so.1
librte_reorder.so.1
librte_ring.so.2
+ librte_sched.so.2
+ librte_security.so.2
librte_table.so.3
librte_timer.so.1
librte_vhost.so.4
Known Issues
------------
* ``AVX-512`` support has been disabled for ``GCC`` builds when ``binutils 2.30``
is detected [1] because of a crash [2]. This can affect ``native`` machine type
build targets on the platforms that support ``AVX512F`` like ``Intel Skylake``
processors, and can cause a possible performance drop. The immediate workaround
is to use ``clang`` compiler on these platforms.
Initial workaround in DPDK v18.11 was to disable ``AVX-512`` support for ``GCC``
completely, but based on information on defect submitted to GCC community [3],
issue has been identified as ``binutils 2.30`` issue. Since currently only GCC
generates ``AVX-512`` instructions, the scope is limited to ``GCC`` and
``binutils 2.30``
- [1]: Commit ("mk: fix scope of disabling AVX512F support")
- [2]: https://bugs.dpdk.org/show_bug.cgi?id=97
- [3]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88096
Tested Platforms
----------------
* Intel(R) platforms with Intel(R) NICs combinations
* CPU
* Intel(R) Atom(TM) CPU C3758 @ 2.20GHz
* Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz
* Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
* Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
* Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
* Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz
* Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz
* OS:
* CentOS 7.4
* CentOS 7.5
* Fedora 25
* Fedora 28
* FreeBSD 11.2
* FreeBSD 12.0
* Red Hat Enterprise Linux Server release 7.4
* Red Hat Enterprise Linux Server release 7.5
* Open SUSE 15
* Wind River Linux 8
* Ubuntu 14.04
* Ubuntu 16.04
* Ubuntu 16.10
* Ubuntu 18.04
* Ubuntu 18.10
* NICs:
* Intel(R) 82599ES 10 Gigabit Ethernet Controller
* Firmware version: 0x61bf0001
* Device id (pf/vf): 8086:10fb / 8086:10ed
* Driver version: 5.2.3 (ixgbe)
* Intel(R) Corporation Ethernet Connection X552/X557-AT 10GBASE-T
* Firmware version: 0x800003e7
* Device id (pf/vf): 8086:15ad / 8086:15a8
* Driver version: 4.4.6 (ixgbe)
* Intel(R) Ethernet Converged Network Adapter X710-DA4 (4x10G)
* Firmware version: 6.80 0x80003cc1
* Device id (pf/vf): 8086:1572 / 8086:154c
* Driver version: 2.7.26 (i40e)
* Intel(R) Corporation Ethernet Connection X722 for 10GbE SFP+ (4x10G)
* Firmware version: 3.33 0x80000fd5 0.0.0
* Device id (pf/vf): 8086:37d0 / 8086:37cd
* Driver version: 2.7.26 (i40e)
* Intel(R) Ethernet Converged Network Adapter XXV710-DA2 (2x25G)
* Firmware version: 6.80 0x80003d05
* Device id (pf/vf): 8086:158b / 8086:154c
* Driver version: 2.7.26 (i40e)
* Intel(R) Ethernet Converged Network Adapter XL710-QDA2 (2X40G)
* Firmware version: 6.80 0x80003cfb
* Device id (pf/vf): 8086:1583 / 8086:154c
* Driver version: 2.7.26 (i40e)
* Intel(R) Corporation I350 Gigabit Network Connection
* Firmware version: 1.63, 0x80000dda
* Device id (pf/vf): 8086:1521 / 8086:1520
* Driver version: 5.4.0-k (igb)
* Intel(R) platforms with Mellanox(R) NICs combinations
* CPU:
* Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz
* Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz
* Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
* Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
* Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
* Intel(R) Xeon(R) CPU E5-2640 @ 2.50GHz
* Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
* OS:
* Red Hat Enterprise Linux Server release 7.6 (Maipo)
* Red Hat Enterprise Linux Server release 7.5 (Maipo)
* Red Hat Enterprise Linux Server release 7.4 (Maipo)
* Red Hat Enterprise Linux Server release 7.3 (Maipo)
* Red Hat Enterprise Linux Server release 7.2 (Maipo)
* Ubuntu 18.10
* Ubuntu 18.04
* Ubuntu 17.10
* Ubuntu 16.04
* SUSE Linux Enterprise Server 15
* MLNX_OFED: 4.4-2.0.1.0
* MLNX_OFED: 4.5-1.0.1.0
* NICs:
* Mellanox(R) ConnectX(R)-3 Pro 40G MCX354A-FCC_Ax (2x40G)
* Host interface: PCI Express 3.0 x8
* Device ID: 15b3:1007
* Firmware version: 2.42.5000
* Mellanox(R) ConnectX(R)-4 10G MCX4111A-XCAT (1x10G)
* Host interface: PCI Express 3.0 x8
* Device ID: 15b3:1013
* Firmware version: 12.24.1000 and above
* Mellanox(R) ConnectX(R)-4 10G MCX4121A-XCAT (2x10G)
* Host interface: PCI Express 3.0 x8
* Device ID: 15b3:1013
* Firmware version: 12.24.1000 and above
* Mellanox(R) ConnectX(R)-4 25G MCX4111A-ACAT (1x25G)
* Host interface: PCI Express 3.0 x8
* Device ID: 15b3:1013
* Firmware version: 12.24.1000 and above
* Mellanox(R) ConnectX(R)-4 25G MCX4121A-ACAT (2x25G)
* Host interface: PCI Express 3.0 x8
* Device ID: 15b3:1013
* Firmware version: 12.24.1000 and above
* Mellanox(R) ConnectX(R)-4 40G MCX4131A-BCAT/MCX413A-BCAT (1x40G)
* Host interface: PCI Express 3.0 x8
* Device ID: 15b3:1013
* Firmware version: 12.24.1000 and above
* Mellanox(R) ConnectX(R)-4 40G MCX415A-BCAT (1x40G)
* Host interface: PCI Express 3.0 x16
* Device ID: 15b3:1013
* Firmware version: 12.24.1000 and above
* Mellanox(R) ConnectX(R)-4 50G MCX4131A-GCAT/MCX413A-GCAT (1x50G)
* Host interface: PCI Express 3.0 x8
* Device ID: 15b3:1013
* Firmware version: 12.24.1000 and above
* Mellanox(R) ConnectX(R)-4 50G MCX414A-BCAT (2x50G)
* Host interface: PCI Express 3.0 x8
* Device ID: 15b3:1013
* Firmware version: 12.24.1000 and above
* Mellanox(R) ConnectX(R)-4 50G MCX415A-GCAT/MCX416A-BCAT/MCX416A-GCAT (2x50G)
* Host interface: PCI Express 3.0 x16
* Device ID: 15b3:1013
* Firmware version: 12.24.1000 and above
* Firmware version: 12.24.1000 and above
* Mellanox(R) ConnectX(R)-4 50G MCX415A-CCAT (1x100G)
* Host interface: PCI Express 3.0 x16
* Device ID: 15b3:1013
* Firmware version: 12.24.1000 and above
* Mellanox(R) ConnectX(R)-4 100G MCX416A-CCAT (2x100G)
* Host interface: PCI Express 3.0 x16
* Device ID: 15b3:1013
* Firmware version: 12.24.1000 and above
* Mellanox(R) ConnectX(R)-4 Lx 10G MCX4121A-XCAT (2x10G)
* Host interface: PCI Express 3.0 x8
* Device ID: 15b3:1015
* Firmware version: 14.24.1000 and above
* Mellanox(R) ConnectX(R)-4 Lx 25G MCX4121A-ACAT (2x25G)
* Host interface: PCI Express 3.0 x8
* Device ID: 15b3:1015
* Firmware version: 14.24.1000 and above
* Mellanox(R) ConnectX(R)-5 100G MCX556A-ECAT (2x100G)
* Host interface: PCI Express 3.0 x16
* Device ID: 15b3:1017
* Firmware version: 16.24.1000 and above
* Mellanox(R) ConnectX(R)-5 Ex EN 100G MCX516A-CDAT (2x100G)
* Host interface: PCI Express 4.0 x16
* Device ID: 15b3:1019
* Firmware version: 16.24.1000 and above
* ARM platforms with Mellanox(R) NICs combinations
* CPU:
* Qualcomm ARM 1.1 2500MHz
* OS:
* Red Hat Enterprise Linux Server release 7.5 (Maipo)
* NICs:
* Mellanox(R) ConnectX(R)-4 Lx 25G MCX4121A-ACAT (2x25G)
* Host interface: PCI Express 3.0 x8
* Device ID: 15b3:1015
* Firmware version: 14.24.0220
* Mellanox(R) ConnectX(R)-5 100G MCX556A-ECAT (2x100G)
* Host interface: PCI Express 3.0 x16
* Device ID: 15b3:1017
* Firmware version: 16.24.0220
* Mellanox(R) BlueField SmartNIC
* Mellanox(R) BlueField SmartNIC MT416842 (2x25G)
* Host interface: PCI Express 3.0 x16
* Device ID: 15b3:a2d2
* Firmware version: 18.24.0328
* SoC ARM cores running OS:
* CentOS Linux release 7.4.1708 (AltArch)
* MLNX_OFED 4.4-2.5.9.0
* DPDK application running on ARM cores inside SmartNIC
* Power 9 platforms with Mellanox(R) NICs combinations
* CPU:
* POWER9 2.2 (pvr 004e 1202) 2300MHz
* OS:
* Ubuntu 18.04.1 LTS (Bionic Beaver)
* NICs:
* Mellanox(R) ConnectX(R)-5 100G MCX556A-ECAT (2x100G)
* Host interface: PCI Express 3.0 x16
* Device ID: 15b3:1017
* Firmware version: 16.23.1020
* OFED:
* MLNX_OFED_LINUX-4.5-1.0.1.0
|
