1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
|
.. SPDX-License-Identifier: BSD-3-Clause
Copyright 2017,2020-2024 NXP
DPAA Poll Mode Driver
=====================
The DPAA NIC PMD (**librte_net_dpaa**) provides poll mode driver
support for the inbuilt NIC found in the **NXP DPAA** SoC family.
More information can be found at `NXP Official Website
<http://www.nxp.com/products/microcontrollers-and-processors/arm-processors/qoriq-arm-processors:QORIQ-ARM>`_.
NXP DPAA (Data Path Acceleration Architecture - Gen 1)
------------------------------------------------------
This section provides an overview of the NXP DPAA architecture
and how it is integrated into the DPDK.
Contents summary
- DPAA overview
- DPAA driver architecture overview
- FMAN configuration tools and library
.. _dpaa_overview:
DPAA Overview
~~~~~~~~~~~~~
Reference: `FSL DPAA Architecture <http://www.nxp.com/assets/documents/data/en/white-papers/QORIQDPAAWP.pdf>`_.
The QorIQ Data Path Acceleration Architecture (DPAA) is a set of hardware
components on specific QorIQ series multicore processors. This architecture
provides the infrastructure to support simplified sharing of networking
interfaces and accelerators by multiple CPU cores, and the accelerators
themselves.
DPAA includes:
- Cores
- Network and packet I/O
- Hardware offload accelerators
- Infrastructure required to facilitate flow of packets between the components above
Infrastructure components are:
- The Queue Manager (QMan) is a hardware accelerator that manages frame queues.
It allows CPUs and other accelerators connected to the SoC datapath to
enqueue and dequeue ethernet frames, thus providing the infrastructure for
data exchange among CPUs and datapath accelerators.
- The Buffer Manager (BMan) is a hardware buffer pool management block that
allows software and accelerators on the datapath to acquire and release
buffers in order to build frames.
Hardware accelerators are:
- SEC - Cryptographic accelerator
- PME - Pattern matching engine
The Network and packet I/O component:
- The Frame Manager (FMan) is a key component in the DPAA and makes use of the
DPAA infrastructure (QMan and BMan). FMan is responsible for packet
distribution and policing. Each frame can be parsed, classified and results
may be attached to the frame. This meta data can be used to select
particular QMan queue, which the packet is forwarded to.
DPAA DPDK - Poll Mode Driver Overview
-------------------------------------
This section provides an overview of the drivers for DPAA:
* Bus driver and associated "DPAA infrastructure" drivers
* Functional object drivers (such as Ethernet).
Brief description of each driver is provided in layout below as well as
in the following sections.
.. code-block:: console
+------------+
| DPDK DPAA |
| PMD |
+-----+------+
|
+-----+------+ +---------------+
: Ethernet :.......| DPDK DPAA |
. . . . . . . . . : (FMAN) : | Mempool driver|
. +---+---+----+ | (BMAN) |
. ^ | +-----+---------+
. | |<enqueue, .
. | | dequeue> .
. | | .
. +---+---V----+ .
. . . . . . . . . . .: Portal drv : .
. . : : .
. . +-----+------+ .
. . : QMAN : .
. . : Driver : .
+----+------+-------+ +-----+------+ .
| DPDK DPAA Bus | | .
| driver |....................|.....................
| /bus/dpaa | |
+-------------------+ |
|
========================== HARDWARE =====|========================
PHY
=========================================|========================
In the above representation, solid lines represent components which interface
with DPDK RTE Framework and dotted lines represent DPAA internal components.
DPAA Bus driver
~~~~~~~~~~~~~~~
The DPAA bus driver is a ``rte_bus`` driver which scans the platform like bus.
Key functions include:
- Scanning and parsing the various objects and adding them to their respective
device list.
- Performing probe for available drivers against each scanned device
- Creating necessary ethernet instance before passing control to the PMD
DPAA NIC Driver (PMD)
~~~~~~~~~~~~~~~~~~~~~
DPAA PMD is traditional DPDK PMD which provides necessary interface between
RTE framework and DPAA internal components/drivers.
- Once devices have been identified by DPAA Bus, each device is associated
with the PMD
- PMD is responsible for implementing necessary glue layer between RTE APIs
and lower level QMan and FMan blocks.
The Ethernet driver is bound to a FMAN port and implements the interfaces
needed to connect the DPAA network interface to the network stack.
Each FMAN Port corresponds to a DPDK network interface.
- PMD also support OH/ONIC mode, where the port works as a HW assisted virtual port
without actually connecting to a Physical MAC.
Features
^^^^^^^^
Features of the DPAA PMD are:
- Multiple queues for TX and RX
- Receive Side Scaling (RSS)
- Packet type information
- Checksum offload
- Promiscuous mode
- IEEE1588 PTP
- OH Port for inter application communication
- ONIC virtual port support
DPAA Mempool Driver
~~~~~~~~~~~~~~~~~~~
DPAA has a hardware offloaded buffer pool manager, called BMan, or Buffer
Manager.
- Using standard Mempools operations RTE API, the mempool driver interfaces
with RTE to service each mempool creation, deletion, buffer allocation and
deallocation requests.
- Each FMAN instance has a BMan pool attached to it during initialization.
Each Tx frame can be automatically released by hardware, if allocated from
this pool.
Allowing & Blocking
-------------------
For blocking a DPAA device, following commands can be used.
.. code-block:: console
<dpdk app> <EAL args> -b "dpaa_bus:fmX-macY" -- ...
e.g. "dpaa_bus:fm1-mac4"
Supported DPAA SoCs
-------------------
- LS1043A/LS1023A
- LS1046A/LS1026A
Prerequisites
-------------
See :doc:`../platform/dpaa` for setup information
- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>`
to setup the basic DPDK environment.
- DPAA driver has dependency on kernel to perform various functionalities.
So kernel and DPDK version should be compatible for proper working.
Refer release notes of NXP SDK guide to match the versions `NXP LSDK GUIDE
<https://www.nxp.com/design/software/embedded-software/linux-software-and-development-tools/layerscape-software-development-kit-v21-08:LAYERSCAPE-SDK>`_.
.. note::
Some part of dpaa bus code (qbman and fman - library) routines are
dual licensed (BSD & GPLv2), however they are used as BSD in DPDK in userspace.
Configuration
-------------
Environment Variables
~~~~~~~~~~~~~~~~~~~~~
DPAA drivers uses the following environment variables to configure its
state during application initialization:
- ``DPAA_NUM_RX_QUEUES`` (default 1)
This defines the number of Rx queues configured for an application, per
port. Hardware would distribute across these many number of queues on Rx
of packets.
In case the application is configured to use lesser number of queues than
configured above, it might result in packet loss (because of distribution).
- ``DPAA_PUSH_QUEUES_NUMBER`` (default 4)
This defines the number of High performance queues to be used for ethdev Rx.
These queues use one private HW portal per queue configured, so they are
limited in the system. The first configured ethdev queues will be
automatically be assigned from the these high perf PUSH queues. Any queue
configuration beyond that will be standard Rx queues. The application can
choose to change their number if HW portals are limited.
The valid values are from '0' to '4'. The values shall be set to '0' if the
application want to use eventdev with DPAA device.
Currently these queues are not used for LS1023/LS1043 platform by default.
- ``DPAA_DISPLAY_FRAME_AND_PARSER_RESULT`` (default 0)
This defines the debug flag, whether to dump the detailed frame
and packet parsing result for the incoming packets.
Driver compilation and testing
------------------------------
Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
for details.
#. Running testpmd:
Follow instructions available in the document
:ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
to run testpmd.
Example output:
.. code-block:: console
./<build_dir>/app/dpdk-testpmd -c 0xff -n 1 \
-- -i --portmask=0x3 --nb-cores=1 --no-flush-rx
.....
EAL: Registered [pci] bus.
EAL: Registered [dpaa] bus.
EAL: Detected 4 lcore(s)
.....
EAL: dpaa: Bus scan completed
.....
Configuring Port 0 (socket 0)
Port 0: 00:00:00:00:00:01
Configuring Port 1 (socket 0)
Port 1: 00:00:00:00:00:02
.....
Checking link statuses...
Port 0 Link Up - speed 10000 Mbps - full-duplex
Port 1 Link Up - speed 10000 Mbps - full-duplex
Done
testpmd>
* Use dev arg option ``drv_ieee1588=1`` to enable IEEE 1588 support
at driver level, e.g. ``dpaa:fm1-mac3,drv_ieee1588=1``.
FMAN Config
-----------
Frame Manager is also responsible for parser, classify and distribute
functionality in the DPAA.
FMAN supports:
Packet parsing at wire speed. It supports standard protocols parsing and
identification by HW (VLAN/IP/UDP/TCP/SCTP/PPPoE/PPP/MPLS/GRE/IPSec).
It supports non-standard UDF header parsing for custom protocols.
Classification / Distribution: Coarse classification based on Key generation
Hash and exact match lookup
FMC - FMAN Configuration Tool
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This tool is available in User Space. The tool is used to configure FMAN
Physical (MAC) or Ephemeral (OH)ports for Parse/Classify/distribute.
The PCDs can be hash based where a set of fields are key input for hash
generation within FMAN keygen. The hash value is used to generate a FQID for
frame. There is a provision to setup exact match lookup too where field
values within a packet drives corresponding FQID.
Currently it works on XML file inputs.
Limitations:
1.For Dynamic Configuration change, currently no support is available.
E.g. enable/disable a port, a operator (set of VLANs and associate rules).
2.During FMC configuration, port for which policy is being configured is
brought down and the policy is flushed on port before new policy is updated
for the port. Support is required to add/append/delete etc.
3.FMC, being a separate user-space application, needs to be invoked from
Shell.
The details can be found in FMC Doc at:
`Frame Manager Configuration Tool <https://www.nxp.com/docs/en/application-note/AN4760.pdf>`_.
FMLIB
~~~~~
The Frame Manager library provides an API on top of the Frame Manager driver
ioctl calls, that provides a user space application with a simple way to
configure driver parameters and PCD (parse - classify - distribute) rules.
This is an alternate to the FMC based configuration. This library provides
direct ioctl based interfaces for FMAN configuration as used by the FMC tool
as well. This helps in overcoming the main limitation of FMC - i.e. lack
of dynamic configuration.
The location for the fmd driver as used by FMLIB and FMC is as follows:
`Kernel FMD Driver
<https://source.codeaurora.org/external/qoriq/qoriq-components/linux/tree/drivers/net/ethernet/freescale/sdk_fman?h=linux-4.19-rt>`_.
OH Port
~~~~~~~
Offline(O/H) port is a type of hardware port
which is able to dequeue and enqueue from/to a QMan queue.
The FMan applies a Parse Classify Distribute (PCD) flow
and (if configured to do so) enqueues the frame back in a QMan queue.
The FMan is able to copy the frame into new buffers and enqueue back to the QMan.
This means these ports can be used to send and receive packets
between two applications as well.
An O/H port have two queues.
One to receive and one to send the packets.
It will loopback all the packets on Tx queue which are received on Rx queue.
-------- Tx Packets ---------
| App | - - - - - - - - - > | O/H |
| | < - - - - - - - - - | Port |
-------- Rx Packets ---------
ONIC
~~~~
To use OH port to communicate between two applications,
we can assign Rx port of an O/H port to Application 1
and Tx port to Application 2
so that Application 1 can send packets to Application 2.
Similarly, we can assign Tx port of another O/H port to Application 1
and Rx port to Application 2
so that Application 2 can send packets to Application 1.
ONIC is logically defined to achieve it.
Internally it will use one Rx queue of an O/H port
and one Tx queue of another O/H port.
For application, it will behave as single O/H port.
+------+ +------+ +------+ +------+ +------+
| | Tx | | Rx | O/H | Tx | | Rx | |
| | - - - > | | - - > | Port | - - > | | - - > | |
| | | | | 1 | | | | |
| | | | +------+ | | | |
| App | | ONIC | | ONIC | | App |
| 1 | | Port | | Port | | 2 |
| | | 1 | +------+ | 2 | | |
| | Rx | | Tx | O/H | Rx | | Tx | |
| | < - - - | | < - - -| Port | < - - -| | < - - -| |
| | | | | 2 | | | | |
+------+ +------+ +------+ +------+ +------+
All the packets received by ONIC port 1 will be send to ONIC port 2 and vice versa.
These ports can be used by DPDK applications just like physical ports.
VSP (Virtual Storage Profile)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The storage profiled are means to provide virtualized interface. A ranges of
storage profiles cab be associated to Ethernet ports.
They are selected during classification. Specify how the frame should be
written to memory and which buffer pool to select for packet storage in
queues. Start and End margin of buffer can also be configured.
Limitations
-----------
Platform Requirement
~~~~~~~~~~~~~~~~~~~~
DPAA drivers for DPDK can only work on NXP SoCs as listed in the
``Supported DPAA SoCs``.
Maximum packet length
~~~~~~~~~~~~~~~~~~~~~
The DPAA SoC family support a maximum of a 10240 jumbo frame. The value
is fixed and cannot be changed. So, even when the ``rxmode.mtu``
member of ``struct rte_eth_conf`` is set to a value lower than 10240, frames
up to 10240 bytes can still reach the host interface.
Multiprocess Support
~~~~~~~~~~~~~~~~~~~~
Current version of DPAA driver doesn't support multi-process applications
where I/O is performed using secondary processes. This feature would be
implemented in subsequent versions.
|
