File: README

package info (click to toggle)
dapl 2.0.19-1
  • links: PTS
  • area: main
  • in suites: squeeze
  • size: 4,220 kB
  • ctags: 4,033
  • sloc: ansic: 39,217; sh: 9,657; makefile: 528
file content (434 lines) | stat: -rw-r--r-- 18,024 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434

==========
1.0 BUILD:
==========

The default build includes a non-debug version of libdat and libdapl-cma uDAPL provider. It will also builds test suites dtest and dapltest and provides manpages for each. This version requires libibverbs and librdmacm installation, IPoIB installation, and IPoIB configuration with an IP address. 

Building :
----------
./autogen.sh 
./configure 
make

Building debug version:
----------------------
./autogen.sh
./configure --enable-debug
make

Build example with OFED 1.2+ prefix (x86_64)
---------------------------------------------
./autogen.sh
./configure --prefix /usr --sysconf=/etc --libdir /usr/lib64 LDFLAGS=-L/usr/lib64 CPPFLAGS="-I/usr/include"
make

Installing:
----------
make install

Note: The development package installs DAT 2.0 include files under /usr/include/dat2 to co-exist with DAT 1.2 /usr/include/dat

NOTE: to link these libraries you must either use libtool and 
specify the full pathname of the library, or use the `-LLIBDIR' 
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,--rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.

===================
2.0 CONFIGURATION:
===================

sample /etc/dat.conf 

#
# DAT 1.2 and 2.0 configuration file
#
# Each entry should have the following fields:
#
# <ia_name> <api_version> <threadsafety> <default> <lib_path> \
#           <provider_version> <ia_params> <platform_params>
#
# For the uDAPL cma provder, specify <ia_params> as one of the following:
#       network address, network hostname, or netdev name and 0 for port
#
# Simple (OpenIB-cma) default with netdev name provided first on list
# to enable use of same dat.conf version on all nodes
#
# Add examples for multiple interfaces and IPoIB HA fail over, and bonding
#
OpenIB-cma u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "ib0 0" ""
OpenIB-cma-1 u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "ib1 0" ""
OpenIB-cma-2 u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "ib2 0" ""
OpenIB-cma-3 u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "ib3 0" ""
OpenIB-bond u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "bond0 0" ""
OpenIB-2-cma u2.0 nonthreadsafe default libdaplcma.so.2 dapl.2.0 "ib0 0" ""
OpenIB-2-cma-1 u2.0 nonthreadsafe default libdaplcma.so.2 dapl.2.0 "ib1 0" ""
OpenIB-2-cma-2 u2.0 nonthreadsafe default libdaplcma.so.2 dapl.2.0 "ib2 0" ""
OpenIB-2-cma-3 u2.0 nonthreadsafe default libdaplcma.so.2 dapl.2.0 "ib3 0" ""
OpenIB-2-bond u2.0 nonthreadsafe default libdaplcma.so.2 dapl.2.0 "bond0 0" ""


=============================
3.0 Bugs/Known issues
=============================



=============================
4.0 SAMPLE uDAPL APPLICATION:
=============================

There are 2 sample programs, with manpages, provided with this package.

(dapl/test/dtest/)

NAME
       dtest - simple uDAPL send/receive and RDMA test

SYNOPSIS
       dtest [-P provider] [-b buf size] [-B burst count][-v] [-c] [-p] [-d] [-s]

       dtest [-P provider] [-b buf size] [-B burst count][-v] [-c] [-p] [-d] [-h HOSTNAME]

DESCRIPTION
       dtest  is a simple test used to exercise and verify the uDAPL interfaces.  At least two instantia-
       tions of the test must be run. One acts as the server and the other the client. The server side of
       the  test,  once invoked listens for connection requests, until timing out or killed. Upon receipt
       of a cd connection request, the connection is established, the server and  client  sides  exchange
       information necessary to perform RDMA writes and reads.

OPTIONS
       -P=PROVIDER
              use PROVIDER to specify uDAPL interface using /etc/dat.conf (default OpenIB-cma)

       -b=BUFFER_SIZE
              use buffer size BUFFER_SIZE for RDMA(default 64)

       -B=BURST_COUNT
              use busrt count BURST_COUNT for interations (default 10)

       -v, verbose output(default off)

       -c, use consumer notification events (default off)

       -p, use polling (default wait for event)

       -d, delay in seconds before close (default off)

       -s, run as server (default - run as server)

       -h=HOSTNAME
              use HOSTNAME to specify server hostname or IP address (default - none)

EXAMPLES
       dtest -P OpenIB-cma -v -s
            Starts a server process with debug verbosity using provider OpenIB-cma.

       dtest -P OpenIB-cma -h server1-ib0

            Starts a client process, using OpenIB-cma provider to connect to hostname server1-ib0.

SEE ALSO
       dapltest(1)

AUTHORS
       Arlin Davis
              <ardavis@ichips.intel.com>

BUGS

/dapl/test/dapltest/

NAME
        dapltest - test for the Direct Access Programming Library (DAPL)

DESCRIPTION
       Dapltest  is  a  set  of tests developed to exercise, characterize, and verify the DAPL interfaces
       during development and porting.  At least two instantiations of the test must be run. One acts  as
       the  server, fielding requests and spawning server-side test threads as needed. Other client invo-
       cations connect to the server and issue test requests. The server side of the test, once  invoked,
       listens  continuously for client connection requests, until quit or killed. Upon receipt of a con-
       nection request, the connection is established, the server and client sides swap  version  numbers
       to  verify that they are able to communicate, and the client sends the test request to the server.
       If the version numbers match, and the test request is well-formed, the server spawns  the  threads
       needed to run the test before awaiting further connections.

USAGE
       dapltest [ -f script_file_name ] [ -T S|Q|T|P|L ] [ -D device_name ] [ -d ] [ -R HT|LL|EC|PM|BE ]

       With  no  arguments,  dapltest runs as a server using default values, and loops accepting requests
       from clients.

       The -f option allows all arguments to be placed in a file, to ease test automation.

       The following arguments are common to all tests:

       [ -T S|Q|T|P|L ]
              Test function to be performed:

              S      - server loop

              Q      - quit, client requests that server wait for any outstanding tests to complete, then
                     clean up and exit

              T      - transaction test, transfers data between client and server

              P      - performance test, times DTO operations

              L      -  limit  test,  exhausts  various  resources, runs in client w/o server interaction
                     Default: S

      [ -D device_name ]
              Specifies the interface adapter name as documented in the /etc/dat.conf  static  configura-
              tion file. This name corresponds to the provider library to open.  Default: none

       [ -d ] Enables  extra  debug  verbosity,  primarily tracing of the various DAPL operations as they
              progress.  Repeating this parameter increases debug spew.  Errors encountered result in the
              test  spewing some explanatory text and stopping; this flag provides more detail about what
              lead up to the error.  Default: zero

       [ -R BE ]
              Indicate the quality of service (QoS) desired.  Choices are:

              HT     - high throughput

              LL     - low latency

              EC     - economy (neither HT nor LL)

              PM     - premium

              BE     - best effort Default: BE

       Usage - Quit test client

           dapltest [Common_Args] [ -s server_name ]

           Quit testing (-T Q) connects to the server to ask it to clean up and
           exit (after it waits for any outstanding test runs to complete).
           In addition to being more polite than simply killing the server,
           this test exercises the DAPL object teardown code paths.
           There is only one argument other than those supported by all tests:

           -s server_name      Specifies the name of the server interface.
                               No default.

       Usage - Transaction test client

           dapltest [Common_Args] [ -s server_name ]
                    [ -t threads ] [ -w endpoints ] [ -i iterations ] [ -Q ]
                    [ -V ] [ -P ] OPclient OPserver [ op3,

           Transaction testing (-T T) transfers a variable amount of data between
:
          client and server.  The data transfer can be described as a sequence of
           individual operations; that entire sequence is transferred ’iterations’
           times by each thread over all of its endpoint(s).

           The following parameters determine the behavior of the transaction test:

           -s server_name      Specifies the name or IP address of the server interface.
                               No default.

           [ -t threads ]      Specify the number of threads to be used.
                               Default: 1

           [ -w endpoints ]    Specify the number of connected endpoints per thread.
                               Default: 1

           [ -i iterations ]   Specify the number of times the entire sequence
                               of data transfers will be made over each endpoint.
                               Default: 1000

           [ -Q ]              Funnel completion events into a CNO.
                               Default: use EVDs

           [ -V ]              Validate the data being transferred.
                               Default: ignore the data

           [ -P ]              Turn on DTO completion polling
                               Default: off

           OP1 OP2 [ OP3, ... ]
                               A single transaction (OPx) consists of:

                               server|client   Indicates who initiates the
                                               data transfer.

                               SR|RR|RW        Indicates the type of transfer:
                                               SR  send/recv
                                               RR  RDMA read
                                               RW  RDMA write
                               Defaults: none

                               [ seg_size [ num_segs ] ]
:

                                              Indicates the amount and format
                                               of the data to be transferred.
                                               Default:  4096  1
                                                         (i.e., 1 4KB buffer)

                               [ -f ]          For SR transfers only, indicates
                                               that a client’s send transfer
                                               completion should be reaped when
                                               the next recv completion is reaped.
                                               Sends and receives must be paired
                                               (one client, one server, and in that
                                               order) for this option to be used.

           Restrictions:

           Due to the flow control algorithm used by the transaction test, there
           must be at least one SR OP for both the client and the server.

           Requesting data validation (-V) causes the test to automatically append
           three OPs to those specified. These additional operations provide
           synchronization points during each iteration, at which all user-specified
           transaction buffers are checked. These three appended operations satisfy
           the "one SR in each direction" requirement.

           The transaction OP list is printed out if -d is supplied.

       Usage - Performance test client

           dapltest [Common_Args] -s server_name [ -m p|b ]
                    [ -i iterations ] [ -p pipeline ] OP

           Performance testing (-T P) times the transfer of an operation.
           The operation is posted ’iterations’ times.

           The following parameters determine the behavior of the transaction test:

           -s server_name      Specifies the name or IP address of the server interface.
                               No default.

           -m b|p              Used to choose either blocking (b) or polling (p)
                               Default: blocking (b)
          [ -i iterations ]   Specify the number of times the entire sequence
                               of data transfers will be made over each endpoint.
                               Default: 1000

           [ -p pipeline ]     Specify the pipline length, valid arguments are in
                               the range [0,MAX_SEND_DTOS]. If a value greater than
                               MAX_SEND_DTOS is requested the value will be
                               adjusted down to MAX_SEND_DTOS.
                               Default: MAX_SEND_DTOS

           OP                  Specifies the operation as follow:

                               RR|RW           Indicates the type of transfer:
                                               RR  RDMA read
                                               RW  RDMA write
                                               Defaults: none

                               [ seg_size [ num_segs ] ]
                                               Indicates the amount and format
                                               of the data to be transferred.
                                               Default:  4096  1
                                                         (i.e., 1 4KB buffer)

       Usage - Limit test client

           Limit testing (-T L) neither requires nor connects to any server
           instance.  The client runs one or more tests which attempt to
           exhaust various resources to determine DAPL limits and exercise
           DAPL error paths.  If no arguments are given, all tests are run.

           Limit testing creates the sequence of DAT objects needed to
           move data back and forth, attempting to find the limits supported
           for the DAPL object requested.  For example, if the LMR creation
           limit is being examined, the test will create a set of
           {IA, PZ, CNO, EVD, EP} before trying to run dat_lmr_create() to
           failure using that set of DAPL objects.  The ’width’ parameter
           can be used to control how many of these parallel DAPL object
           sets are created before beating upon the requested constructor.
           Use of -m limits the number of dat_*_create() calls that will
           be attempted, which can be helpful if the DAPL in use supports
           essentailly unlimited numbers of some objects.
           The limit test arguments are:

           [ -m maximum ]      Specify the maximum number of dapl_*_create()
                               attempts.
                               Default: run to object creation failure

           [ -w width ]        Specify the number of DAPL object sets to
                               create while initializing.
                               Default: 1

           [ limit_ia ]        Attempt to exhaust dat_ia_open()

           [ limit_pz ]        Attempt to exhaust dat_pz_create()

           [ limit_cno ]       Attempt to exhaust dat_cno_create()

           [ limit_evd ]       Attempt to exhaust dat_evd_create()

           [ limit_ep ]        Attempt to exhaust dat_ep_create()

           [ limit_rsp ]       Attempt to exhaust dat_rsp_create()

           [ limit_psp ]       Attempt to exhaust dat_psp_create()

           [ limit_lmr ]       Attempt to exhaust dat_lmr_create(4KB)

           [ limit_rpost ]     Attempt to exhaust dat_ep_post_recv(4KB)

           [ limit_size_lmr ]  Probe maximum size dat_lmr_create()

                               Default: run all tests

EXAMPLES
       dapltest -T S -d -D OpenIB-cma

                               Starts a server process with debug verbosity.

       dapltest -T T -d -s host1-ib0 -D OpenIB-cma -i 100 client SR 4096 2 server SR 4096 2

                               Runs a transaction test, with both sides
                               sending one buffer with two 4KB segments,
                              one hundred times.

       dapltest -T P -d -s host1-ib0 -D OpenIB-cma -i 100 SR 4096 2

                               Runs a performance test, with the client
                               sending one buffer with two 4KB segments,
                               one hundred times.

       dapltest -T Q -s host1-ib0 -D OpenIB-cma

                               Asks the server to clean up and exit.

       dapltest -T L -D OpenIB-cma -d -w 16 -m 1000

                               Runs all of the limit tests, setting up
                               16 complete sets of DAPL objects, and
                               creating at most a thousand instances
                               when trying to exhaust resources.

       dapltest -T T -V -d -t 2 -w 4 -i 55555 -s linux3 -D OpenIB-cma client RW 4096 1 server RW  2048  4
       client SR 1024 4 server SR 4096 2 client SR 1024 3 -f server SR 2048 1 -f

                               Runs a more complicated transaction test,
                               with two thread using four EPs each,
                               sending a more complicated buffer pattern
                               for a larger number of iterations,
                               validating the data received.

       BUGS   (and  To Do List)

           Use of CNOs (-Q) is not yet supported.

           Further limit tests could be added.