1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
|
DTPools Release 0.0
DTPools is a datatype library used to test MPI communication routines with
different datatype combinations. DTPools' interface is used to create pools
of datatypes, each having a specified signature (i.e., native type + count).
Every pool supports different datatype layouts (defined internally by the
library). For a list of the available layouts, go to section: "4. Supported
Derived Datatype layouts".
This README is organized as follows:
1. DTPools API
2. Testing with DTPools
3. Supported Derived Datatypes
4. Supported Derived Datatype layouts
5. Extending DTPools
6. TODOs
----------------------------------------------------------------------------
1. DTPools API
==============
Follows a list of DTPools interfaces used for datatype testing:
* int DTP_pool_create(MPI_Datatype basic_type, int basic_count, DTP_t *dtp)
Create a new basic pool with defined datatype signature.
- basic_type: native datatype part of signature
- basic_count: native datatype count part of signature
- dtp: datatype pool object
* int DTP_pool_create_struct(int basic_type_count, MPI_Datatype *basic_types, int *basic_counts, DTP_t *dtp)
Create a new struct pool with defined signature.
- basic_type_count: number of native datatypes in struct
- basic_type: array of native datatypes
- basic_counts: array of native datatype counts
- dtp: datatype pool object
* int DTP_pool_free(DTP_t dtp)
Free a previously created datatype pool.
- dtp: datatype pool object
* int DTP_obj_create(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count)
Create a datatype object (at index obj_idx) inside the specified pool. Also initialize
the buffer elements using start, stride and count.
- dtp: datatype pool object
- obj_idx: number of datatype inside the pool to be created
- val_start: start of initialization value for buffer at index obj_idx
- val_stride: increment for next element in buffer
- val_count: total number of elements to be initialized in buffer
* int DTP_obj_free(DTP_t dtp, int obj_idx)
Free a previously created datatype object inside the specified pool.
- dtp: datatype pool object
- obj_idx: number of datatype inside the pool to be freed
* int DTP_obj_buf_check(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count)
Checks whether the received buffer (used in communication routine) matches the sent buffer.
- dtp: datatype pool object
- obj_idx: number of datatype inside the pool to be checked
- val_start: start of checking value for buffer at index obj_idx
- val_stride: increment for next checked element in buffer
- val_count: total number of elements to be checked in buffer
----------------------------------------------------------------------------
2. Testing with DTPools
=======================
Follows a simple test application that uses DTPools:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include "mpi.h"
#include "dtpools.h"
#define BASIC_TYPE_MAX_COUNT (1024)
int main(int argc, char *argv[]) {
int i, j, err;
int num_err = 0;
int basic_type_count;
int myrank = 0;
MPI_Request req;
DTP_t send_dtp, recv_dtp;
MPI_Init(NULL, NULL);
basic_type_count = BASIC_TYPE_MAX_COUNT;
err = DTP_pool_create(MPI_INT, basic_type_count, &send_dtp);
if (err != DTP_SUCCESS) {
/* error hanling */;
}
err = DTP_pool_create(MPI_INT, basic_type_count * 2, &recv_dtp);
if (err != DTP_SUCCESS) {
/* error hanling */;
}
for (i = 0; i < send_dtp->DTP_num_objs; i++) {
err = DTP_obj_create(send_dtp, i, 0, 2, basic_type_count);
if (err != DTP_SUCCESS) {
/* error handling */;
}
for (j = 0; j < recv_dtp->DTP_num_objs; j++) {
err = DTP_obj_create(recv_dtp, j, 0, 0, 0);
if (err != DTP_SUCCESS) {
/* error handling */;
}
MPI_Irecv(recv_dtp->DTP_obj_array[j].DTP_obj_buf,
recv_dtp->DTP_obj_array[j].DTP_obj_count,
recv_dtp->DTP_obj_array[j].DTP_obj_type,
myrank, 0, MPI_COMM_WORLD, &req);
MPI_Send(send_dtp->DTP_obj_array[i].DTP_obj_buf,
send_dtp->DTP_obj_array[i].DTP_obj_count,
send_dtp->DTP_obj_array[i].DTP_obj_type,
myrank, 0, MPI_COMM_WORLD);
MPI_Wait(&req, MPI_STATUS_IGNORE);
if (DTP_obj_buf_check(recv_dtp, j, 0, 2, basic_type_count) != DTP_SUCCESS) {
num_err++;
}
DTP_obj_free(recv_dtp, j);
}
DTP_obj_free(send_dtp, i);
}
DTP_pool_free(send_dtp);
DTP_pool_free(recv_dtp);
if (num_err > 0) {
fprintf(stdout, " No Errors\n");
fflush(stdout);
}
MPI_Finalize();
return err;
}
----------------------------------------------------------------------------
3. Supported Derived Datatypes
==============================
Currently the following derived datatype are supported:
* MPI_Type_contiguous
* MPI_Type_vector
* MPI_Type_create_hvector
* MPI_Type_indexed
* MPI_Type_create_hindexed
* MPI_Type_create_indexed_block
* MPI_Type_create_hindexed_block
* MPI_Type_create_subarray
* MPI_Type_create_struct
The following native datatypes are also supported:
* MPI_CHAR,
* MPI_WCHAR,
* MPI_SHORT,
* MPI_INT,
* MPI_LONG,
* MPI_LONG_LONG_INT,
* MPI_UNSIGNED_CHAR,
* MPI_UNSIGNED_SHORT,
* MPI_UNSIGNED,
* MPI_UNSIGNED_LONG,
* MPI_UNSIGNED_LONG_LONG,
* MPI_FLOAT,
* MPI_DOUBLE,
* MPI_LONG_DOUBLE,
* MPI_INT8_T,
* MPI_INT16_T,
* MPI_INT32_T,
* MPI_INT64_T,
* MPI_UINT8_T,
* MPI_UINT16_T,
* MPI_UINT32_T,
* MPI_UINT64_T,
* MPI_C_COMPLEX,
* MPI_C_FLOAT_COMPLEX,
* MPI_C_DOUBLE_COMPLEX,
* MPI_C_LONG_DOUBLE_COMPLEX,
* MPI_FLOAT_INT,
* MPI_DOUBLE_INT,
* MPI_LONG_INT,
* MPI_2INT,
* MPI_SHORT_INT,
* MPI_LONG_DOUBLE_INT
----------------------------------------------------------------------------
4. Supported Derived Datatype layouts
=====================================
The following layouts for derived datatypes are currently supported:
* Simple layout
- DTPI_OBJ_LAYOUT_SIMPLE__CONTIG: type_count = 1; type_stride = -; type_blklen = basic_type_count;
- DTPI_OBJ_LAYOUT_SIMPLE__VECTOR: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
- DTPI_OBJ_LAYOUT_SIMPLE__HVECTOR: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
- DTPI_OBJ_LAYOUT_SIMPLE__INDEXED: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
- DTPI_OBJ_LAYOUT_SIMPLE__HINDEXED: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
- DTPI_OBJ_LAYOUT_SIMPLE__BLOCK_INDEXED: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
- DTPI_OBJ_LAYOUT_SIMPLE__BLOCK_HINDEXED: type_count = basic_type_count; type_stride = 2; type_blklen = 1;
* Complex layout
- DTPI_OBJ_LAYOUT_LARGE_BLK__VECTOR: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__HVECTOR: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__INDEXED: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__HINDEXED: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__BLOCK_INDEXED: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__BLOCK_HINDEXED: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__SUBARRAY_C: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK__SUBARRAY_F: type_count = small; type_blklen = large; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__VECTOR: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__HVECTOR: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__INDEXED: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__HINDEXED: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__BLOCK_INDEXED: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__BLOCK_HINDEXED: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__SUBARRAY_C: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_CNT__SUBARRAY_F: type_count = large; type_blklen = small; type_stride = type_blklen + 1;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__VECTOR: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__HVECTOR: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__INDEXED: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__HINDEXED: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__BLOCK_INDEXED: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__BLOCK_HINDEXED: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__SUBARRAY_C: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_BLK_STRD__SUBARRAY_F: type_count = small; type_blklen = large; type_stride = type_blklen * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__VECTOR: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__HVECTOR: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__INDEXED: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__HINDEXED: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__BLOCK_INDEXED: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__BLOCK_HINDEXED: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__SUBARRAY_C: type_count = large; type_blklen = small; type_stride = type_count * 2;
- DTPI_OBJ_LAYOUT_LARGE_CNT_STRD__SUBARRAY_F: type_count = large; type_blklen = small; type_stride = type_count * 2;
----------------------------------------------------------------------------
5. Extending DTPools
====================
Extending DTPools with new datatype layouts is as simple as adding the
type descriptor in `test/mpi/dtpools/include/dtpools_internal.h`, the
corresponding type create and buf check functions in
`test/mpi/dtpools/src/dtpools_internal.c`, and including the new layout to
the pool create function in `test/mpi/dtpools/src/dtpools.c`.
Additionally the type create function should also be added to creators
function vector `DTPI_Init_creators`.
Example:
/* dtpools_internal.h */
enum {
...,
DTPI_OBJ_LAYOUT_MYLAYOUT__NESTED_VECTOR,
...
};
int DTPI_Nested_vector_create(struct DTPI_Par *par, DTP_t dtp);
int DTPI_Nested_vector_check_buf(struct DTPI_Par *par, DTP_t dtp);
/* dtpools_internal.c */
void DTPI_Init_creators(DTPI_Creator * creators) {
...
creators[DTPI_OBJ_LAYOUT_MYLAYOUT__NESTED_VECTOR] = DTPI_Nested_vector_create;
}
int DTPI_Nested_vector_create(struct DTPI_Par *par, DTP_t dtp) {
...
}
int DTPI_Nested_vector_check_buf(struct DTPI_Par *par, DTP_t dtp) {
...
}
/* dtpools.c */
int DTP_obj_create(DTP_t dtp, int obj_idx, int val_start, int var_stride, int val_count) {
...
switch(obj_idx) {
case XXX:
...
break;
case DTPI_OBJ_LAYOUT_MYLAYOUT__NESTED_VECTOR:
/* set up parameters for create function */
par.core.type_count = X(count); /* signature count */
par.core.type_blklen = Y(count);
par.core.type_stride = Z(count);
break;
default:;
}
...
}
----------------------------------------------------------------------------
6. TODOs
========
Follows a list of known issues that should be fixed in a future release:
1. Resized datatypes (using MPI_TYPE_CREATE_RESIZED) are not currently supported.
2. The framework does not provide an interface to reset the type buffer:
`DTP_obj_reset(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count)`
3. The interface should return an object handle that can be used to directly
reference the created datatype, its count, and buffer instead of accessing
directly the object array:
`DTP_obj_create(DTP_t dtp, int obj_idx, int val_start, int val_stride, int val_count, DTP_Obj_t *obj)`
4. Currently datatypes in the pools have count of 1. The framework should be extended
to support counts > 1.
|
