File: udf_plugin_development.md

package info (click to toggle)
netcdf 1%3A4.10.0-1
  • links: PTS, VCS
  • area: main
  • in suites: forky, sid
  • size: 116,236 kB
  • sloc: ansic: 281,201; sh: 14,777; cpp: 6,000; yacc: 2,612; makefile: 2,025; lex: 1,218; javascript: 280; xml: 173; awk: 2
file content (351 lines) | stat: -rw-r--r-- 8,100 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
UDF Plugin Development Guide {#udf_plugin_development}
====================================

# Introduction {#udf_dev_intro}

This guide explains how to develop plugins for netCDF-C user-defined formats (UDFs). It covers dispatch table implementation, plugin structure, building, testing, and deployment.

# Plugin Architecture {#udf_dev_arch}

A UDF plugin consists of:
1. **Dispatch table**: Structure with function pointers implementing the netCDF API
2. **Initialization function**: Called during plugin loading to register the dispatch table
3. **Format-specific code**: Your implementation of file I/O and data operations

## Plugin Lifecycle

1. Library initialization (`nc_initialize()`)
2. RC file parsing (if configured)
3. Plugin library loading (`dlopen`/`LoadLibrary`)
4. Init function location (`dlsym`/`GetProcAddress`)
5. Init function execution
6. Dispatch table registration via `nc_def_user_format()`
7. Plugin remains loaded for process lifetime

# Dispatch Table Structure {#udf_dev_dispatch}

The `NC_Dispatch` structure is defined in `include/netcdf_dispatch.h`. It contains function pointers for all netCDF operations.

## Required Fields

```c
typedef struct NC_Dispatch {
    int model;              /* NC_FORMATX_UDF0 through NC_FORMATX_UDF9 */
    int dispatch_version;   /* Must be NC_DISPATCH_VERSION */
    
    /* Function pointers for all netCDF operations */
    int (*create)(...);
    int (*open)(...);
    /* ... many more functions ... */
} NC_Dispatch;
```

## Minimal Dispatch Table Example

```c
#include "netcdf_dispatch.h"

static NC_Dispatch my_dispatcher = {
    NC_FORMATX_UDF0,        /* Use UDF slot 0 */
    NC_DISPATCH_VERSION,    /* Current ABI version */
    
    NC_RO_create,           /* Read-only: use predefined function */
    my_open,                /* Custom open function */
    NC_RO_redef,
    NC_RO__enddef,
    NC_RO_sync,
    my_abort,
    my_close,
    NC_RO_set_fill,
    my_inq_format,
    my_inq_format_extended,
    
    /* Inquiry functions - can use NC4_* defaults */
    NC4_inq,
    NC4_inq_type,
    /* ... continue for all required functions ... */
};
```

## Pre-defined Functions

Use these for operations your format doesn't support:

- **Read-only**: `NC_RO_*` - Returns NC_EPERM
- **Not NetCDF-4**: `NC_NOTNC4_*` - Returns NC_ENOTNC4
- **Not NetCDF-3**: `NC_NOTNC3_*` - Returns NC_ENOTNC3
- **No-op**: `NC_NOOP_*` - Returns NC_NOERR
- **Default implementations**: `NCDEFAULT_*` - Generic implementations
- **NetCDF-4 inquiry**: `NC4_*` - Use internal metadata model

# Initialization Function {#udf_dev_init}

The initialization function is called when your plugin is loaded.

## Function Signature

```c
int plugin_init(void);
```

## Requirements

1. Must be exported (not static)
2. Must call `nc_def_user_format()` to register dispatch table
3. Should return NC_NOERR on success, error code on failure
4. Name must match RC file INIT key

## Example Implementation

```c
#include <netcdf.h>

/* Your dispatch table */
extern NC_Dispatch my_dispatcher;

/* Initialization function - must be exported */
int my_plugin_init(void)
{
    int ret;
    
    /* Register dispatch table with magic number */
    ret = nc_def_user_format(NC_UDF0 | NC_NETCDF4, 
                             &my_dispatcher,
                             "MYFMT");
    if (ret != NC_NOERR)
        return ret;
    
    /* Additional initialization if needed */
    /* ... */
    
    return NC_NOERR;
}
```

# Implementing Dispatch Functions {#udf_dev_functions}

## Open Function

```c
int my_open(const char *path, int mode, int basepe, size_t *chunksizehintp,
            void *parameters, const NC_Dispatch *dispatch, int ncid)
{
    /* Open your file format */
    /* Populate internal metadata structures */
    /* Return NC_NOERR on success */
}
```

## Close Function

```c
int my_close(int ncid, void *v)
{
    /* Clean up resources */
    /* Close file handles */
    return NC_NOERR;
}
```

## Format Inquiry Functions

```c
int my_inq_format(int ncid, int *formatp)
{
    if (formatp)
        *formatp = NC_FORMAT_NETCDF4;  /* Or appropriate format */
    return NC_NOERR;
}

int my_inq_format_extended(int ncid, int *formatp, int *modep)
{
    if (formatp)
        *formatp = NC_FORMAT_NETCDF4;
    if (modep)
        *modep = NC_NETCDF4;
    return NC_NOERR;
}
```

# Building Plugins {#udf_dev_build}

## Unix/Linux/macOS

```makefile
# Makefile for UDF plugin
CC = gcc
CFLAGS = -fPIC -I/usr/local/include
LDFLAGS = -shared -L/usr/local/lib -lnetcdf

myplugin.so: myplugin.c
	$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $<

install:
	cp myplugin.so /usr/local/lib/
```

## Windows

```batch
REM Build UDF plugin on Windows
cl /LD /I"C:\netcdf\include" myplugin.c /link /LIBPATH:"C:\netcdf\lib" netcdf.lib
```

## CMake Example

```cmake
cmake_minimum_required(VERSION 3.10)
project(MyPlugin)

find_package(netCDF REQUIRED)

add_library(myplugin SHARED myplugin.c)
target_link_libraries(myplugin netCDF::netcdf)
target_include_directories(myplugin PRIVATE ${netCDF_INCLUDE_DIRS})

install(TARGETS myplugin LIBRARY DESTINATION lib)
```

# Testing Your Plugin {#udf_dev_test}

## Unit Testing

```c
/* test_plugin.c */
#include <netcdf.h>
#include <assert.h>

extern NC_Dispatch my_dispatcher;
extern int my_plugin_init(void);

int main() {
    int ret;
    NC_Dispatch *disp;
    
    /* Test initialization */
    ret = my_plugin_init();
    assert(ret == NC_NOERR);
    
    /* Verify registration */
    ret = nc_inq_user_format(NC_UDF0, &disp, NULL);
    assert(ret == NC_NOERR);
    assert(disp == &my_dispatcher);
    
    printf("Plugin tests passed\n");
    return 0;
}
```

## Integration Testing

```c
/* test_integration.c */
#include <netcdf.h>

int main() {
    int ncid, ret;
    
    /* Initialize and register plugin */
    my_plugin_init();
    
    /* Test file operations */
    ret = nc_open("testfile.dat", NC_UDF0, &ncid);
    if (ret != NC_NOERR) {
        fprintf(stderr, "Open failed: %s\n", nc_strerror(ret));
        return 1;
    }
    
    /* Test operations */
    int format;
    nc_inq_format(ncid, &format);
    
    nc_close(ncid);
    printf("Integration test passed\n");
    return 0;
}
```

## RC File Testing

Create `.ncrc`:
```ini
NETCDF.UDF0.LIBRARY=/path/to/myplugin.so
NETCDF.UDF0.INIT=my_plugin_init
NETCDF.UDF0.MAGIC=MYFMT
```

Test automatic loading:
```c
int main() {
    /* Plugin loads automatically during nc_initialize() */
    int ncid;
    nc_open("file_with_magic.dat", 0, &ncid);  /* Auto-detects format */
    nc_close(ncid);
    return 0;
}
```

# Debugging {#udf_dev_debug}

## Enable NetCDF Logging

```bash
export NC_LOG_LEVEL=3
./test_program
```

## Check Symbol Exports

```bash
# Unix
nm -D libmyplugin.so | grep init

# Windows
dumpbin /EXPORTS myplugin.dll
```

## GDB Debugging

```bash
gdb ./test_program
(gdb) break my_plugin_init
(gdb) run
(gdb) backtrace
```

## Common Issues

### Plugin not loaded
- Check RC file syntax
- Verify both LIBRARY and INIT are present
- Use absolute path for LIBRARY

### Init function not found
- Ensure function is not static
- Check function name matches INIT key
- Verify symbol is exported

### ABI version mismatch
- Recompile against current netCDF-C headers
- Check NC_DISPATCH_VERSION value

# Best Practices {#udf_dev_best}

1. **Error Handling**: Return appropriate NC_E* error codes
2. **Memory Management**: Clean up in close/abort functions
3. **Thread Safety**: Use thread-safe operations if needed
4. **Logging**: Use nclog functions for diagnostic output
5. **Documentation**: Document your format and API
6. **Testing**: Test all code paths thoroughly
7. **Versioning**: Version your plugin and document compatibility

# Example Plugin {#udf_dev_example}

See `examples/C/udf_plugin_example/` for a complete working plugin implementation.

# Reference {#udf_dev_ref}

- Dispatch table definition: `include/netcdf_dispatch.h`
- Pre-defined functions: `libdispatch/dreadonly.c`, `libdispatch/dnotnc*.c`
- Example implementations: `libhdf5/hdf5dispatch.c`, `libsrc/nc3dispatch.c`
- Test plugins: `nc_test4/test_plugin_lib.c`