1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
|
# Using EZTrace
## Viewing available modules
EZTrace uses modules that are in charge of instrumenting
libraries. You can use `eztrace_avail` to list the available modules:
```
$ eztrace_avail
3 stdio Module for stdio functions (read, write, select, poll, etc.)
2 pthread Module for PThread synchronization functions (mutex, semaphore,
spinlock, etc.)
6 papi Module for PAPI Performance counters
1 omp Module for OpenMP parallel regions
4 mpi Module for MPI functions
5 memory Module for memory functions (malloc, free, etc.)
```
The list of modules to compile is selected at CMake time. If some
modules are missing, check your cmake configuration.
While EZTrace ships plugins for the main parallel programming
libraries (eg. MPI, OpenMP, CUDA, ...), you can define new plugins for your own application/library. See the [Plugin page](plugin.md) for instructions.
## Running an application with EZTrace
`eztrace` runs an application and generates an execution trace. It
needs a list of modules to load that can be specified with `-t`:
```
$ eztrace -t "posixio memory" ./my_program my_arg1 my_arg2
[P0T0] Starting EZTrace (pid: 260513)...
[...]
[P0T0] Stopping EZTrace (pid:260513)...
```
This generates an execution trace `<my_program>_trace` in the current directory.
## Running an MPI application with EZTrace
When tracing an MPI application, you need to add the `-m` flag to EZTrace:
```
$ mpirun -np 4 eztrace -m -t "mpi posixio" ./my_app my_arg1 my_arg2
```
## Other options
`eztrace` usage:
```
Usage: eztrace [OPTION] program [arg1 arg2 ...]
-t "plugin1 plugin2 ... pluginN" Select a list of plugins
-o <directory> Select the output directory
-l <directory> Select a plugin directory
-f Enable EZTRACE_FLUSH
-d Debug mode
-? -h Display this help and exit
```
## Tracing a program functions
By default, EZTrace trace functions located in shared libraries. If
you want to trace the main application functions, you can ask the
compiler to instrument functions, and EZTrace will trace the
instrumented functions.
To instrument a program, add `-finstrument-functions` to the
compilation flags, and `-rdynamic` to the link flags:
```
$ gcc -c foo.c -o foo.o -finstrument-functions
$ gcc -o foo foo.o -rdynamic
```
Then, use EZTrace `compiler_instrumentation` module:
```
$ eztrace -t compiler_instrumentation ./foo
[...]
```
By default, the compiler instruments all the functions. To control which functions should be traced, you can:
- specify to the compiler the functions to exclude (eg with `-finstrument-functions-exclude-function-list=sym,sym,...`)
- specify to EZTrace the functions to exclude with (eg set `EZTRACE_EXCLUDE_LIST="sym1 sym2 ..."`)
## Environment variables
Here is a list of the environment variables that you can use for tuning EZTrace.
### General-purpose variables
- `EZTRACE_TRACE_DIR`: specifies the directory in which the traces are created
(default: ${program_name}_trace). You can also use the `-o` option in eztrace.
- `EZTRACE_LIBRARY_PATH`: specifies the directories in which eztrace can find
eztrace modules (default: none) You can also use the `-l` option in eztrace.
- `EZTRACE_TRACE`: specifies the list of eztrace modules to load (default: the
list of available modules) You can also use the `-t` option in eztrace.
- `EZTRACE_BUFFER_SIZE`: specifies the size (in byte) of the buffer in which
eztrace stores events (default: 16MB)
- `EZTRACE_SIGALARM`: ask EZTrace to stop the application every x ms in order to
collect information (such as hardware counters). (default: 0)
### Error-handling variables
- `EZTRACE_SIGNAL_HANDLER`: enables eztrace signal handling (default: disabled)
- `EZTRACE_DEBUGGER`: when an error occurs, eztrace waits so that you can attach gdb to the process to investigate (this can be useful when debugging MPI programs)
### Hardware counters-related variables
- `EZTRACE_PAPI_COUNTERS`: selects hardware events to trace using the PAPI
library, e.g. `export EZTRACE_PAPI_COUNTERS="PAPI_L3_TCM PAPI_FP_INS"`. Please
note that the list of supported events as well as the number of events,
which can be traced simultaneously, vary depending on the processor type.
This information can be retrieved using 'papi_avail' and the processor
documentation.
- `EZTRACE_PAPI_SAMPLE_INTERVAL`: select the minimum interval (in microseconds)
between each check of the PAPI counters (default: 100)
### CUDA-related variables
- `EZTRACE_CUDA_CUPTI_DISABLED`: disable the use of CUPTI in EZTrace. This
option disables the recording of events that happen on the GPU.
### IOTracer-related variables
- `IOTRACER_TRACED_FILTER`: should IOTracer trace a specific file (`file`) or directory (`dir`) ?
- `IOTRACER_TRACED_FILE`: TODO
- `IOTRACER_TRACED_INODE`: inode of the file/directory to watch
### Compiler_instrumentation variables
- `EZTRACE_EXCLUDE_LIST`: specify to the functions to exclude (eg `export EZTRACE_EXCLUDE_LIST="foo bar baz"`)
## Tutorials
[Several tutorials are available here](https://gitlab.com/eztrace/eztrace-tutorials).
|
