1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
|
#### Clone the repo and compile the benchmark runner
```
git clone https://github.com/duckdb/duckdb
cd duckdb
BUILD_BENCHMARK=1 BUILD_TPCH=1 make
```
#### List all available benchmarks
`build/release/benchmark/benchmark_runner --list`
#### Run a single benchmark
`build/release/benchmark/benchmark_runner benchmark/micro/nulls/no_nulls_addition.benchmark`
The output will be printed to `stdout` in CSV format, in the following format:
```
name run timing
benchmark/micro/nulls/no_nulls_addition.benchmark 1 0.121234
benchmark/micro/nulls/no_nulls_addition.benchmark 2 0.121702
benchmark/micro/nulls/no_nulls_addition.benchmark 3 0.122948
benchmark/micro/nulls/no_nulls_addition.benchmark 4 0.122534
benchmark/micro/nulls/no_nulls_addition.benchmark 5 0.124102
```
You can also specify an output file using the `--out` flag. This will write only the timings (delimited by newlines) to that file.
```
build/release/benchmark/benchmark_runner benchmark/micro/nulls/no_nulls_addition.benchmark --out=timings.out
cat timings.out
0.182472
0.185027
0.184163
0.185281
0.182948
```
#### Regex
You can also use a regex to specify which benchmarks to run. Be careful of shell expansion of certain regex characters (e.g. `*` will likely be expanded by your shell, hence this requires proper quoting or escaping).
`build/release/benchmark/benchmark_runner "benchmark/micro/nulls/.*" `
#### Run all benchmarks
Not specifying any argument will run all benchmarks.
`build/release/benchmark/benchmark_runner`
#### Other options
`--info` gives you some other information about the benchmark.
```
build/release/benchmark/benchmark_runner benchmark/micro/nulls/no_nulls_addition.benchmark --info
display_name:NULL Addition (no nulls)
group:micro
subgroup:nulls
```
`--query` will print the query that is run by the benchmark.
```
SELECT MIN(i + 1) FROM integers
```
`--profile` will output a query tree (pretty printed), primarily intended for interactive use.
```
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││ Query Profiling Information ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
SELECT MIN(i + 1) FROM integers
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││ Total Time: 0.176s ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
┌───────────────────────────┐
│ UNGROUPED_AGGREGATE │
│ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │
│ min(#0) │
│ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │
│ 1 │
│ (0.03s) │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ PROJECTION │
│ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │
│ +(i, 1) │
│ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │
│ 100000000 │
│ (0.05s) │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ SEQ_SCAN │
│ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │
│ integers │
│ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │
│ i │
│ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │
│ 100000000 │
│ (0.08s) │
└───────────────────────────┘
```
|
