1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
|
# bpftrace development guide
This document features basic guidelines and recommendations on how to do
bpftrace development. Please read it carefully before submitting pull requests
to simplify reviewing and to speed up the merge process.
## Building
The project supports the following recommended build workflows. Please choose
the one that works the best for you.
### Nix build
Nix is the most convenient way to build and test bpftrace. Nix will manage
all of bpftrace's build and runtime dependencies. It also has the advantage
of being used by the CI, so you are more likely to shake out errors before
submitting your change and seeing the CI fail.
The Nix build is documented in [nix.md](./nix.md).
### Distro build
The "distro build" is the more traditional way to build bpftrace. It relies on
you installing all of bpftrace's build and runtime dependencies on your host
and then calling into `cmake`.
Please be aware that bpftrace has strict dependencies on new versions of
`libbpf` and `bcc`. They are two of bpftrace's most important dependencies and
we plan on tracking their upstream quite closely over time.
As a result, while the distro build should work well on distros with newer
packages, developers on distros that lag more behind (for example Debian) may
want to consider using the Nix build. Or manually building and installing
`bcc` and `libbpf`.
The distro build is documented in [INSTALL.md](../INSTALL.md#generic-build-process).
## [Tests](../tests/README.md)
Every contribution should (1) not break the existing tests and (2) introduce new
tests if relevant. See existing tests for inspiration on how to write new ones. [Read more on the different kinds and how to run them](../tests/README.md).
## Continuous integration
CI executes the above tests in a matrix of different LLVM versions on NixOS.
The jobs are defined in `.github/workflows/ci.yml`.
### Running the CI
CI is automatically run on all branches and pull requests on the main repo. We
recommend to enable the CI (GitHub Actions) on your own fork, too, which will
allow you to run the CI against your testing branches.
### Debugging CI failures
It may often happen that tests pass on your local setup but fail in one of the
CI environments. In such a case, it is useful to reproduce the environment to
debug the issue.
To reproduce the NixOS jobs (from `.github/workflows/ci.yml`):
1. Acquire the job environment from the GHA UI: 
1. Run `.github/include/ci.py` with the relevant environment variables set
Example `ci.py` invocations:
```
$ NIX_TARGET=.#bpftrace-llvm10 ./.github/include/ci.py
```
```
$ NIX_TARGET=.#bpftrace-llvm11 \
CMAKE_BUILD_TYPE=Release \
RUNTIME_TEST_DISABLE="probe.kprobe_offset_fail_size,usdt.usdt probes - file based semaphore activation multi process" \
./.github/include/ci.py
```
### Known issues
Some tests are known to be flaky and sometimes fail in the CI environment. The
list of known such tests:
- runtime test `usdt.usdt probes - file based semaphore activation multi
process` ([#2410](https://github.com/bpftrace/bpftrace/issues/2402))
What usually helps, is restarting the CI. This is simple on your own fork but
requires one of the maintainers for pull requests.
### Virtual machine tests (vmtests)
In CI we run a subset of runtime tests under a controlled kernel by taking
advantage of nested virtualization on CI runners. For these tests, we use
[vmtest](https://github.com/danobi/vmtest) to manage the virtual machine.
The instructions in the above "Debugging CI failures" section also work
for the vmtest-ed runtime tests. But if you want to manually try something
quick and dirty in a CI kernel, you can do something like the following:
```bash
$ nix develop
(nix:nix-shell-env) $ vmtest -k $(nix build --print-out-paths .#kernel-6_12)/bzImage -- ./build/src/bpftrace -V
=> bzImage
===> Booting
===> Setting up VM
===> Running command
bpftrace v0.21.0-344-g3acb
```
While we'll defer to `vmtest` documentation for full details, one neat fact
worth pointing out is that `vmtest` will map the current running userspace into
the VM. This means you can run binaries built on your host from inside the
guest, eg. your development build of bpftrace.
## Coding guidelines
This is not about the formatting of the source code (we have `clang-format`
for that). Rather, it's about the semantics of the code and what language
features we try to use / avoid.
Please see [coding_guidelines.md](./coding_guidelines.md) for a full treatment
on the topic.
## Code style
We use clang-format with our custom config for formatting code. This was
[introduced](https://github.com/bpftrace/bpftrace/pull/639) after a lot of code
was already written. Instead of formatting the whole code base at once and
breaking `git blame` we're taking an incremental approach, each new/modified bit
of code needs to be formatted.
The CI checks this too, if the changes don't adhere to our style the job will fail.
### Using clang-format
[git clang-format](https://github.com/llvm/llvm-project/blob/main/clang/tools/clang-format/git-clang-format)
can be used to easily format commits, e.g. `git clang-format upstream/master`
### Avoid 'fix formatting' commits
We want to avoid `fix formatting` commits. Instead every commit should be
formatted correctly.
### Comment style
Strongly prefer C++-style comments for single line and block comments. C-style
comments are still useable for nested comments within a single line, e.g. to
leave an annotation on a specific argument or parameter. In the future, there
may be considerations for automated documentation based on comments, but this
is not currently done.
`bpftrace` itself supports both C-style and C++-style comment blocks. There is
currently no decision on recommended comment style, and both are used freely.
## Merging pull requests
Please squash + rebase all pull requests (with no merge commit). In other words,
there should be one commit in master per pull request. This makes generating
changelogs both trivial and precise with the least amount of noise.
The exception to this is PRs with complicated changes. If this is the case and
the commits are well structured, a rebase + merge (no merge commit) is acceptable.
The rule of thumb is the commit titles should make sense in a changelog.
## Changelog
The changelog is for end users. It should provide them with a quick summary of
all changes important to them. Internal changes like refactoring or test changes
do not belong to it.
### Maintaining the changelog
To avoid having write a changelog when we do a release (which leads to useless
changelog or a lot of work) we write them as we go. That means that every PR
that has a user impacting change must also include a changelog entry.
As we include the PR number in the changelog format this can only be done after
the PR has been opened.
If it is a single commit PR we include the changelog in that commit, when the PR
consists of multiple commits it is OK to add a separate commit for the changelog.
## bpftrace internals
For more details on bpftrace internals, see
[internals_development.md](internals_development.md).
|
