1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
# Allocation Counting Test
This briefly describes how the allocation counting test works.
## How does it work?
This is possibly the simplest implementation that counts memory allocations (`malloc` and friends) and frees (mostly `free`). It just maintains two atomic variables which count the number of mallocs and the number of frees respectively. We run a simple HTTP1 example -- 1000 requests and responses generated by a simple SwiftNIO based client and server -- and then evaluate the number of mallocs and frees. The difference `mallocs - frees` should be pretty much 0 and the number of `mallocs` should remain stable (or decrease) across commits. We can't establish a perfect baseline as the exact number of allocations depends on your operating system, libc and Swift version.
### How are the functions hooked?
Usually in UNIX it's enough to just define a function, for example
```C
void free(void *ptr) { ... }
```
in the main binary and all modules will use this `free` function instead of the real one from the `libc`. For Linux, this is exactly what we're doing, the `bootstrap` binary defines such a `free` function in its `main.c`. On Darwin (macOS/iOS/...) however that is not the case and you need to use [dyld's interpose feature](https://books.google.co.uk/books?id=K8vUkpOXhN4C&lpg=PA73&ots=OMjhRWWwUu&dq=dyld%20interpose&pg=PA73#v=onepage&q=dyld%20interpose&f=false). The odd thing is that dyld's interposing _only_ works if it's in a `.dylib` and not from a binary's main executable. Therefore we need to build a slightly strange SwiftPM package:
- `bootstrap`: The main executable's main module (written in C) so we can hook the `free` function on Linux.
- `BootstrapSwift`: A SwiftPM module (written in Swift) called in from `bootstrap` which implements the actual SwiftNIO benchmark (and therefore depends on the `NIO` module).
- `HookedFunctions`: A separate SwiftPM package that builds a shared library (`.so` on Linux, `.dylib` on Darwin) which contains the `replacement_malloc`, `replacement_free`, etc functions which just increment an atomic integers representing the number of operations. On Darwin, we use `DYLD_INTERPOSE` in this module, interposing libc functions with our `replacement_` functions. This needs to be a separate SwiftPM package as otherwise its code would just live inside of the `bootstrap` executable and the dyld interposing feature wouldn't work.
- `AtomicCounter`: SwiftPM package (written in C) that implements the atomic counters. It needs to be a separate package as both `BoostrapSwift` (to read the allocation counter) as well as `HookedFree` (to increment the allocation counter) depend on it.
## What benchmark is run?
We run a single TCP connection over which 1000 HTTP requests are made by a client written in NIO, responded to by a server also written in NIO. We re-run the benchmark 10 times and return the lowest number of allocations that has been made.
## Why do I have to set a baseline?
By default this test should always succeed as it doesn't actually compare the number of allocations to a certain number. The reason is that this number varies ever so slightly between operating systems and Swift versions. At the time of writing on macOS we got roughly 326k allocations and on Linux 322k allocations for 1000 HTTP requests & responses. To set a baseline simply run
```bash
export MAX_ALLOCS_ALLOWED_1000_reqs_1_conn=327000
```
or similar to set the maximum number of allocations allowed. If the benchmark exceeds these allocations the test will fail.
|
