1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
|
See http://fscked.org/writings/SHM/shm-4.html.
In addition to the tactics mentioned in that paper, NJAMD implements some
other optimizations and tricks to minimize memory overhead and maximize speed
and functionality.
First off, there are 3 versions of the library, overflow, underflow, and
strict underflow. Each follows the set of options described in njamd(3) which
are passed through the environment.
The library itself is LD_PRELOAD'ed (it can also be linked, on systems that
don't support preloading). At the first malloc, a primary initialization is
done, in which some basic variables are set. Allocation is optimized by
mmapping a large section of anonymous memory (default is 20 megs) called a
slab, and then allocating sections off this slab. Note: we don't use sbrk()
for two reasons: 1. Just in case standard malloc gets called somehow, and 2.
because it's easier to let the kernel manage the address space and just as
fast, since we jump to kernel space anyways. Because of this linear mechanism,
we are able to set two markers that indicate what region of memory was
allocated before the environment became available. Care must be taken to free
this memory with the default free mechanism, and to make sure the memory is
mapped to to no access, so it is never used again. Also, we must be careful to
use the processors default alignment when freeing this memory, as opposed to
environment supplied alignment.
After the user's code has entered main(), a secondary initializer is called
that reads the environment and sets up the various options, including which
allocator to use. The way entry to main is detected is by declaring a function
with attribute constructor. When it executes, it sets a variable stating that
main has been entered. On the next malloc, we can run the second stage
initializer. The one problem with this approach is that C++ also uses
attribute constructor for executing constructors of globally declared objects.
If a globally declared object is constructed between our constructor and main,
and it calls malloc, a segmentation fault may result. We have done some
testing, and it seems that under Linux when the library is LD_PRELOADed, our
constructor is the very last one called, immediately before main.
Once we know that the code has entered main, we can extrapolate the return
address in the user's code of a malloc call. This is done by the fact that in
ELF, shared libraries start high above main, the heap grows up from main. The
top of the heap (which we do not modify ourselves), can be found by a
call of sbrk(0). We can then search the call stack for the first return
address lower than the initial value of sbrk(0), which was stored during the
first initializer. This allows us to determine where in the /user's code/ the
allocation actually took place. This means that we only have to wrap malloc
and calloc and friends, and any allocator that calls them can be traced to
user code. For example, we can tell the user he leaked memory in strdup at
such and such a line in his code. In njamd >= v0.7, muliple call stacks are
recorded, up to the value specified in TRACE_DEPTH in ./include/lib/njamd.h.
The four versions of the library (overflow, underflow, strict, and none)
differ in their placement of the page of protected memory. Overflow places the
protected page after the user's buffer. When this is done, the user's buffer
must be offset a certain number of bytes to end at this boundary. Underflow
places the buffer length before the user's buffer, and then a protected page
before that. This is because we must know the length of the buffer to do
anything with it, and this minor leeway is a tradeoff for not faulting that
protected page. The strict underflow version (mem_sunder.c) does not make
this tradeoff, and faults the protected page and stores the length in it,
trading off roughly a 2X increase in memory and 25% decrease in speed for the
ability to catch underflows of only one byte. With the "none" option, standard
libc malloc is wrapped via dlopen(2), and memory accounting is done. This is
provided for people who simply want to check memory leaks only. No protected
pages are placed near the memory, however sentinal values are place there, so
the user can be alerted of corrupted memory by the next call to free. As a
result, this option is much faster and lighter than the rest. Of course, the
NJAMD_CHK_FREE options do not apply when this option is set, and have the
effective value of none.
Alignment is also covered by this system. It is detected via an autoconf test,
but can be tuned by an env variable. Alignment of n bytes means that the
overflow and fast underflow versions will not detect overwrites of n bytes or
less until the next free. Consistency checking is performed on free to
determine the integrity of non-protected regions before AND after the buffer
(due to alignment, or the version of the library). So at the worst, you always
know of a memory error by the time you free the memory.
We do memory leak accounting by storing the info pointer before or after the
user's buffer, depending on the version of the library. This pointer indexes
into a heap table created in the intialization phase. The heap table is
actually a memory mapped file that can be set to persist after the program
exits. When the address space is being recyled (ie NO_CHK_FREE is set), a
stack is maintained of free heap table entries. Leaked memory is detected by
scanning this heap table for entires that are non-zero. A utlity has been
written that allows you to scan this saved heap (ie the array of structs) for
a specific address, namely an address access that caused a segmentation fault,
giving you the location where this address was allocated, and possibly where
it was freed. A function is also provided (__nj_ptr_info(void *)) that can be
called from gdb to lookup an arbitrary address.
The front end interacts with the library by specifying a port for the library
to connect to via an environment variable. When the library is running under
the front end, all messages are written to this socket instead of stderr. The
front end also redirects input and output to gdb using pipes.
- Mike Perry <mikepery@fscked.org>
|
