.TH LIBNJAMD 3  "NJAMD - 5 Aug 01"
.SH NAME
NJAMD - Not Just Another Malloc Debugger
.SH SYNOPSIS
.nf
.B #include <stdlib.h>
.sp
.BI "void *calloc(size_t " "nmemb" ", size_t " "size" ");"
.nl
.BI "void *malloc(size_t " "size" ");"
.nl
.BI "void free(void " "*ptr" ");"
.nl
.BI "void *realloc(void " "*ptr" ", size_t "  "size" ");"
.sp
.BI "new, new[], delete, delete[]"
.sp
.B #include <string.h>
.sp
.BI "char *strdup(const char *s);"
.sp
.BI "And so much more..."
.sp
.I "Environment Variables"
.sp
.B "export LD_PRELOAD=libnjamd.so"
.nl
.BI "export NJAMD_PROT=val"
.nl
.BI "export NJAMD_CHK_FREE=val"
.nl
.BI "export NJAMD_CALLSTACK_LIMIT=val (NEW!)"
.nl
.BI "export NJAMD_NO_FREE_INFO=1"
.nl
.BI "export NJAMD_ALIGN=num"
.nl
.BI "export NJAMD_DUMP_LEAKS_ON_EXIT=num"
.nl
.BI "export NJAMD_DUMP_STATS_ON_EXIT=1"
.nl
.BI "export NJAMD_DUMP_CORE=soft,hard"
.nl
.BI "export NJAMD_PERSISTENT_HEAP=1"
.nl
.BI "export NJAMD_TRACE_LIBS=1"
.nl
.BI "export NJAMD_NO_TRACE=1"
.nl
.BI "export NJAMD_ALLOW_READ=1"
.nl
.BI "export NJAMD_ALLOW_FREE_0=1"
.nl
.BI "export NJAMD_ALLOW_MALLOC_0=1"
.nl
.BI "export NJAMD_MUTABLE_ALLOC=1 (NEW!)"
.nl
.BI "kill -USR1 <pid>"
.sp
.I "Extra Functions (NEW!)"
.sp
.BI "void __nj_ptr_info(void *ptr);"
.nl
.BI "void __nj_dump_leaks(void);"
.nl
.BI "void __nj_change_prot_type(char *newtype);"
.nl
.BI "void __nj_change_free_type(char *newtype);"
.nl
.BI "void __nj_change_default_align(int newalign);"
.nl
.BI "void __nj_change_trace_libs(int val);"
.nl
.BI "void __nj_change_free_info(int val);"
.fi
.SH DESCRIPTION
This manpage describes the library subsystem of
.IR NJAMD "," 
which can be used standalone, with the front end njamdqt (BROKEN: Needs the output to be standardized),
from within 
.BR gdb "(1)"
or from within any other debugger. It also comes with a very useful utility
.BR njamdpm "(1) "
that allows you to do postmortem heap analysis. (BROKEN: Who wants to rewrite njamdpm for the new??)
.LP
.I NJAMD
is a full featured malloc debugger. That is, it protects against all common
dynamic memory bugs, including overflow, underflow, writes to freed memory, 
and memory leaks, all without recompiling or even relinking your executable.
In addition, it is able to trace memory leaks even through arbitrary library 
functions that wrap 
.BR malloc "(3), "
such as 
.BR strdup "(3), "
GUI widget allocators, and even C++ new and delete.
.LP
Normally, when a program does something illegal with its dynamic memory (such
as writing past the end of a buffer returned by 
.BR malloc "(3), " 
ie: an overflow), its
execution may not immediately terminate. Instead, bizzarre and unexpected
results can occur later on during program execution. This is due to the fact
that malloc implementations store book keeping information before and after
allocated segments. So overwriting these regions won't cause your program to
crash right away, but will cause chaos during subsequent malloc requests, or
even during usuage of memory returned from otherwise seemingly valid 
.B malloc()
.LP
.I NJAMD
changes all this. It provides immediate notification (through segmentation
fault) if you do anything illegal with your memory. Using your favorite
debugger, you can pinpoint the source of error to the line, and even to the
assembly instruction.
.SH USAGE
.SS "Overview"
With one exception, the behavior of the debugger is controlled entirely
through the following environment variables. In fact, the debugger needn't
even be linked to your program on systems that suport the LD_PRELOAD
environment variable. Unless otherwise stated, these environment variables are
tested .B for existance only.
That is, doing something like export NJAMD_ALLOW_READ=0 will 
.B still
allow read access past the ends of buffers.
.SS "Environment Variables"
.TP
.I LD_PRELOAD=library
This environment variable instructs the dynamic linker to override system
supplied functions with those in a specified library, namely ours.
To use
.I LIBNJAMD
to debug your programs, enter the equivalent to
.sp
.B export LD_PRELOAD=libnjamd.so
.sp
into your shell. All subsequent programs run from that shell will then use 
.IR LIBNJAMD "'s "
allocator routines instead of those in the standard libc.
.sp
Alternatively, to debug only one command, enter 
.sp
.B LD_PRELOAD=libnjamd.so <program>
.sp
.I NOTE
on Irix systems, the variable is 
.B _RLD_LIST
and the syntax is 
.nl
.BR _RLD_LIST=libnjamd.so:DEFAULT "."
Otherwise the behavior is the exact same.
.TP
.I NJAMD_PROT=val
To use
.I LIBNJAMD
to protect against overflows (accessing memory past the end of an allocated buffer), 
enter the equivalent to
.sp
.B export NJAMD_PROT=overflow
.sp
into your shell. This is the default mode of operation.
.sp
There are two ways to protect against underflows (accessing memory before an allocated
buffer), strict and weak. Weak is
considerably faster than strict, and uses half as much memory. However, weak
will only catch underflows greater than 4 or 8 bytes, depending on your
archetecture.
.sp
To protect against ALL underflows, enter
.sp
.B export NJAMD_PROT=strict
.sp
To protect against most larger underflows, enter
.sp
.B export NJAMD_PROT=underflow
.sp
For memory leak checking only, enter
.sp
.B export NJAMD_PROT=none
.sp
This option uses standard libc malloc, and is thus is much faster and lighter 
than the other options, for people who just want memory leak accounting. Note
that it is unavailable on platforms that don't support
.BR dlopen "(2). "
Also, this option will misreport leaked memory by one malloc on some platforms
(GNU/Linux w/ glibc 2.1), becuase malloc calls itself to set up some data
structures.
.sp
Do note that each version of the library performs consistancy checks so that
you know if the opposite error occured when you try to free that block. For
example, when you free a buffer, the overflow version checks to make sure that the data before your
buffer hasn't changed, and the underflow versions checks to make sure that the
data after your buffer hasn't changed. So at the worst, you always
know of a memory error by the time you free the memory. This even applies to
the "none" option.
.TP
.I NJAMD_CHK_FREE=val
There are various methods of handling accesses to freed memory, and each has
its advantages and disadvantages. If you select
.B NJAMD_PROT=none
this setting has no effect (NJAMD operates as if you selected no free
protection).
.sp
.B export NJAMD_CHK_FREE=segv
.sp
The default method is to protect freed memory. A double free will yield a
segmentation fault and no error message, and any access to a memory region
freed by free or realloc will cause a segmentation fault.
.sp
.B export NJAMD_CHK_FREE=error
.sp
This option both protects freed memory and provides you with some sort
of notification when you try to free a chunk twice. While this causes no physical memory loss, it does pollute the address
space a bit, and can bog down the operating system kernel with excessive mappings to keep track of. The BSD's especially are hurt by this option, and
Linux has a limit of 65536 mappings, which can be used up pretty quickly.
.sp
.B export NJAMD_CHK_FREE=none
.sp
This method provides no protection of freed memory. Writes to freed memory may
produce the same bizzare and unpredicatble results as when using a normal
malloc implementation. In addition, this option is 10 times faster than the
other error checking schemes due to a free-list cache put in NJAMD > 0.9.0 (In
fact, NJAMD_PROT=over with no free checking is FASTER than NJAMD_PROT=none for
the allocfree benchmark/test!)
.sp
Use of this option is recommended when allocation
intensive progams run out of memory under either of the preceding options, or
if NJAMD is prohibively slow.
Do note that
by default, some OS's do not allow you to map the entire address space. You 
must use 
.BR sysctl "(2) "
to allow this. In Linux, for example, you must issue 
.sp
.B sysctl -w vm.overcommit_memory=1
.sp
to use the entire address space. Try doing this instead of turning off the
checking of freed memory. Also, look into the kernel_mods patches that came
with the source distribution.
.sp
In addition, if you are trying to use this option because you are running out
of ADDRESS space, you might want to try the next option:
.I (NJAMD_NO_FREE_INFO)
.sp
.B export NJAMD_CHK_FREE=nofree
.sp
Setting this option will cause NJAMD to neglect to free ANY memory. 
I can't think of any circumstances where this would be useful, but it was very easy to
implement :)
.TP
.I NJAMD_CALLSTACK_LIMIT=val
In NJAMD > 0.9.2, callstacks became indefinately long. This can take up a lot 
of memory, as well as making it almost impossible to reuse callstack slots.
Using this environment variable will not only put a cap on the maximum
callstack depth, but it is also required if you want to use the next option.
.TP
.I NJAMD_NO_FREE_INFO=1
Setting this option will cause NJAMD to internally conserve heap table space
by not recording information about freed segments. Translation, things will go
quicker and take less memory in some cases, but you will lose information
about where segments are freed. This option may be necessary to debug HUGE
applications on 64bit procs, when using 
.I NJAMD_PROT=none
.TP
.I NJAMD_ALIGN=num
In order for programs to work correctly under certain archetectures (ie, sparc
and most other RISC CPUs), 
.B malloc 
must return memory aligned to a certain number of bytes if you want that
section of memory to contain pointers and floating point values. The alignment 
of your archetecture is detected automatically when you install 
.I NJAMD. 
However, note that aligning memory to
.I n 
bytes will cause the overflow detection to miss overflows of up to 
.I n 
bytes.
If you are on a RISC CPU, but know that alignment is not an issue in your 
program (ie, if it only deals with strings), then
.sp
.B export NJAMD_ALIGN=1
to set alignment to 1 byte.
.TP
.I NJAMD_DUMP_LEAKS_ON_EXIT=1
Setting this option instructs 
.I LIBNJAMD
to dump memory leak diagnostics to the front end (or standard error when
running standalone) upon program termination. With the advent of indefinate
callstack lengths in NJAMD 0.9.3, the amount of return address 
information dumped is no longer controlled by this setting, but by
.IR NJAMD_CALLSTACK_LIMIT "."
.sp
Do note that it is common
practice for short-lived programs such as 
.BR ls "(1) "
to simply exit without freeing memory.
.sp
.B export NJAMD_DUMP_LEAKS_ON_EXIT=1
.TP
.I NJAMD_DUMP_STATS_ON_EXIT=1
Setting this environment variable instructs NJAMD to dump a short summary of
memory usage versus address space usage. This option will help you figure out
how much overhead is being used by NJAMD, and how much address space in total
was needed to debug your application. In other words, it will either give you
an excuse to buy a 64 bit CPU, or a few more RAM chips ;)
.TP
.I NJAMD_DUMP_CORE=hard,soft
By default, NJAMD will catch all deadly signals in order to perform cleanup,
provide statistics, and give its own backtrace of when the fault occurred. 
However, when you are using a debugger, this behavior is not always desirable. 
.sp
If you would like NJAMD to perform cleanup and statistics information, but
would also like a core file, then 
.sp
.B export NJAMD_DUMP_CORE=soft
.sp
Using soft core dumping will cause the return address information 
.I INSIDE 
the core file to make no sense, but it will allow NJAMD to provide a call
stack dump upon exit. Using softcore also limits the coresize to 4megs
(defined through NJ_LIMIT_SOFTCORE in the source).
.sp
If you would like the core file to be perfectly valid and complete at the
expense of statistics and post-mortem heap integrity, then
.sp
.B export NJAMD_DUMP_CORE=hard
.TP
.I NJAMD_PERSISTENT_HEAP=1
Setting this option will cause the program heap to remain after execution in a
file forrmatted ./njamd-<pid>-heap. Unlike other malloc debuggers, saving 
the heap in 
.I NJAMD
imposes no extra performance overhead on the system. The heap is always mapped
to a file in tmp, but this file only persists if this option is set. To
utilize this file, use the njamd post-mortem utility
.BR njamdpm "(1). "
Be advised that the heap can take up as much as 8 megs on 32 bit systems.
ls may report it as 8 megs due to the lseek. The actual size is much
less. Use 
.BR du "(1) "
.sp
.B export NJAMD_PERSISTENT_HEAP=1
.TP
.I NJAMD_TRACE_LIBS=1
NJAMD's default action is to ignore shared library return addresses and only 
give you return addresses in the statically linked portion of the program.
This can be
a problem if your program consists of a large amount of supporting libraries,
and the static section is simply a main loop. So instead, to provide return
addresses in the libraries, set
.sp
.B export NJAMD_TRACE_LIBS=1
.TP
.I NJAMD_NO_TRACE=1
When debugging programs that use libraries compiled with optimization greater
than -O2 or with -fomit-frame-pointer, you must disable tracing, or NJAMD will
segfault. This isn't really NJAMD's fault. Gcc (see 'info gcc') claims that if
__builtin_return_address is unavailable it will just return NULL. Instead it
segmentation faults. I've notified the gcc team, but received no response.
.sp
.B export NJAMD_NO_TRACE=1
.TP
.I NJAMD_ALLOW_READ=1
Glibc versions up to and including 2.1.2 had a bug relating to the 
.BR sscanf "(3) "
code that was tripped by this library. As much as we would like to, 
unfortunately, we cannot take credit for being the first to discover this bug.
The glibc folk found and fixed the bug in version 2.1.3. At any rate, this
option is provided as a workaround. It allows reads past the end of a buffer 
to not segfault your program. The default is to forbid read, write, and
execute attempts.
.sp
.B export NJAMD_ALLOW_READ=1
.TP
.IR NJAMD_ALLOW_MALLOC_0=1 ", " NJAMD_ALLOW_FREE_0=1
ANSI C specifies that 
.IR malloc "'s" 
of 0 are legal, and that 
.IR free "'s" 
of 
.IR NULL 
are illegal. However, NJAMD's default action is to warn when either takes
place, as either usually indicates a bug (or at the very least, an unneeded
call). To turn this behavior off for either free or malloc, issue
.sp
.BR "export NJAMD_ALLOW_FREE_0=1" ", or"
.br
.B export NJAMD_ALLOW_MALLOC_0=1
respectively.
.TP
.IR NJAMD_ALLOW_MUTABLE_ALLOC=1
The ability to change allocation types duruing program exectution was added in 
NJAMD > 0.9.3.

.SS "Extra Functions"
The following functions are provided and may be called either from within your
program, or from 
.BR gdb "(1) "
without recompiling your executable.
.TP
.BR "void __nj_ptr_info(void *addr);"
This function will print out all available information on an address addr.
Addr need not be directly returned from malloc, it can point anywhere in a
malloced buffer. (Useful for finding out info about where Seg faulted memory
came from).
.TP
.BR "void __nj_dump_leaks(void);"
This will cause all non-freed memory to be reported to the screen.
.TP
.BR "void __nj_change_prot_type(char *newtype);"
If you used the environment variable 
.IR NJ_MUTABLE_PROT_TYPE ", " 
this function
will allow you to change the allocation types on the fly. The argument is a
string that is the same as the 
.I NJAMD_PROT
environment variable.

.TP
.BR "void __nj_change_free_type(char *newtype);"
This function will allow you to change the freed memory protection type on the
fly. The argument is a string that is the same as the 
.I NJAMD_CHK_FREE
environment variable.

.TP
.BR "void __nj_change_default_align(int newalign);"
This allows you to change the default alignment on the fly. It must be a power of two.
.TP
.BR "void __nj_change_trace_libs(int val);"
This function allows you to change the value of the 
.I NJAMD_NO_TRACE
setting. Its argument is either true or false, true if you want callstack
traces through libraries.

.TP
.BI "void __nj_change_free_info(int val);"
This function changes the value of the
.I NJAMD_NO_FREE_INFO
setting. Its argument is either true or false, true if you want information
about freed segments. Note that in order to set this to false, you must have
set the 
.I NJAMD_CALLSTACK_LIMIT
option, so that we can reuse the freed segments' information and callstack
blocks easily.

.SS "Debugging from within gdb"
Using 
.I LIBNJAMD
from within gdb is simple. The command to set environment variables in gdb is
.B set env VARIABLE=value.
So, to instruct gdb to use 
.IR LIBNJAMD ", "
issue
.sp
.B set env LD_PRELOAD=libnjamd.so
.sp
from 
.I INSIDE
gdb. Issuing an LD_PRELOAD command to your shell before starting gdb causes
gdb to use that library as well, which means gdb would be using NJAMD, and
unless you're on the gdb development team, you're probably not interested in
debugging gdb :)
.sp
All other options can be set in the same mannor from with gdb, or in the
shell's environment outside gdb. 
.sp
You can obtain memory leak information at any point by setting a break point,
and then issuing
.B signal SIGUSR1
at that breakpoint. This will provide a memory leak dump as described above. 
In addition, so will
.br
.B call __nj_dump_leaks(__nj_display_depth)

.sp
You can also get information about any address used by njamd by issuing 
.B call __nj_ptr_info(address)
.sp
This will dump out a call stack of the malloc or free that contains that
address. This is very useful for gaining information about a segmentation
fault. Ie if the segfault occurs on a line that does buf[i] = 2, issue
.br
.B call __nj_ptr_info(&buf[i])
to gdb.
.sp
To get gdb to translate these return addresses into something meaningful,
issue 
.sp
.B info line *0xaddress
to obtain the line number of the allocation request, or 
.sp
.B list *0xaddress 
to see the adjacent code as well.
.sp
Another neat trick you may find handy for tracking down things like free(NULL)
and malloc(0) and other behavior that produces warnings is to set a breakpoint
at __nj_eprintf so that you can determine the location of the offending 
instruction. __nj_eprintf is 
.IR NJAMD "'s "
general purpose error function. It is called to print out any NJAMD warning or
error message you see. Note that if you are using LD_PRELOAD, you may have to
set a breakpoint in main and start the program before setting this breakpoint 
for gdb to know that __nj_eprintf is a valid symbol.
.sp
Furthermore, any of the 
.B Extra Functions
may be called directly from gdb using 
.BR call "."

.SS "Dumping memory information"
Issuing a kill -USR1 to a program running under
.I LIBNJAMD
will cause memory usage diagnostics to be dumped to standard error. These are
human readable lines of the form
.sp
.B njamd: Memory leak of XX bytes allocated at address 0xXXXXXXX.
.sp
The address given is the actual address of the corresponding memory allocation
in 
.I your 
.IR code ". "
In future versions of
.IR NJAMD ", " 
scripts or runtime functionality will be provided to translate these addresses
on the fly to functions and line numbers. Until then,
use 
.BR gdb "(1) "
to translate them for you.

.SH BUGS
If anything goes wrong, please read the NOTES file in the NJAMD source 
directory,
.I ESPECIALLY
before reporting any bugs. Many platforms and compiler flag combinations
cause problems. Please read that file for more info. Also, whenever sending me
a bug report, please send a self-contained code snippet to reproduce the bug.

.SH PERFORMANCE
If you notice large performance problems, especially on the xBSD's, 
.sp
.B export NJAMD_CHK_FREE=none
.sp
Setting the checking of freed memory to none allows for an optimization that
can speed up NJAMD 10 fold, making it comparable to libc malloc in some usage
cases!
.sp
Furturemore, by default NJAMD uses the minimum alignment possible that won't cause a
segfault. Increasing this alignment may increase performance on some
archetectures, because you won't lose cycles for unaligned memory accesses.
Try setting
.sp
.B export NJAMD_ALIGN=4 or 8
.sp
Setting the callstack limit to a small value, or setting 
.I NJAMD_NO_TRACE
will speed things up quite a bit, because then NJAMD doesn't have to record
return addresses.
.sp
.B export NJAMD_NO_TRACE=1
.sp
Also, using the 
.I NJAMD_MUTABLE_ALLOC
option will make things run slower while freeing memory, as NJAMD must then determine the allocation mechanism used for each block before freeing it.

.SH NOTES
I've discovered that
the limit of 65536 mappings in Linux includes permission variations. In other
words, you will not be able to debug large apps due to this limit. To get
around this, apply either fix_map-<kernel_version>.patch or
proc_map-<kernel_version>.patch to your kernel, and recompile. See the 
.I ./kernel_mod/README
file for more info.
.sp
Core dump sizes are limited to 4 megs (and are probably next to useless)
because of the huge amount of mapping that takes place. Some OS's (again,
those evil BSD's ;) actually
zero-fill mapped but unfaulted memory as it is dumped to disk, causing a core
dump to take a horrendous amout of time and disk space. 
.sp 
For information on how the system works, read the programmers documentation in
the source tree, as well as the Doxygen generated docs, and check out my (admittantly incomplete) Shared Memory
HOWTO:
.nl
.B http://fscked.org/proj/njamd/docs/internal/html/
.nl
.B http://fscked.org/writings/SHM/shm.html 
.SH AUTHORS
.nf
Mike Perry <mikepery@fscked.org> - libnjamd
Steve Engelhardt <sengelha@yahoo.com> - Front End
.fi
.ft
.SH "SEE ALSO"
.BR http://freshmeat.net/appindex/development/debugging.html
.sp
.BR njamdpm "(1), " efence "(3), " malloc "(3), " mmap "(2), " mprotect "(2)"