Valgrind FAQ
Release 3.3.1 4 June 2008
~~~~~~~~~~~~~~~~~~~~~~~~~

Table of Contents
1. Background
2. Compiling, installing and configuring
3. Valgrind aborts unexpectedly
4. Valgrind behaves unexpectedly
5. Memcheck doesn't find my bug
6. Miscellaneous
7. How To Get Further Assistance

------------------------------------------------------------------------
1. Background
------------------------------------------------------------------------

1.1. How do you pronounce "Valgrind"?

The "Val" as in the world "value". The "grind" is pronounced with a 
short 'i' -- ie. "grinned" (rhymes with "tinned") rather than "grined" 
(rhymes with "find"). 

Don't feel bad: almost everyone gets it wrong at first.

------------------------------------------------------------------------

1.2. Where does the name "Valgrind" come from?

From Nordic mythology. Originally (before release) the project was named 
Heimdall, after the watchman of the Nordic gods. He could "see a hundred 
miles by day or night, hear the grass growing, see the wool growing on a 
sheep's back" (etc). This would have been a great name, but it was 
already taken by a security package "Heimdal". 

Keeping with the Nordic theme, Valgrind was chosen. Valgrind is the name 
of the main entrance to Valhalla (the Hall of the Chosen Slain in 
Asgard). Over this entrance there resides a wolf and over it there is 
the head of a boar and on it perches a huge eagle, whose eyes can see to 
the far regions of the nine worlds. Only those judged worthy by the 
guardians are allowed to pass through Valgrind. All others are refused 
entrance. 

It's not short for "value grinder", although that's not a bad guess.


------------------------------------------------------------------------
2. Compiling, installing and configuring
------------------------------------------------------------------------

2.1. When I trying building Valgrind, 'make' dies partway with an 
assertion failure, something like this: 

    % make: expand.c:489: allocated_variable_append: 
            Assertion 'current_variable_set_list->next != 0' failed.
    
It's probably a bug in 'make'. Some, but not all, instances of version 
3.79.1 have this bug, see 
www.mail-archive.com/bug-make@gnu.org/msg01658.html. Try upgrading to a 
more recent version of 'make'. Alternatively, we have heard that 
unsetting the CFLAGS environment variable avoids the problem. 

------------------------------------------------------------------------

2.2. When I try to build Valgrind, 'make' fails with /usr/bin/ld: cannot 
find -lc collect2: ld returned 1 exit status 

You need to install the glibc-static-devel package.


------------------------------------------------------------------------
3. Valgrind aborts unexpectedly
------------------------------------------------------------------------

3.1. Programs run OK on Valgrind, but at exit produce a bunch of errors 
involving __libc_freeres() and then die with a segmentation fault. 

When the program exits, Valgrind runs the procedure __libc_freeres() in 
glibc. This is a hook for memory debuggers, so they can ask glibc to 
free up any memory it has used. Doing that is needed to ensure that 
Valgrind doesn't incorrectly report space leaks in glibc. 

Problem is that running __libc_freeres() in older glibc versions causes 
this crash. 

Workaround for 1.1.X and later versions of Valgrind: use the 
--run-libc-freeres=no flag. You may then get space leak reports for 
glibc allocations (please don't report these to the glibc people, since 
they are not real leaks), but at least the program runs. 

------------------------------------------------------------------------

3.2. My (buggy) program dies like this:

    valgrind: m_mallocfree.c:442 (bszW_to_pszW): Assertion 'pszW >= 0' failed.

If Memcheck (the memory checker) shows any invalid reads, invalid writes 
or invalid frees in your program, the above may happen. Reason is that 
your program may trash Valgrind's low-level memory manager, which then 
dies with the above assertion, or something similar. The cure is to fix 
your program so that it doesn't do any illegal memory accesses. The 
above failure will hopefully go away after that. 

------------------------------------------------------------------------

3.3. My program dies, printing a message like this along the way:

    vex x86->IR: unhandled instruction bytes: 0x66 0xF 0x2E 0x5

Older versions did not support some x86 and amd64 instructions, 
particularly SSE/SSE2/SSE3 instructions. Try a newer Valgrind; we now 
support almost all instructions. If it still breaks, file a bug report. 

Another possibility is that your program has a bug and erroneously jumps 
to a non-code address, in which case you'll get a SIGILL signal. 
Memcheck may issue a warning just before this happens, but it might not 
if the jump happens to land in addressable memory. 

------------------------------------------------------------------------

3.4. I tried running a Java program (or another program that uses a 
just-in-time compiler) under Valgrind but something went wrong. Does 
Valgrind handle such programs? 

Valgrind can handle dynamically generated code, so long as none of the 
generated code is later overwritten by other generated code. If this 
happens, though, things will go wrong as Valgrind will continue running 
its translations of the old code (this is true on x86 and amd64, on 
PowerPC there are explicit cache flush instructions which Valgrind 
detects and honours). You should try running with --smc-check=all in 
this case. Valgrind will run much more slowly, but should detect the use 
of the out-of-date code. 

Alternatively, if you have the source code to the JIT compiler you can 
insert calls to the VALGRIND_DISCARD_TRANSLATIONS client request to mark 
out-of-date code, saving you from using --smc-check=all. 

Apart from this, in theory Valgrind can run any Java program just fine, 
even those that use JNI and are partially implemented in other languages 
like C and C++. In practice, Java implementations tend to do nasty 
things that most programs do not, and Valgrind sometimes falls over 
these corner cases. 

If your Java programs do not run under Valgrind, even with 
--smc-check=all, please file a bug report and hopefully we'll be able to 
fix the problem. 


------------------------------------------------------------------------
4. Valgrind behaves unexpectedly
------------------------------------------------------------------------

4.1. My program uses the C++ STL and string classes. Valgrind reports 
'still reachable' memory leaks involving these classes at the exit of 
the program, but there should be none. 

First of all: relax, it's probably not a bug, but a feature. Many 
implementations of the C++ standard libraries use their own memory pool 
allocators. Memory for quite a number of destructed objects is not 
immediately freed and given back to the OS, but kept in the pool(s) for 
later re-use. The fact that the pools are not freed at the exit() of the 
program cause Valgrind to report this memory as still reachable. The 
behaviour not to free pools at the exit() could be called a bug of the 
library though. 

Using gcc, you can force the STL to use malloc and to free memory as 
soon as possible by globally disabling memory caching. Beware! Doing so 
will probably slow down your program, sometimes drastically. 

* With gcc 2.91, 2.95, 3.0 and 3.1, compile all source using the STL 
with -D__USE_MALLOC. Beware! This was removed from gcc starting with 
version 3.3. 

* With gcc 3.2.2 and later, you should export the environment variable 
GLIBCPP_FORCE_NEW before running your program. 

* With gcc 3.4 and later, that variable has changed name to 
GLIBCXX_FORCE_NEW. 

There are other ways to disable memory pooling: using the malloc_alloc 
template with your objects (not portable, but should work for gcc) or 
even writing your own memory allocators. But all this goes beyond the 
scope of this FAQ. Start by reading 
http://gcc.gnu.org/onlinedocs/libstdc++/faq/index.html#4_4_leak: 
<http://gcc.gnu.org/onlinedocs/libstdc++/faq/index.html#4_4_leak> if you 
absolutely want to do that. But beware: allocators belong to the more 
messy parts of the STL and people went to great lengths to make the STL 
portable across platforms. Chances are good that your solution will work 
on your platform, but not on others. 

------------------------------------------------------------------------

4.2. The stack traces given by Memcheck (or another tool) aren't 
helpful. How can I improve them? 

If they're not long enough, use --num-callers to make them longer.

If they're not detailed enough, make sure you are compiling with -g to 
add debug information. And don't strip symbol tables (programs should be 
unstripped unless you run 'strip' on them; some libraries ship 
stripped). 

Also, for leak reports involving shared objects, if the shared object is 
unloaded before the program terminates, Valgrind will discard the debug 
information and the error message will be full of ??? entries. The 
workaround here is to avoid calling dlclose() on these shared objects. 

Also, -fomit-frame-pointer and -fstack-check can make stack traces 
worse. 

Some example sub-traces:

* With debug information and unstripped (best):

    Invalid write of size 1
       at 0x80483BF: really (malloc1.c:20)
       by 0x8048370: main (malloc1.c:9)
    
* With no debug information, unstripped:

    Invalid write of size 1
       at 0x80483BF: really (in /auto/homes/njn25/grind/head5/a.out)
       by 0x8048370: main (in /auto/homes/njn25/grind/head5/a.out)
    
* With no debug information, stripped:

    Invalid write of size 1
       at 0x80483BF: (within /auto/homes/njn25/grind/head5/a.out)
       by 0x8048370: (within /auto/homes/njn25/grind/head5/a.out)
       by 0x42015703: __libc_start_main (in /lib/tls/libc-2.3.2.so)
       by 0x80482CC: (within /auto/homes/njn25/grind/head5/a.out)
    
* With debug information and -fomit-frame-pointer:

    Invalid write of size 1
       at 0x80483C4: really (malloc1.c:20)
       by 0x42015703: __libc_start_main (in /lib/tls/libc-2.3.2.so)
       by 0x80482CC: ??? (start.S:81)
    
* A leak error message involving an unloaded shared object:

    84 bytes in 1 blocks are possibly lost in loss record 488 of 713
       at 0x1B9036DA: operator new(unsigned) (vg_replace_malloc.c:132)
       by 0x1DB63EEB: ???
       by 0x1DB4B800: ???
       by 0x1D65E007: ???
       by 0x8049EE6: main (main.cpp:24)
    
------------------------------------------------------------------------

4.3. The stack traces given by Memcheck (or another tool) seem to have 
the wrong function name in them. What's happening? 

Occasionally Valgrind stack traces get the wrong function names. This is 
caused by glibc using aliases to effectively give one function two 
names. Most of the time Valgrind chooses a suitable name, but very 
occasionally it gets it wrong. Examples we know of are printing 'bcmp' 
instead of 'memcmp', 'index' instead of 'strchr', and 'rindex' instead 
of 'strrchr'. 

------------------------------------------------------------------------

4.4. My program crashes normally, but doesn't under Valgrind, or vice 
versa. What's happening? 

When a program runs under Valgrind, its environment is slightly 
different to when it runs natively. For example, the memory layout is 
different, and the way that threads are scheduled is different. 

Most of the time this doesn't make any difference, but it can, 
particularly if your program is buggy. For example, if your program 
crashes because it erroneously accesses memory that is unaddressable, 
it's possible that this memory will not be unaddressable when run under 
Valgrind. Alternatively, if your program has data races, these may not 
manifest under Valgrind. 

There isn't anything you can do to change this, it's just the nature of 
the way Valgrind works that it cannot exactly replicate a native 
execution environment. In the case where your program crashes due to a 
memory error when run natively but not when run under Valgrind, in most 
cases Memcheck should identify the bad memory operation. 


------------------------------------------------------------------------
5. Memcheck doesn't find my bug
------------------------------------------------------------------------

5.1. I try running "valgrind --tool=memcheck my_program" and get 
Valgrind's startup message, but I don't get any errors and I know my 
program has errors. 

There are two possible causes of this.

First, by default, Valgrind only traces the top-level process. So if 
your program spawns children, they won't be traced by Valgrind by 
default. Also, if your program is started by a shell script, Perl 
script, or something similar, Valgrind will trace the shell, or the Perl 
interpreter, or equivalent. 

To trace child processes, use the --trace-children=yes option.

If you are tracing large trees of processes, it can be less disruptive 
to have the output sent over the network. Give Valgrind the flag 
--log-socket=127.0.0.1:12345 (if you want logging output sent to port 
12345 on localhost). You can use the valgrind-listener program to listen 
on that port: 

    valgrind-listener 12345
    
Obviously you have to start the listener process first. See the manual 
for more details. 

Second, if your program is statically linked, most Valgrind tools won't 
work as well, because they won't be able to replace certain functions, 
such as malloc(), with their own versions. A key indicator of this is if 
Memcheck says: All heap blocks were freed -- no leaks are possible when 
you know your program calls malloc(). The workaround is to avoid 
statically linking your program. 

------------------------------------------------------------------------

5.2. Why doesn't Memcheck find the array overruns in this program?

    int static[5];
    
    int main(void)
    {
      int stack[5];
    
      static[5] = 0;
      stack [5] = 0;
              
      return 0;
    }
    
Unfortunately, Memcheck doesn't do bounds checking on static or stack 
arrays. We'd like to, but it's just not possible to do in a reasonable 
way that fits with how Memcheck works. Sorry. 


------------------------------------------------------------------------
6. Miscellaneous
------------------------------------------------------------------------

6.1. I tried writing a suppression but it didn't work. Can you write my 
suppression for me? 

Yes! Use the --gen-suppressions=yes feature to spit out suppressions 
automatically for you. You can then edit them if you like, eg. combining 
similar automatically generated suppressions using wildcards like '*'. 

If you really want to write suppressions by hand, read the manual 
carefully. Note particularly that C++ function names must be mangled 
(that is, not demangled). 

------------------------------------------------------------------------

6.2. With Memcheck's memory leak detector, what's the difference between 
"definitely lost", "possibly lost", "still reachable", and "suppressed"? 

The details are in the Memcheck section of the user manual.

In short:

* "definitely lost" means your program is leaking memory -- fix it!

* "possibly lost" means your program is probably leaking memory, unless 
you're doing funny things with pointers. 

* "still reachable" means your program is probably ok -- it didn't free 
some memory it could have. This is quite common and often reasonable. 
Don't use --show-reachable=yes if you don't want to see these reports. 

* "suppressed" means that a leak error has been suppressed. There are 
some suppressions in the default suppression files. You can ignore 
suppressed errors. 

------------------------------------------------------------------------

6.3. Memcheck's uninitialised value errors are hard to track down, 
because they are often reported some time after they are caused. Could 
Memcheck record a trail of operations to better link the cause to the 
effect? Or maybe just eagerly report any copies of uninitialised memory 
values? 

We'd love to improve these errors, but we don't know how to do it 
without huge performance penalties. 

You can use the client request VALGRIND_CHECK_VALUE_IS_DEFINED to help 
track these errors down -- work backwards from the point where the 
uninitialised error occurs, checking suspect values until you find the 
cause. This requires editing, compiling and re-running your program 
multiple times, which is a pain, but still easier than debugging the 
problem without Memcheck's help. 

As for eager reporting of copies of uninitialised memory values, this 
has been suggested multiple times. Unfortunately, almost all programs 
legitimately copy uninitialised memory values around (because compilers 
pad structs to preserve alignment) and eager checking leads to hundreds 
of false positives. Therefore Memcheck does not support eager checking 
at this time. 


------------------------------------------------------------------------
7. How To Get Further Assistance
------------------------------------------------------------------------

Please read all of this section before posting.

If you think an answer is incomplete or inaccurate, please e-mail 
valgrind@valgrind.org: <valgrind@valgrind.org>. 

Read the appropriate section(s) of the Valgrind Documentation: 
<http://www.valgrind.org/docs/manual/index.html>. 

Read the Distribution Documents: 
<http://www.valgrind.org/docs/manual/dist.html>. 

Search: <http://search.gmane.org> the valgrind-users: 
<http://news.gmane.org/gmane.comp.debugging.valgrind> mailing list 
archives, using the group name gmane.comp.debugging.valgrind. 

Only when you have tried all of these things and are still stuck, should 
you post to the valgrind-users mailing list: 
<http://lists.sourceforge.net/lists/listinfo/valgrind-users>. In which 
case, please read the following carefully. Making a complete posting 
will greatly increase the chances that an expert or fellow user reading 
it will have enough information and motivation to reply. 

Make sure you give full details of the problem, including the full 
output of valgrind -v <your-prog>, if applicable. Also which Linux 
distribution you're using (Red Hat, Debian, etc) and its version number. 

You are in little danger of making your posting too long unless you 
include large chunks of Valgrind's (unsuppressed) output, so err on the 
side of giving too much information. 

Clearly written subject lines and message bodies are appreciated, too.

Finally, remember that, despite the fact that most of the community are 
very helpful and responsive to emailed questions, you are probably 
requesting help from unpaid volunteers, so you have no guarantee of 
receiving an answer.