Valgrind

1. Background
1.1. How do you pronounce 'Valgrind'?
1.2. Where does the name 'Valgrind' come from?
  1. Valgrind Memcheck
  2. Valgrind Invalid Read Of Size 1
Leaks

Valgrind's Memcheck tool detects a comprehensive set of memory errors, including reads and writes of unallocated or freed memory and memory leaks. The Valgrind User Manual and its Memcheck section describe in detail how Valgrind and Memcheck work, the options for use, and more details about what causes false positives.

  1. Understanding Valgrind Output. Valgrind is a program that checks for both memory leaks and runtime errors. A memory leak occurs whenever you allocate memory using keywords like new or malloc, without subsequently deleting or freeing that memory before the program exits.
  2. Valgrind reports 4 blocks lost - those are the four names we forgot to free.They are still reachable because their pointers are still in the lines array. For a clean run of valgrind, as expected in all your CS50 labs, the program would need to actually free every block of memory it allocates.

Valgrind Memcheck

2. Compiling, installing and configuring
2.1. When I trying building Valgrind, 'make' dies partway with an assertion failure, something like this:
2.2. When I try to build Valgrind, 'make' fails with/usr/bin/ld: cannot find -lccollect2: ld returned 1 exit status
3. Valgrind aborts unexpectedly
3.1. Programs run OK on Valgrind, but at exit produce a bunch of errors involving __libc_freeres() and then die with a segmentation fault.
3.2. My (buggy) program dies like this:
3.3. My program dies, printing a message like this along the way:
3.4. I tried running a Java program (or another program that uses a just-in-time compiler) under Valgrind but something went wrong. Does Valgrind handle such programs?
4. Valgrind behaves unexpectedly
4.1. My program uses the C++ STL and string classes. Valgrind reports 'still reachable' memory leaks involving these classes at the exit of the program, but there should be none.
4.2. The stack traces given by Memcheck (or another tool) aren't helpful. How can I improve them?
4.3. The stack traces given by Memcheck (or another tool) seem to have the wrong function name in them. What's happening?
5. Memcheck doesn't find my bug
5.1. I try running 'valgrind --tool=memcheck my_program' and get Valgrind's startup message, but I don't get any errors and I know my program has errors.
5.2. Why doesn't Memcheck find the array overruns in this program?
6. Miscellaneous
6.1. I tried writing a suppression but it didn't work. Can you write my suppression for me?
6.2. With Memcheck's memory leak detector, what's the difference between 'definitely lost', 'possibly lost', 'still reachable', and 'suppressed'?
7. How To Get Further Assistance

1. Background

1.1. How do you pronounce 'Valgrind'?
1.2. Where does the name 'Valgrind' come from?
1.1.How do you pronounce 'Valgrind'?

The 'Val' as in the world 'value'. The 'grind' is pronounced with a short 'i' -- ie. 'grinned' (rhymes with 'tinned') rather than 'grined' (rhymes with 'find').

Don't feel bad: almost everyone gets it wrong at first.

1.2.Where does the name 'Valgrind' come from?

From Nordic mythology. Originally (before release) the project was named Heimdall, after the watchman of the Nordic gods. He could 'see a hundred miles by day or night, hear the grass growing, see the wool growing on a sheep's back' (etc). This would have been a great name, but it was already taken by a security package 'Heimdal'.

Keeping with the Nordic theme, Valgrind was chosen. Valgrind is the name of the main entrance to Valhalla (the Hall of the Chosen Slain in Asgard). Over this entrance there resides a wolf and over it there is the head of a boar and on it perches a huge eagle, whose eyes can see to the far regions of the nine worlds. Only those judged worthy by the guardians are allowed to pass through Valgrind. All others are refused entrance.

It's not short for 'value grinder', although that's not a bad guess.


2. Compiling, installing and configuring

2.1. When I trying building Valgrind, 'make' dies partway with an assertion failure, something like this:
2.2. When I try to build Valgrind, 'make' fails with/usr/bin/ld: cannot find -lccollect2: ld returned 1 exit status
2.1.When I trying building Valgrind, 'make' dies partway with an assertion failure, something like this:

It's probably a bug in 'make'. Some, but not all, instances of version 3.79.1 have this bug, see www.mail-archive.com/bug-make@gnu.org/msg01658.html. Try upgrading to a more recent version of 'make'. Alternatively, we have heard that unsetting the CFLAGS environment variable avoids the problem.

2.2.When I try to build Valgrind, 'make' fails with

You need to install the glibc-static-devel package.

Valgrind

3. Valgrind aborts unexpectedly

3.1. Programs run OK on Valgrind, but at exit produce a bunch of errors involving __libc_freeres() and then die with a segmentation fault.
3.2. My (buggy) program dies like this:
3.3. My program dies, printing a message like this along the way:
3.4. I tried running a Java program (or another program that uses a just-in-time compiler) under Valgrind but something went wrong. Does Valgrind handle such programs?
3.1.Programs run OK on Valgrind, but at exit produce a bunch of errors involving __libc_freeres() and then die with a segmentation fault.

When the program exits, Valgrind runs the procedure __libc_freeres() in glibc. This is a hook for memory debuggers, so they can ask glibc to free up any memory it has used. Doing that is needed to ensure that Valgrind doesn't incorrectly report space leaks in glibc.

Problem is that running __libc_freeres() in older glibc versions causes this crash.

WORKAROUND FOR 1.1.X and later versions of Valgrind: use the --run-libc-freeres=no flag. You may then get space leak reports for glibc-allocations (please _don't_ report these to the glibc people, since they are not real leaks), but at least the program runs.

3.2.My (buggy) program dies like this:

If Memcheck (the memory checker) shows any invalid reads, invalid writes and invalid frees in your program, the above may happen. Reason is that your program may trash Valgrind's low-level memory manager, which then dies with the above assertion, or something like this. The cure is to fix your program so that it doesn't do any illegal memory accesses. The above failure will hopefully go away after that.

3.3.My program dies, printing a message like this along the way:

Older versions did not support some x86 instructions, particularly SSE/SSE2 instructions. Try a newer Valgrind; we now support almost all instructions. If it still happens with newer versions, if the failing instruction is an SSE/SSE2 instruction, you might be able to recompile your program without it by using the flag -march to gcc. Either way, let us know and we'll try to fix it.

Another possibility is that your program has a bug and erroneously jumps to a non-code address, in which case you'll get a SIGILL signal. Memcheck may issue a warning just before this happens, but they might not if the jump happens to land in addressable memory.

3.4.I tried running a Java program (or another program that uses a just-in-time compiler) under Valgrind but something went wrong. Does Valgrind handle such programs?

Valgrind can handle dynamically generated code, so long as none of the generated code is later overwritten by other generated code. If this happens, though, things will go wrong as Valgrind will continue running its translations of the old code (this is true on x86 and AMD64, on PPC32 there are explicit cache flush instructions which Valgrind detects). You should try running with --smc-check=all in this case; Valgrind will run much more slowly, but should detect the use of the out-of-date code.

Alternativaly, if you have the source code to the JIT compiler you can insert calls to the VALGRIND_DISCARD_TRANSLATIONS client request to mark out-of-date code, saving you from using --smc-check=all.

Apart from this, in theory Valgrind can run any Java program just fine, even those that use JNI and are partially implemented in other languages like C and C++. In practice, Java implementations tend to do nasty things that most programs do not, and Valgrind sometimes falls over these corner cases.

If your Java programs do not run under Valgrind, even with --smc-check=all, please file a bug report and hopefully we'll be able to fix the problem.


4. Valgrind behaves unexpectedly

4.1. My program uses the C++ STL and string classes. Valgrind reports 'still reachable' memory leaks involving these classes at the exit of the program, but there should be none.
4.2. The stack traces given by Memcheck (or another tool) aren't helpful. How can I improve them?
4.3. The stack traces given by Memcheck (or another tool) seem to have the wrong function name in them. What's happening?
4.1.My program uses the C++ STL and string classes. Valgrind reports 'still reachable' memory leaks involving these classes at the exit of the program, but there should be none.

First of all: relax, it's probably not a bug, but a feature. Many implementations of the C++ standard libraries use their own memory pool allocators. Memory for quite a number of destructed objects is not immediately freed and given back to the OS, but kept in the pool(s) for later re-use. The fact that the pools are not freed at the exit() of the program cause Valgrind to report this memory as still reachable. The behaviour not to free pools at the exit() could be called a bug of the library though.

Using gcc, you can force the STL to use malloc and to free memory as soon as possible by globally disabling memory caching. Beware! Doing so will probably slow down your program, sometimes drastically.

  • With gcc 2.91, 2.95, 3.0 and 3.1, compile all source using the STL with -D__USE_MALLOC. Beware! This is removed from gcc starting with version 3.3.

  • With gcc 3.2.2 and later, you should export the environment variable GLIBCPP_FORCE_NEW before running your program.

  • With gcc 3.4 and later, that variable has changed name to GLIBCXX_FORCE_NEW.

There are other ways to disable memory pooling: using the malloc_alloc template with your objects (not portable, but should work for gcc) or even writing your own memory allocators. But all this goes beyond the scope of this FAQ. Start by reading http://gcc.gnu.org/onlinedocs/libstdc++/ext/howto.html#3 if you absolutely want to do that. But beware:

  1. there are currently changes underway for gcc which are not totally reflected in the docs right now ('now' 26 Apr 03)

  2. allocators belong to the more messy parts of the STL and people went to great lengths to make it portable across platforms. Chances are good that your solution will work on your platform, but not on others.

4.2.The stack traces given by Memcheck (or another tool) aren't helpful. How can I improve them?

If they're not long enough, use --num-callers to make them longer.

If they're not detailed enough, make sure you are compiling with -g to add debug information. And don't strip symbol tables (programs should be unstripped unless you run 'strip' on them; some libraries ship stripped).

Also, for leak reports involving shared objects, if the shared object is unloaded before the program terminates, Valgrind will discard the debug information and the error message will be full of ??? entries. The workaround here is to avoid calling dlclose() on these shared objects.

Also, -fomit-frame-pointer and -fstack-check can make stack traces worse.

Some example sub-traces:

  • With debug information and unstripped (best):

  • With no debug information, unstripped:

  • With no debug information, stripped:

  • With debug information and -fomit-frame-pointer:

  • A leak error message involving an unloaded shared object:

4.3.The stack traces given by Memcheck (or another tool) seem to have the wrong function name in them. What's happening?

Occasionally Valgrind stack traces get the wrong function names. This is caused by glibc using aliases to effectively give one function two names. Most of the time Valgrind chooses a suitable name, but very occasionally it gets it wrong. Examples we know of are printing 'bcmp' instead of 'memcmp', 'index' instead of 'strchr', and 'rindex' instead of 'strrchr'.


5. Memcheck doesn't find my bug

5.1. I try running 'valgrind --tool=memcheck my_program' and get Valgrind's startup message, but I don't get any errors and I know my program has errors.
5.2. Why doesn't Memcheck find the array overruns in this program?
5.1.I try running 'valgrind --tool=memcheck my_program' and get Valgrind's startup message, but I don't get any errors and I know my program has errors.

There are two possible causes of this.

First, by default, Valgrind only traces the top-level process. So if your program spawns children, they won't be traced by Valgrind by default. Also, if your program is started by a shell script, Perl script, or something similar, Valgrind will trace the shell, or the Perl interpreter, or equivalent.

To trace child processes, use the --trace-children=yes option.

If you are tracing large trees of processes, it can be less disruptive to have the output sent over the network. Give Valgrind the flag --log-socket=127.0.0.1:12345 (if you want logging output sent to port 12345 on localhost). You can use the valgrind-listener program to listen on that port:

Obviously you have to start the listener process first. See the manual for more details.

Second, if your program is statically linked, most Valgrind tools won't work as well, because they won't be able to replace certain functions, such as malloc(), with their own versions. A key indicator of this is if Memcheck says:

when you know your program calls malloc(). The workaround is to avoid statically linking your program.

5.2.Why doesn't Memcheck find the array overruns in this program?

Unfortunately, Memcheck doesn't do bounds checking on static or stack arrays. We'd like to, but it's just not possible to do in a reasonable way that fits with how Memcheck works. Sorry.


6. Miscellaneous

6.1. I tried writing a suppression but it didn't work. Can you write my suppression for me?
6.2. With Memcheck's memory leak detector, what's the difference between 'definitely lost', 'possibly lost', 'still reachable', and 'suppressed'?
6.1.I tried writing a suppression but it didn't work. Can you write my suppression for me?

Yes! Use the --gen-suppressions=yes feature to spit out suppressions automatically for you. You can then edit them if you like, eg. combining similar automatically generated suppressions using wildcards like '*'.

If you really want to write suppressions by hand, read the manual carefully. Note particularly that C++ function names must be _mangled_.

6.2.With Memcheck's memory leak detector, what's the difference between 'definitely lost', 'possibly lost', 'still reachable', and 'suppressed'?

The details are in the Memcheck section of the user manual.

In short:

  • 'definitely lost' means your program is leaking memory -- fix it!

  • 'possibly lost' means your program is probably leaking memory, unless you're doing funny things with pointers.

  • 'still reachable' means your program is probably ok -- it didn't free some memory it could have. This is quite common and often reasonable. Don't use --show-reachable=yes if you don't want to see these reports.

  • 'suppressed' means that a leak error has been suppressed. There are some suppressions in the default suppression files. You can ignore suppressed errors.


Valgrind Invalid Read Of Size 1

7. How To Get Further Assistance

Please read all of this section before posting.

If you think an answer is incomplete or inaccurate, please e-mail valgrind@valgrind.org.

Read the appropriate section(s) of the Valgrind Documentation.

Read the Distribution Documents.

Search the valgrind-users mailing list archives, using the group name gmane.comp.debugging.valgrind.

Only when you have tried all of these things and are still stuck, should you post to the valgrind-users mailing list. In which case, please read the following carefully. Making a complete posting will greatly increase the chances that an expert or fellow user reading it will have enough information and motivation to reply.

Make sure you give full details of the problem, including the full output of valgrind -v <your-prog>, if applicable. Also which Linux distribution you're using (Red Hat, Debian, etc) and its version number.

You are in little danger of making your posting too long unless you include large chunks of Valgrind's (unsuppressed) output, so err on the side of giving too much information.

Clearly written subject lines and message bodies are appreciated, too.

Finally, remember that, despite the fact that most of the community are very helpful and responsive to emailed questions, you are probably requesting help from unpaid volunteers, so you have no guarantee of receiving an answer.