This section is for people who already use an older version of the malloc debug library and want to have a quick glance whether it's useful to update.
Compiling with RM_TEST_DEPTH set to 0 resulted in compiling errors. This is fixed in this version.
This version adopts a patch by calloc which
is not explicitely wrapped in the library. I considered it not
necessary because calloc should use malloc internally.
But heīs right, one cannot be sure and should not rely on internal
features!
He also proposed to add the standard external "C"
wrapping for C++ in the header. I wasnīt sure at first because I never
planned this library to be used for C++ projects (itīs not perfectly
useful for that purpose). But who am I to tell you what to do?
So itīs added, too.
One disadvantage of previous releases was that only the calls of functions from the malloc library in prepared files (i.e. files which include rmalloc.h) were tracked. This could result in subtle errors when memory was released in a prepared file which was allocated elsewhere (resulting in a double or false delete error) and especially vice versa. This made it necessary to write wrappers for library functions returning malloced space (you can still find an example for getcwd in the source).
Version 2.00beta1 is able to track every call to functions from the
malloc library. To achieve this it uses a special malloc library
provided by
As always there is a little disadvantage. You cannot switch the
malloc debugging off as easily as before by just removing the
definition of the preprocessor macro MALLOC_DEBUG. Now
you have to remove the rmalloc.o from the linkage line, too.
I have tested this version of the malloc debug library with different projects under Linux and OSF1 (DEC Alpha) for some time but still consider it beta. There weren't any problems yet but because it's a critical library I would like to give it some more testing before giving it my ok.
The previous version (1.14) is very well-tested by me and several other users on a bunch of different unix systems. Therefore it is still available in the download area. See the file rmdebug.html in the packet for the correct information of that version.
You can switch off the new feature by removing the definition
of TRACK_EXTERNAL_CALLS from the file
rmalloc.c.
When writing C programs many of the bugs are memory related. They may result in strange behaviour which can arouse far away from the erroneous position.
I once spent about thirty hours hunting for an annoying bug. The setting
was bad. In a big project where a program (for which we don't had the
source) read in libraries (written by us) at runtime and then used them
a segmentation fault occured in my part when allocating memory. I had
just written some arcane stuff concerning binary file i/o and wasn't
surprised that much. Of course it was not possible to use a debugger
and we had to switch to good old printf and visual trace
(step the code in wetware memory line by line) debugging. But I did
not find nothing, everything seemed wonderful -- except that the program
still kept crashing.
It was a weekend and I hadn't had Internet access that time (it's seems so long ago that I remember dinosaurs stamping by my window). So I wrote the first version of the malloc debug library to see what happened. I linked it to my library and -- did not find anything. So at last I linked it to the libraries of my coworkers and indeed there was an off-by-one error in someone else's library. We love C, don't we?
Since then I always use it in my projects from the very beginning and do only switch it off for release versions. It's an unrenouncable improvement to find memory bugs as early as possible without doing any more extra work than including a header and setting a command line switch and adding an object file in the Makefile.
It's a simple thing. The malloc debug library implements wrappers for the normal heap handling functions:
malloccallocreallocfreestrdupgetcwd
(this is thought as an example how to wrap other library functions
returning malloc'd memory which is not longer really necessary since
version 2.00 -- see the Tips and Tricks section
on how to wrap special functions instead)
When allocating memory the wrapper functions demand some more memory from the system and use this extra space to write some special bytefields before and after the buffer they finally return to the user. Some of the extra space before the buffer is used for extra information e.g. about the file position where the allocation took place. When freeing the buffer (and maybe more often) the bytefields are controlled whether they are unchanged. If there is a change the program is aborted immediately with an error message indicating the position where the erroneous buffer was allocated and where the error was detected. E.g. a common error:
<MALLOC_DEBUG> Corrupted block end (possibly written past the end)
should be: a5a5a5a5 5b5b5b5b abababab aa55aa55
is: 00a5a5a5 5b5b5b5b abababab aa55aa55
block was allocated in rtest.c:141 [4 Bytes]
error was detected in rtest.c:143
Looks like string allocated one byte too short
(missing the closing zero)
The abort allows you to switch on your debugger and inspect things more closely.
DISCLAIMER:
This tool is released to
the public domain. You may download/use/change/redistribute this
utility as you like but you do this at your own risk without any
warranty.
This tool is only intended to work on Unix-like operation
systems using ANSI-C. Other systems (e.g. MS Windows) or not ANSI
compatible compilers (esp. preprocessors) are not supported.
Better but beta...
The archive rmalloc200beta.tgz is a gzipped tar archive, has about 50k and contains the files:
Get rmalloc200beta.tgz over HTTP
Get rmalloc200beta.tgz over ftp
Less features but as stable as could be.
The archive rmalloc.tgz is a gzipped tar archive, has about 15k and contains the files:
Get rmalloc.c and rmalloc.h from downloading
rmalloc.tgz from above. Compile rmalloc.c to
rmalloc.o and include rmalloc.h in each of your
files MALLOC_DEBUG defined and link rmalloc.o after
all other libraries (i.e. just before libc) to it.
That's all to use it.
First follow the Installation Guide.
If your sources are compiled with MALLOC_DEBUG the calls
to the standard malloc library are exchanged by the debug malloc library
functions. If you don't define MALLOC_DEBUG the calls stay
as they are. Be sure that you have included rmalloc.h everywhere!
Also be sure that you recompile everything after adding or removing
MALLOC_DEBUG in your Makefile.
If using the TRACK_EXTERNAL_CALLS
feature (which is new since version 2.00 and on by default), the system's
malloc library ist overlayed by a special version donated by
The package includes a Makefile and a file rtest.c. Just say make and you can run rtest which should give you some output similar to this:
<MALLOC_STATS> ============ STATISTICS (rtest.c:339) =============
<MALLOC_STATS> Nothing allocated.
<MALLOC_STATS> ============= END OF STATISTICS =============
------------------
Running test 0...
------------------
<MALLOC_DEBUG> rmalloc -- malloc wrapper V 2.00beta1
by Rammi <mailto:rammi@quincunx.escape.de>
Info on http://www.escape.de/user/quincunx/rmdebug.html
Compiled with following options:
testing: only actual block
external calls: tracked
ext. errors: don't abort
eloquence: OFF
realloc(0): NOT ALLOWED
free(0): NOT ALLOWED
flags: USED
alignment: 8
pre space: 32
post space: 16
hash tab size: 257
<MALLOC_STATS> ============ STATISTICS (rtest.c:88) =============
<MALLOC_STATS> 1000 x 8 Bytes in rtest.c:82
<MALLOC_STATS> *Variable* 8000 Bytes
<MALLOC_STATS> *Static* 0 Bytes
<MALLOC_STATS> *External* 0 Bytes
<MALLOC_STATS> ---------------------------------------------
<MALLOC_STATS> *TOTAL* 8000 Bytes
<MALLOC_STATS> =============================================
<MALLOC_STATS> *Internal* 1000 traced Blocks
<MALLOC_STATS> = 48000 surplus Bytes
<MALLOC_STATS> ============= END OF STATISTICS =============
<MALLOC_STATS> ============ STATISTICS (rtest.c:94) =============
<MALLOC_STATS> 1000 x 8 Bytes in rtest.c:82
<MALLOC_STATS> 20 x 8 Bytes in rtest.c:91
<MALLOC_STATS> *Variable* 8160 Bytes
<MALLOC_STATS> *Static* 0 Bytes
<MALLOC_STATS> *External* 0 Bytes
<MALLOC_STATS> ---------------------------------------------
<MALLOC_STATS> *TOTAL* 8160 Bytes
<MALLOC_STATS> =============================================
<MALLOC_STATS> *Internal* 1020 traced Blocks
<MALLOC_STATS> = 48960 surplus Bytes
<MALLOC_STATS> ============= END OF STATISTICS =============
<MALLOC_STATS> ============ STATISTICS (rtest.c:103) =============
<MALLOC_STATS> 20 x 8 Bytes in rtest.c:91
<MALLOC_STATS> 1000 x 16 Bytes in rtest.c:98
<MALLOC_STATS> *Variable* 16160 Bytes
<MALLOC_STATS> *Static* 0 Bytes
<MALLOC_STATS> *External* 0 Bytes
<MALLOC_STATS> ---------------------------------------------
<MALLOC_STATS> *TOTAL* 16160 Bytes
<MALLOC_STATS> =============================================
<MALLOC_STATS> *Internal* 1020 traced Blocks
<MALLOC_STATS> = 48960 surplus Bytes
<MALLOC_STATS> ============= END OF STATISTICS =============
<MALLOC_DEBUG> Corrupted block end (possibly written past the end)
should be: a5a5a5a5 5b5b5b5b abababab aa55aa55
is: a5a5a53f 5b5b5b5b abababab aa55aa55
block was allocated in rtest.c:98 [16 Bytes]
error was detected in rtest.c:117
------------------
Running test 1...
------------------
<MALLOC_DEBUG> Corrupted block end (possibly written past the end)
should be: a5a5a5a5 5b5b5b5b abababab aa55aa55
is: 00a5a5a5 5b5b5b5b abababab aa55aa55
block was allocated in rtest.c:142 [3 Bytes]
error was detected in rtest.c:144
Looks like string allocated one byte too short
(missing the null byte)
------------------
Running test 2...
------------------
<MALLOC_DEBUG> Corrupted block end (possibly written past the end)
should be: a5a5a5a5 5b5b5b5b abababab aa55aa55
is: 326c6f6e 67005b5b abababab aa55aa55
block was allocated in rtest.c:161 [2 Bytes]
error was detected in rtest.c:163
Looks somewhat like a too long string,
ending with "2long"
------------------
Running test 3...
------------------
<MALLOC_DEBUG> Corrupted block end (possibly written past the end)
should be: a5a5a5a5 5b5b5b5b abababab aa55aa55
is: 786a0608 5b5b5b5b abababab aa55aa55
block was allocated in rtest.c:181 [4 Bytes]
error was detected in rtest.c:185
First 4 bytes of overwritten memory can be interpreted
as a pointer to a block allocated in:
rtest.c:182 [8 Bytes]
------------------
Running test 4...
------------------
<MALLOC_DEBUG> Double or false delete
Heap adress of block: 0x8056710
Detected in rtest.c:240
Trying identification (may be incorrect!):
Allocated in rtest.c:225 [2 Bytes]
------------------
Running test 5...
------------------
<MALLOC_DEBUG> Double or false delete
Heap adress of block: 0x12345678
Detected in rtest.c:258
------------------------------
All tests passed successfully.
------------------------------
You can tweak the behaviour of the library in two ways:
You can change the overall behaviour by setting different switches in rmalloc.c. Doing this has the advantage that you only have to recompile and relink rmalloc.c without touching anything else.
Set this to one of the following three values: 0, 1, or 2.
With 0 only a minimum type of malloc debugging is included. Each allocated block is only tested when you free it, there is no statistics available. Even if this uses the least memory overhead I never use it.
With 1 you will get a statistic of memory in use when you leave your program or when you use a special macro in your code. Each block is only tested automatically when you free it. You can test every block by using special macros. For me this is the optimum of performance deterioration versus available information. This is the default.
2 is the same as 1 with the addition that every allocated block will be checked on every access of any of the malloc functions. This will really slow down your program. I sometimes use this when I cannot get a grip on an error and want to reduce the place to look for the error.
If defined every call to the malloc library functions is tracked and
controlled. In this case rmalloc.o must be the last
object to be linked after all libraries (but just before libc which
is normally appended by the C compiler). Of course the file position
is only supported for files where you included rmalloc.h
with MALLOC_DEBUG set.
For a release version you have to remove the rmalloc.o file
from the link line. (You will be reminded if you have forgotten to do
so when the first malloc results in the startup text to be printed).
If undefined, only the malloc library calls from the files where you
included rmalloc.h with MALLOC_DEBUG set are
tracked. This can result in problems if memory is allocated in a
prepared and released in a non-prepared file or vice versa.
This option only makes sense if TRACK_EXTERNAL_CALLS is set.
If this option is not set, errors occuring in memory allocated by libraries do not abort the program. You will get an error message but the program will run forth (and the error will occur and maybe destroy the malloc arena). So use with care.
If set all errors will abort the program.
If defined every action of the library (allocating, freeing) will be outputtet. This can really get a lot. When I have a Double or false delete error I pipe the output into a file and look if (and where) the given pointer is already freed before.
Defining this allows the setting of special flags for every allocated block. I always use it but it needs extra heap space. See RM_SET section below for details.
Define this to allow realloc(NULL, ...).
To achieve high portability of my programs I don't allow
this in my programs.
Define this if you don't consider free(NULL) an error.
I have this undefined because I use NULL always as a special
value.
To use this macros you have to make changes to your code. I agree that this
is ugly but you will get something for it. All these macros expand to nothing
when MALLOC_DEBUG is undefined.
Invokes a complete test of all allocated blocks. It has only effect when
RM_TEST_DEPTH is at least set to 1.
Example:
RM_TEST; /* tests ALL memory chunks on heap */
Invokes a complete test of all allocated blocks. Prints out a statistic
of allocated blocks to stderr. It has only effect when
RM_TEST_DEPTH is at least set to 1.
Example:
RM_STAT; /* shows allocated memory */
Sometimes you have a functions which gets a chunk of memory, initializes
it to some default values and returns this to the user. For the malloc
debug library only the place of the allocation counts which makes it
difficult to track down memory leaks becasue all the structures allocated
in a function like that will have the same file position. With this macro
you can set the file position to the position where the macro was invoked.
Example:
struct complicated *cpointer = get_new_complicated_struct();
RM_RETAG(cpointer); /* file pos is now set to here */
Set a special flag for the allocated memory. You have to define
WITH_FLAGS in rmalloc.c or this macro will
have no effect. For the moment there are two flags defined:
static char *buffer = NULL;
static int length = 0;
[...]
if (newlength > length) {
if (buffer == NULL) {
buffer = malloc(length = newlength);
RM_SET(buffer, RM_STATIC); /* this is never freed */
}
else {
buffer = realloc(buffer, length = newlength);
/* RM_SET(buffer, RM_STATIC); not necessary because flags are kept! */
}
if (buffer == NULL) {
error(NOMEM);
}
}
ELOQUENT mode.
strdup sets this flag automatically. Example:
struct numstr {
int number;
char *name;
};
[...]
struct numstr *foo = malloc(sizeof(struct numstr));
foo->name = calloc(1, 2);
foo->name[0] = 'X'; /* sheer nonsense */
RM_SET(foo->name, RM_STRING);
This library was used on several platforms (at least Irix, Solaris, HPUX, Digital Unix, and Linux) by different users. It can help you find common errors like (moderate) overwriting of memory bounds or memory leaks.
It is by no means perfect. Writing to dangling pointers may or may not be found. Long overwrites (more than 16 bytes) will probably cause a crash of the library itself or the underlying malloc routines.
Despite of the warnings above I use it always (except for release versions) and haven't had these problems. But of course I know what I am doing.
Some programs already wrap the malloc library functions with own routines (e.g. Xt uses XtMalloc, XtFree etc.). This is normally to allow easier error handling. It has the disadvantage that the position information given by the library is not very useful because all memory is allocated in the same function.
The following preprocessor trick which needs at least version 2.00 sets
the file position transparently (i.e. without any changes to your code),
the example is given for XtMalloc:
#endif /* !R_MALLOC_H */ line:
#ifdef MALLOC_DEBUG
#ifndef XtMalloc
/*
* This works only if XtMalloc is not a macro itself
*/
# define XtMalloc(x) RM_RETAG(XtMalloc_WRAPPER(x))
# define XtMalloc_WRAPPER XtMalloc
#else /* XtMalloc is defined */
/*
* When XtMalloc is a macro look it up. If it is defined in terms of malloc
* there's nothing to do. If it's defined with another function like
* #define XtMalloc(size) _XtMalloc(size, __FILE__, __LINE__)
* you undefine it and take the above line with the RM_RETAG wrap:
*/
# undef XtMalloc
# define XtMalloc(size) RM_RETAG(_XtMalloc(size, __FILE__, __LINE__))
#endif /* XtMalloc */
#endif /* MALLOC_DEBUG */
This will wrap every XtMalloc call in your code with a
RM_RETAG call which sets the
correct file position for the allocated block.
There are lots of other libraries which do a similar thing (it's a common problem). If you have already used another library and you are content with it, there is possibly no need to switch. If this is your first encounter with something like this you should give it a try.
I haven't had much need to use other malloc debug libraries, of course. I used a very early release of Purify and I have tested electric fence some time ago.
I liked Purify a lot but it is incredible expensive and makes your program slow because every memory access is monitored. So it is probably only used if you really know that you have problems. So if the problems don't show during test phase it's not used. But as we all know the problem will show when we present our program to the customer.
If your boss listens to you tell him to buy Purify anyway (it's not your money and you will be glad to have it when you need it). But until then you should use something else.
Electric fence is freeware, too, but it uses a different concept compared to my stuff. It is programming your machine's MMU to protect the space behind the allocated blocks. If you access that space you will immediately get a segmentation fault. This is neat because you are really lead to the place where you actually write over the bounds. It will even react on reading beyond bounds. And it is fast because it uses hardware. But it uses more heap space (which tends to make it slower on big programs) and it doesn't find all errors when writing over the bounds because of alignment problems. Especially the problem of Test 1 of rtest is not always found.
If you need further information feel free to ask me.
This file is a copy of http://www.escape.de/user/quincunx/rmdebug.html. You will find the latest information there (It is really a copy so maybe you are already there).
A thousand thanks are going to Doug Lea for releasing his malloc library to the public domain. This tool uses a slightly modified version of his library V2.6.5. See his article A Memory Allocator for more information about the arcane science of memory allocation.