C Developer in a FreeBSD World (Part 2)

In the “The Journey of a C developer in a FreeBSD World”, I described the changes that occur when you land in a BSD system coming from Linux. Now you, dear readers, get ready to get your C/C++ code working in both platforms; this time we will look into the debugging side. Indeed, FreeBSD’s libc has jemalloc builtin. OpenBSD contains its specific implementation, called ottomalloc.


As a C/C++ developer, you have concerns about memory leakage, corrupted memory. In the previous issue, the article “GDB debugger” perfectly described its proper usage. Its reading is greatly recommended. We’ll focus on the memory allocators.

In OpenBSD, several options are available via the MALLOC_OPTIONS environment variable or the global malloc_ options variable changeable from within your C/C++ code. To enable a specific option, it is the uppercase letter. To disable it, it is the lowercase counterpart.

The malloc statistics are disabled by default, for performance matters. In order to enable these, the OpenBSD source code is needed and we just need to uncomment this line in lib/libc/stdlib/malloc.c:

/* #define MALLOC_STATS */

Then we can just recompile the libc:

cd lib/libc
> make obj && make depend && make install

Now, malloc_dump symbol is available!

> nm /usr/lib/libc.a | grep malloc_dump
00000790 T malloc_dump

One I find quite useful is the junk option (enabled by default since 5.6 release).

Indeed, after an allocation, the memory area is filled with 0xd0. When it is freed, it is filled with 0xdf. What you can spot easily is when you try to use a previously freed memory pointer.

int
main(int argc, char *argv[])
{
char *p = new char[4];
strlcpy(p, “foo”, 4);
std::cout << p << std::endl;
delete p;
std::cout << p << std::endl;
return (0);
}
…

You ought to see this kind of output:

> ./test
 foo
 XXXXXXXXXXXX
 => we have our filled free pointer ...
 …

 …
 > env MALLOC_OPTIONS=j ./test => let’s disable the junk
 option
 foo
 foo
 => not good at all, we can believe the p pointer is still
 valid at this point ...
 …

As you can see above, the junk option is really useful and the performance hit is quite acceptable so it is quite advised to keep this option on, even in production.

The Freeguard option, F, is useful for detecting double free.

int
 main(int argc, char *argv[])
 {
 ...

 char *p = malloc(sizeof(char) * count);
 …
 free(p);
 …
 <i.e no realloc meanwhile …>
 …
 free(p);
 return (0);
 }
 > ./test
 => The double free not caught ...


 > env MALLOC_OPTIONS=F ./test
 test(7086) in free(): error: bogus pointer (double free?)
 0x7820972ff40
 Abort trap (core dumped)
 => The double free is caught

Another flag useful for debugging, A (for abort, enabled by default) which simply coredumps the current process.

Previously, we compiled the libc to enable the malloc statistics, hence to enable D (for Dump) statistics. So compiled with debug symbols:

...

 int
 main(int argc, char *argv[])
 {
 char *p = malloc(4096);
 malloc_dump(2);
 return (0);
 }

 > ./test
 => from within the code, malloc_dump prints on the given
 file descriptor those statistics ...
 Malloc dir of test at 0x11d6c987f3d0
 Region slots free 511/512
 Finds 0/0
 Inserts 1/0
 Deletes 0/0
 Cheap reallocs 0/0
 Free chunk structs:
 Free pages cached: 0
 slot) hash d type page f
 size [free/n]
 65) # 65 0 pages 0x11d6a9f84000 0x11d6aa3ed86d 4096
 In use 16384
 Guarded 0
 Leak report
 f sum # avg
 0x11d6aa3ed86d 4096 1 4096

Here we got the faulty address, 0x11d6aa3ed86d, when the pointer was allocated but never freed.

Or we can call malloc_dump from within gdb

…
> gdb ./test
After putting a breakpoint to exit, we call call malloc_
dump to print on stderr

malloc_dump(2)
…
0xfa011001270 4096 1 4096
…
list *0xfa011001270

We retrieve our faulty code here:

int
main(int argc, char *argv[])
{
char *p = malloc(4096);
malloc_dump(2);
return (0);
}

Apart of MALLOC_OPTIONS, OpenBSD protects well against stack overflow’s issues. The stack protector flag is implied; there’s no need to add it for the compilation.

int
main(int argc, char *argv[])
{
char buf[4];
strcpy(buf, “foobar”);
printf(“%sn”, buf);
}…
> cc -g -O2 -o test test.c
/tmp/ccPGztFB.o(.text+0x1db): In function ‘main’:
: warning: strcpy() is almost misused, please use
strlcpy()
…
> ./test
foobar
Abort trap (core dumped)

When possible, it is advised to use strlcpy (the compiler is nice enough to warn you about that), except if you’re 100% confident about the source you attempt to copy.

int
main(int argc, char *argv[])
{
char buf[4];
strlcpy(buf, “foobar”, 7);
printf(“%sn”, buf);
}…
> cc -g -O2 -o test test.c
test.c: In function ‘int main()’:
test.c:9 warning: array size (4) smaller than bound length (7)
=> you have been warned that your buffer is really too
small ...

For FreeBSD, it is slightly different but we can retrieve similar options with jemalloc, like junk.

As a developer, you might need to make sure that MALLOC_PRODUCTION is not defined in either /etc/src.conf and /etc/make.conf. Although it brings significant performance improvements, the debugging capabilities are lost.

...


int
main(int argc, char *argv[])
{
char *p = new char[4];
strlcpy(p, “foo”, 4);
std::cout << p << std::endl;
delete p;
std::cout << p << std::endl;
return (0);
}

> ./test
foo
ZZZZZZZZZZ
…
> setenv MALLOC_CONF “junk:false”
> ./test
foo
foo

It is possible to dump statistics via stats_print options:

> setenv MALLOC_CONF “stats_print:true”
> ./test
…
___ Begin jemalloc statistics ___
Version: 3.6.0-0-g46c0af68bd248b04df75e4f92d5fb804c3d75340
Assertions enabled
Run-time option settings:
opt.abort: true
opt.lg_chunk: 22
opt.dss: “secondary”
opt.narenas: 32
opt.lg_dirty_mult: 3
opt.stats_print: true
opt.junk: true
opt.quarantine: 0
opt.redzone: false
opt.zero: false
opt.utrace: false
opt.xmalloc: false
opt.tcache: true
opt.lg_tcache_max: 15
CPUs: 8
Arenas: 32
Pointer size: 8
Quantum size: 16
Page size: 4096
Min active:dirty page ratio per arena: 8:1
Maximum thread-cached size class: 32768
Chunk size: 4194304 (2^22)
Allocated: 4096, active: 4096, mapped: 8388608
Current active ceiling: 4194304
chunks: nchunks highchunks curchunks
2 2 2
huge: nmalloc ndalloc allocated
0 0 0

arenas[0]:
assigned threads: 1
dss allocation precedence: secondary
dirty pages: 1:0 active:dirty, 0 sweeps, 0 madvises, 0
purged
allocated nmalloc ndalloc nrequests
small: 0 0 0 0
large: 4096 1 0 1
total: 4096 1 0 1
active: 4096
mapped: 4194304
bins: bin size regs pgs allocated nmalloc
ndalloc nrequests nfills nflushes
newruns reruns curruns
[0..27]
large: size pages nmalloc ndalloc nrequests curruns
4096 1 1 0 1 1
[1017]
--- End jemalloc statistics ---

or from within the code:

#include <malloc_np.h>

…
void
m_stats(void *args __unused, const char *data)
{
if (data != NULL)
printf(“%sn”);
}
...
int
main(int argc, char *argv[])
{
char *p = malloc(4096);
/* could simply make the first argument as NULL */
malloc_stats_print(m_stats, NULL, NULL);
return (0);
}

With the utrace option, it adds an entry for ktrace…

> setenv MALLOC_CONF “utrace:true”
> ktrace ./test
> kdump
…
1245 test CALL utrace(0x7fffffffeaa0,0x18)
1245 test USER 0x801006000 = malloc(4096)
1245 test RET utrace 0
1245 test CALL utrace(0x7fffffffeaa8,0x18)
1245 test USER free(0x801006000)

…

As you can see without any third party tool, in FreeBSD/OpenBSD we have those features which greatly help with debugging. Even if the price to pay is having lower performance results, it is worth it during the development at least.


 

About The Author

David Carlier has been a developer since 2001, has been using BSD since 2004 and has worked for a mobile based position as a C/C++ developer in Ireland since 2012. During his spare time, he contributes to various BSD projects, especially FreeBSD, and writes some articles for BSDMag.

 

ESG Labs: TrueNAS Technical Report
Download Enterprise Storage Guide Button
iXsystems values privacy for all visitors. Learn more about how we use cookies and how you can control them by reading our Privacy Policy.