Collecting stack traces

One of the most helpful things the heap manager can do for you when investigating memory issues is to capture stack traces for each heap allocation. You can enable the “collect stack traces” heap flag using the gflags GUI or from within WinDbg:

0:006> !gflag +ust
Current NtGlobalFlag contents: 0x00001040
    hpc - Enable heap parameter checking
    ust - Create user mode stack trace database

This means that for each heap block (one located at 0x0bbf7308 in this case), you can see where it was allocated by using the -a (show all information) option:

0:006> !heap -p -a 0bbf7308
    address 0bbf7308 found in
    _HEAP @ a630000
      HEAP_ENTRY Size Prev Flags    UserPtr UserSize - state
        0bbf7308 0073 0000  [07]   0bbf7310    00380 - (busy)
        Trace: 401c
        7c96d6dc ntdll!RtlDebugAllocateHeap+0x000000e1
        7c949d18 ntdll!RtlAllocateHeapSlowly+0x00000044
        7c91b298 ntdll!RtlAllocateHeap+0x00000e64
        78134333 MSVCR80!malloc+0x00000077

But the obvious problem with this is that the stack trace always stops at malloc. Something’s allocating some memory? You don’t say…

It turns out that this is a well understood and documented issue with the Microsoft VC++ runtime, variously known as msvcrt, msvcr70, msvcr71, msvcr80, msvcr90, etc. Unfortunately they’re all built using the stack frame pointer omission optimisation. Well they’re built with the -O1 (favour small code) option, which implies -Oy. This means that the fast stack-walking algorithm the heap manager uses stops at functions without a return address. The only way to get a decent trace in this situation would be to use the DbgHelp API along with the .pdb files, which would be far too slow to do at each allocation site.

“Fixing” it

So, given that the source for the runtime library ships as part of Visual Studio, maybe it would be possible to build it without the -Oy option?

My first attempt at building it failed miserably with errors like:

NMAKE : fatal error U1073: don't know how to make 'build\intel\mt_obj\startup.lib'

Luckily this excellent page helped me get past this to a point where I could actually get a DLL built.

The next stage was to modify the build scripts to use different compiler switches. This was simply a case of changing line 69 of makefile.sub from:
CFLAGS=$(CFLAGS) -O1
to:
CFLAGS=$(CFLAGS) -O1 -Oy-

I thought I may have to also modify the build scripts to output a version of the DLL with the same name as the file I was replacing, msvcr80.dll, directly, in case there were internal references to the name in embedded manifests. There’s a section at the top of the build script for choosing a name for your private version of the library, but it strongly discourages use of the “reserved” MSVC* names. Luckily it turns out not to be necessary; the DLL is constructed in such a way as to be rename-able without any ill effects. I could build _sample_.dll (the default output name) and simply copy it to the destination directory in the SxS tree (C:\WINNT\WinSxS\x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.3053_x-ww_b80fa8ca) and rename it.

Result

Now I get the expected full stack trace (names have been changed to protect the innocent):

0:006> !heap -p -a 0bbf7308
    address 0bbf7308 found in
    _HEAP @ a630000
      HEAP_ENTRY Size Prev Flags    UserPtr UserSize - state
        0bbf7308 0073 0000  [07]   0bbf7310    00380 - (busy)
        Trace: 401c
        7c96d6dc ntdll!RtlDebugAllocateHeap+0x000000e1
        7c949d18 ntdll!RtlAllocateHeapSlowly+0x00000044
        7c91b298 ntdll!RtlAllocateHeap+0x00000e64
        78134333 MSVCR80!malloc+0x00000077
        7816207f MSVCR80!operator new+0x0000001d
        fa92336 leakydll!std::allocator,std::allocator > > >::allocate+0x00000016
        fa9879b leakydll!std::vector >::resize+0x0000005b
        ...
        ...etc...

That’ll make it much easier to work out what’s happening and who’s responsible. Of course, you should be careful with this modified version. Only use it on development machines, and make sure it doesn’t escape into the wild: with great power comes great responsibility.

Private data

This is the data that is explicitly allocated by the process, or blocks of memory that contain the allocated data. So when you’re allocating from the heap, for instance, using code like:

char *p = new char[1024*1024];

you can see how the heap manager manages address space; creating reserved segments of a fixed size, then committing parts of them as required.
When a segment is full, it will create a new one of double the size.

Allocate first 256*1024 bytes:

After allocating first 256KB

After allocating another 256KB

And another...

And yet another...

Causes of out of memory errors

Fragmentation

If you’re experiencing out of memory errors, the first thing to check is the largest free block size. Obviously if you’re requesting more than the available size, your allocation will fail. Remember that large requests may be made implicitly by mechanisms that are outside your control. For instance, Windows heaps expand geometrically, attempting to grab a segment of double the previous size each time, e.g. 16, 32, 64, 128MB. As you can see, allocating a single byte when the 64MB segment is full will result in the heap manager trying to reserve a 128MB block.

So once you’ve seen that the largest free block is small, the question then is: why? It’s either due to genuine memory exhaustion or address space fragmentation.

Heap contention

Fragmentation can be due to multiple heaps contending for contiguous address space. This is typically the case when you have a mix of VC++ code that is built using different versions of the runtime libraries (msvcrt.dll, msvcr70.dll, msvcr80.dll, msvcr90.dll), each of which manages it’s own heap.

Remember that heap segments need to be reserved as a contiguous block of address space, but if your virtual memory is fragmented, the heap manager may not be able to obtain one. In this case, it will fall-back to creating segments of smaller sizes. The problem is that there’s a limit to the number of segments that can be created – a measly 32 in Windows XP – so if fragmentation causes it to create more, smaller segments this limit may be reached. If a new segment cannot be created you’ll get out of memory errors.

If you suspect this, you can use !heap -m to see details of the segment count and sizes for each heap. To identity the heap associated with each version of the MSVC runtime, ensure you’ve got the appropriate symbols loaded and then use dd msvcr80!_crtheap L1 to see the address.

Address space exhaustion

It may also be possible that you really have exhausted your 2GB address space. This can happen when your process wastes lots of address space due to allocation granularity. For example calling VirtualAlloc without an address will cause the OS to choose one for you, and as the documentation states, this will be rounded to a multiple of the allocation granularity, 64KB. So if you happen to allocate lots of objects of only a few bytes with direct calls to VirtualAlloc, you will waste almost 64KB a time. Although this might not seem significant in a 2GB address space, it all adds up.

One of the symptoms of address space exhaustion is DLLs failing to load. I noticed recently that a COM CoCreateInstance call was failing because the only address space left to load the DLL into was way up into the area usually reserved for OS DLLs such as ntdll.dll.

Other tips

By default the allocations are ordered by Address (the first column) and, because things are generally allocated in increasingly higher locations in memory, this can serve as a useful “timeline” of the app’s allocations. It’s not guaranteed though: DLLs that have an explicit load address don’t follow this pattern (for example, VBE6.DLL always loads at 0×65000000). You can use it to see roughly when threads and heaps are created and files are mapped though.

Summary

So, I hope you find this information useful in interpreting the output from VMMap. It’s a very good way of getting visibility on the state of your processes and it’s certainly more intuitive than having to use the !address and !heap commands in WinDbg.

Good hunting!

Like most of my posts, this assumes that it’s not feasible to go through all your source code, and say, replace all instances of new with a version that tracks usage (the approach used by the debug CRT). As well as being logistically infeasible, this also tends to miss allocations that don’t go via new, for example, direct calls to HeapAlloc.

In the past, I’ve seen some code trying to use the Win32 heap functions to try and find out the amount of memory allocated by the process. It used GetProcessHeaps, HeapWalk and HeapSize to sum all the block sizes and get an overall memory in use figure, but in my experience it was extremely slow and unreliable.

What was really required was something that gave a figure similar to the “private bytes” counter in perfmon. If you didn’t know, this is the counter you need to be watching if you’re looking for memory leaks in a process. For goodness sake don’t use the “Mem Usage” column in Task Manager; this is in fact (almost) the working set size and it doesn’t correlate exactly with memory explicitly allocated by the process. It includes additional things including space occupied by the loaded DLLs. Also, the working set will shrink if the app is paged out, although it still has the memory allocated. To see an example of this in action, open Excel and a large spreadsheet, calc it, look in Task manager and you’ll see a large number (if not, you’re obviously not looking at a real spreadsheet). Then minimise the Excel window. You’ll see the mem usage value plummet as the working set is ”trimmed” – probably by a call to SetProcessWorkingSetSize. The OS does this because it expects the app won’t be being used, so it makes sense to free up physical memory for use by other processes.

So essentially what I want to do is get the perfmon “private bytes” value programmatically as my app is running, and this can be achieved using the Performance Data Helper (PDH) library. It provides an API to access the performance counters in a similar way to the perfmon GUI.

It uses the concept of “queries”; you create a query, add a counter to it, then collect the query data as required (not forgetting to remove the counter and close the query when you’re done).

The first thing to do is open the query:

Then add the required counters (this code assumes you’re looking at a process on the current machine):

At this point you’re ready to start polling for updates. At periodic intervals you can collect the query data and do with it what you will:

Luckily for the Private Bytes counter we’ve got the simplest type of counter to ‘decode’; a raw counter value, essentially just a number. We don’t need to do any further manipulation on it to get the information we need, like having to divide by some frequency.