I’m a big fan of the Mono.Cecil library for doing reflection (and more!) with .NET. I’ve had issues in the past where the built-in .NET reflection (using Assembly.ReflectionOnlyLoad) attempts to load dependent libraries as you iterate over exposed types, even though it’s not supposed to (unfortunately I don’t have a repro to hand). This makes it very difficult to work on an assembly without having all of its dependencies available. Cecil doesn’t have this problem because it accesses the assembly in a lower-level way.

I downloaded and installed Mono (the latest version is 2.8) and referenced it from the installed Mono GAC location. Then it was just a matter of a handful of lines of F# (after a bit of spelunking through the Cecil API to find the methods relating to assembly references).

#r @"C:\Program Files (x86)\Mono-2.8\lib\mono\gac\Mono.Cecil\0.6.9.0__0738eb9f132ed756\Mono.Cecil.dll"
#r @"gachelper.dll"
 
open System
open System.IO
open Mono.Cecil
 
let getReferencedAssemblies (asm : string) : string list =
    let ad = AssemblyFactory.GetAssembly asm
    let refs = ad.MainModule.AssemblyReferences
    refs 
    |> Seq.cast 
    |> Seq.fold (fun found (r : AssemblyNameReference) -> 
        let fullPath = ref ""
        match gachelper.GAC.TryGetFullPath (r.Name, fullPath) with
        | false -> found @ [ Path.Combine [|(System.IO.Path.GetDirectoryName asm); r.Name + ".dll"|] ]
        | true -> found @ [!fullPath]        
        ) []

As you can see I’ve also used a little C++/CLI wrapper around the GAC to get access to full paths of installed assemblies (which only works with the Microsoft CLR), but I’ll talk about that in a separate post, or you can grab the code here.

Now, we can call our function to get the set of dependencies, e.g. by using F# interactive, we also happen to be getting the dependencies *of* FSI:

> getReferencedAssemblies @"C:\Program Files (x86)\FSharp-2.0.0.0\bin\fsi.exe";;
val it : string list =
  ["C:\Windows\Microsoft.Net\assembly\GAC_32\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll";
   "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\FSharp.Core\v4.0_4.0.0.0__b03f5f7f11d50a3a\FSharp.Core.dll";
   "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\FSharp.Compiler\v4.0_4.0.0.0__b03f5f7f11d50a3a\FSharp.Compiler.dll";
   "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Windows.Forms\v4.0_4.0.0.0__b77a5c561934e089\System.Windows.Forms.dll";
   "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\FSharp.Compiler.Interactive.Settings\v4.0_4.0.0.0__b03f5f7f11d50a3a\FSharp.Compiler.Interactive.Settings.dll";
   "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\FSharp.Compiler.Server.Shared\v4.0_4.0.0.0__b03f5f7f11d50a3a\FSharp.Compiler.Server.Shared.dll";
   "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll"]

This is, of course, the tip of the iceberg in terms of Cecil functionality; there are lots of far more interesting things you can do – like IL extraction and re-writing – things which are impossible to do with the Microsoft reflection API.

I ended up using this code to generate DGML graphs that can be opened and explored using VS2010. This functionality comes “in the box” with VS2010 Architecture Explorer in the Ultimate Edition – but who can afford that…? We can do much the same ourselves by just using the information we get from Cecil and spitting out DGML directly. The files can be opened read-only in the Premium edition for perusal.

For the curious, here’s the (somewhat fugly and imperative) F# code to generate the DGML file. It creates a minimal file that only contains Link elements; Visual Studio will “fill in the blanks” by adding the nodes themselves when you open the file.

let genDgml asm (out : string) =
    let doc = XmlDocument()
    let nsURI = "http://schemas.microsoft.com/vs/2009/dgml"
    let nsmgr = XmlNamespaceManager(doc.NameTable)
    nsmgr.AddNamespace("", nsURI)
    let root = doc.CreateElement("DirectedGraph", nsURI)
    let links = doc.CreateElement("Links", nsURI)
    let rec genRefs added asm =
        getReferencedAssemblies asm
        |> List.fold (fun added dep -> 
            let link = doc.CreateElement("Link", nsURI)
            link.SetAttribute("Source", Path.GetFileNameWithoutExtension asm)
            link.SetAttribute("Target", Path.GetFileNameWithoutExtension dep)
            ignore <| links.AppendChild link
            if not <| (added |> Map.containsKey dep)
            then genRefs (added.Add (dep, false)) dep
            else added
        ) added
    ignore <| genRefs Map.empty asm
    ignore <| root.AppendChild links
    ignore <| doc.AppendChild root
    doc.Save out

The DGML in cluster view

It can be quite enlightening, as well as useful, to see your app dependencies laid-bare before you…

This is such a common source of bugs, and such an important requirement, that from Vista onwards Microsoft introduced a new set of functions in the Windows API explicitly to support it: One-Time Initialization.

And even if you get away with doing naughty things in DllMain now, don’t think that it’ll stay that way forever. We got away with it for years, then when .NET came along it introduced all sorts of additional correctness checks. For instance, the Managed Debugging Assistant (MDA) in Visual Studio will shout loudly should you attempt to run managed code during DllMain.

Managed Debugging Assistant 'LoaderLock' has detected a problem in 'C:\YourApp.vshost.exe'.
Additional Information: Attempting managed execution inside OS Loader lock. Do not attempt to run managed code inside a DllMain or image initialization function since doing so can cause the application to hang.

And it’s easier than you think to do so. For example, calling something as innocuous as GetWindowText can result in managed code being run.

How can you get around this?

One of the approaches I’ve used has been to make use of Win32 asynchronous procedure calls. Specifically you can call QueueUserAPC to add a function to the queue, and this can contain the initialisation you would’ve otherwise done in DllMain.

However there is significant gotcha regarding use of APCs: your function will not be called until the thread is in an “alertable wait state”. This means you (or some other code on the thread) need to call an alertable wait function such as SleepEx, WaitForSingleObjectEx and specify TRUE for the alertable parameter.

Once your APC is getting called successfully you’ll be in much better place; your code will be executed outside of the scope of the dreaded OS loader lock and you’ll be doing things by the book, hopefully avoiding all the potential pitfalls that lie in wait within DllMain.

To spoil the ending somewhat, we already knew the precise situation that was causing the problem. This is different to the average issue, where most of the time’s spent in isolating the conditions that cause it. In this case we had an excellent positive and negative repro case.

The users were attempting to use the legacy .local approach to changing DLL probing. This was originally implemented around the time of Windows 2000, before the current side-by-side system existed, and basically short-circuited the normal DLL search order to inject the current directory as the first potential location. See the DLL redirection MSDN page for more information.

There was already a version of the assembly in question installed in the GAC, they just wanted to use the .local file to override it. The trouble was, with certain builds of the DLL (we’ll call it XYZ.dll) in the directory along with the redirection file (appname.exe.local), an fatal InvalidProgramException would be generated by the .NET execution engine. With other builds of the same DLL it worked. Mysterious.

So the first thing to do was to try and find out exactly where the exception was occurring. I ramped up WinDbg, and took a look at what exceptions were being thrown:

(42c.1e90): CLR notification exception - code e0444143 (first chance)
(42c.1e90): Unknown exception - code 02345678 (first chance)
(42c.1e90): C++ EH exception - code e06d7363 (first chance)
(42c.1e90): C++ EH exception - code e06d7363 (first chance)
(42c.1e90): C++ EH exception - code e06d7363 (first chance)
(42c.1e90): CLR exception - code e0434f4d (first chance)

The last one generated a second chance exception and the exit. I changed the exception filter set to stop on .NET notifications – which are generated when assemblies are loaded – and for C++ exceptions:

0:000> sxe clrn
0:000> sxe eh

After restarting and running on from the CLR notification (generated whenever an assembly is loaded) I got a stack trace on the C++ exception that was being generated within the CLR:

0:000> kP
ChildEBP RetAddr
0012eea4 78158e89 KERNEL32!RaiseException+0x53
0012eedc 79fce58e MSVCR80!_CxxThrowException+0x46
0012ef0c 7a015309 mscorwks!RealCOMPlusThrow+0xd8
0012f2ac 79f0e795 mscorwks!UnsafeJitFunction+0x365
0012f350 79e87f52 mscorwks!MethodDesc::MakeJitWorker+0x1c1
0012f3a8 79e8809e mscorwks!MethodDesc::DoPrestub+0x486
0012f3f8 00341f3e mscorwks!PreStubWorker+0xeb

Hmmmm, interesting, so it looks like the JIT compiler is failing. I also noticed that there was a strange exception that I didn’t recognise:

(42c.1e90): Unknown exception - code 02345678 (first chance)

You’ve got to love those “made up” numbers. I added an exception filter for it and restarted again (good job this failure was quick to manifest).

Now I got an even more precise fault location:

0:000> kP
ChildEBP RetAddr
0012ec44 790af56c KERNEL32!RaiseException+0x53
0012ec60 7909deea mscorjit!getJit+0x38
0012ec7c 79066362 mscorjit!Compiler::fgFindJumpTargets+0x33d
0012ece0 790614c6 mscorjit!Compiler::fgFindBasicBlocks+0x4d
0012ed20 79061236 mscorjit!Compiler::compCompile+0x2bf
0012edb4 7906118c mscorjit!jitNativeCode+0xb8
0012edec 79f0f9cf mscorjit!CILJit::compileMethod+0x3d
0012ee58 79f0f945 mscorwks!invokeCompileMethodHelper+0x72
0012ee9c 79f0f8da mscorwks!invokeCompileMethod+0x31
0012eef4 79f0ea33 mscorwks!CallCompileMethodWithSEHWrapper+0x84
0012f2ac 79f0e795 mscorwks!UnsafeJitFunction+0x230
0012f350 79e87f52 mscorwks!MethodDesc::MakeJitWorker+0x1c1
0012f3a8 79e8809e mscorwks!MethodDesc::DoPrestub+0x486
0012f3f8 00341f3e mscorwks!PreStubWorker+0xeb

So now its looking like something’s going wrong deep down in the JITter. After stepping through the disassembly for a while I decided to have a look at precisely what method was being compiled at this point. You can use the DumpMD command from the sos extension to examine the method, which is the first parameter to UnsafeJitFunction (you can see parameters in the stack trace by using kb):

0:000> !DumpMD 0161ad08
Method Name: XXX()
Class: 01670a40
MethodTable: 0161ad18
mdToken: 0601626b
Module: 00d3323c
IsJitted: no
m_CodeOrIL: ffffffff

Now this is suspicious; the method as listed (XXX – name changed to protect the innocent) isn’t what we were expecting. In fact, it’s completely different. Let’s see what IL is being JITted:

0:000> !DumpIL 161ad08
error decoding IL

OK, so this is completely broken metadata.

Now, given that we were using the .local, we would have expected to see the DLL being loaded from the same directory as the application, so lets take a look and see what we’ve got loaded:

0:000> !DumpDomain
--------------------------------------
System Domain: 7a3bc8b8
...snip...
Assembly: 001abe28 [C:\WINNT\assembly\GAC_MSIL\XYZ.DLL]
ClassLoader: 001aa0b0
SecurityDescriptor: 001aa018
Module Name
00d3323c C:\WINNT\assembly\GAC_MSIL\XYZ.DLL

So this looks like the culprit; we’re still loading the version from the GAC, even though there’s a .local file present. Just to verify this I set a breakpoint on CreateFileW to show me which DLLs were being loaded and by what:

0:000> bu KERNEL32!CreateFileW "dpu (@esp+4) L1"

This tells the debugger to display the unicode string that is the first parameter to CreateFileW whenever it’s hit.
There are obviously lots of DLLs loaded, even in this simple case, but eventually it led me to the point where the GAC version was being loaded:

0012cf44 001ac5a0 "C:\WINNT\assembly\GAC_MSIL\XYZ.dll"
eax=00000000 ebx=00000001 ecx=79e7f683 edx=7c90eb94 esi=001ac4e0 edi=7c80ac0f
eip=7c810760 esp=0012cf40 ebp=0012cf64 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
KERNEL32!CreateFileW:
7c810760 8bff mov edi,edi

0:000> kp L30
ChildEBP RetAddr
0012cf3c 79e90f12 KERNEL32!CreateFileW
0012cf64 79f29306 mscorwks!WszCreateFile+0x72
0012cfac 79ed27ac mscorwks!PEImage::GetFileHandle+0x60
0012cfdc 79e981ae mscorwks!PEImage::GetLayoutInternal+0xcf
0012d020 79e98e69 mscorwks!PEImage::GetLayout+0x8a
0012d0bc 79e98934 mscorwks!RuntimeOpenImageInternal+0x103
0012d104 79e988a9 mscorwks!GetAssemblyMDInternalImportEx+0x9d
0012d11c 79ec314d mscorwks!CreateMetaDataImport+0x16
0012d13c 79ec30ee mscorwks!CAssemblyManifestImport::Init+0x35
0012d164 79ed39ca mscorwks!CreateAssemblyManifestImport+0x53
0012d180 79ed3943 mscorwks!CreateAssemblyFromManifestFile+0x48
0012d3e8 79ed380f mscorwks!CheckExistsInGAC+0x179
0012d418 79ed320a mscorwks!CreateAssemblyFromCacheLookup+0x9b
...snip...

So then I waited until the exception was fired, and used !dlls to show me what other DLLs were loaded:

0:000> !dlls
...
0x00243a68: C:\Temp\interopx\XYZ.dll
Base 0x036a0000 EntryPoint 0x00000000 Size 0x0095c000
Flags 0x90404000 LoadCount 0x00000001 TlsIndex 0x00000000
LDRP_ENTRY_PROCESSED
LDRP_COR_IMAGE
...

Ouch. We’ve loaded some metadata from the c:\winnt\assembly\GAC_MSIL\XYZ.dll version, and then we’ve actually loaded the c:\temp\interopx\XYZ.dll module. It turned out that in some cases the metadata matched, and we got away with it, and at other times (for different builds) it didn’t and we hit the InvalidProgramException.

So, the moral of the story is, don’t mix .local and the .NET assembly loader. And if you do, prepare to break out your WinDbg skills.

Why? Well, because it’s too easy to use them as an excuse for not defining your shared library interfaces properly.

The reason this is on my mind recently is that several hundred, yes, you heard that right, several hundred DLLs have been released by my group over the last, ooh, 10 years or so. They are all still in use. Each of them has burned into it a copy of the library that deals with interfacing with Excel. That means each of these has it’s own little internal copy of the current state-of-the-art. The problem with that is; the state-of-the-art moves on. And how do you go about updating the DLLs that are already in production? You have to re-release them. In an environment where thes DLLs are used for marking the profit and loss on a large derivatives trading book, that’s not a small undertaking. And it’s made worse if, say the DLL in question was last released with a different version of the compiler.

My approach would be to refactor this shared static library (.lib) into a stand-alone DLL.

At this point, people start saying “oh, but then you’ve got a single point of failure, if you release a broken version of that DLL, everything will stop working!”. Not exactly a compelling argument. If the functionality of the DLL is well defined, and there are well known entry points it should be easy to put together a comprehensive black-box test suite. In fact we already do that with all our other DLLs (COM servers). The fact that this shared library *isn’t* a DLL has meant that it’s fallen through the testing cracks; another good reason to refactor it.

The internal interface to the shared library is already relatively well defined. It has a set of header files that define all of the functions and classes that are consumed by others. It’s a relatively small step to compile it as a DLL, rather than a static library. The problem then becomes one of maintenance, dealing with the inevitable changes to the external interface in a backwardly compatible way.

And that’s the problem. It requires some effort. Elsewhere in our codebase we use COM as a magic cure-all for avoiding having to deal with versioning: interface immutability rules. All interfaces are public, no published interface ever changes, object identity is based purely on interfaces supported. If you haven’t got these crutches to rely on, then you have to enforce the rules yourself, which can be both logistically and technically difficult when you’re dealing with C++.

But it’s not impossible. And I really think it would be better than having hundreds of DLLs all containing subtley different versions of the same code, and being unable to change behaviour across the board without having to build, test and release them all.

Maybe you’ve got a different opinion?