Simples?

Of course the idiomatic way to avoid problems at the boundary between your F# code and the outside world is to ensure that you only expose a small set of compatible types. This works pretty well if your clients are also .NET languages. For instance you can do things like exposing your collections as seq, rather than say, a native F# list, and this will mean your collections can be consumed as IEnumerable. The only problem is it means you’ve got the added burden of maintaining this mapping layer, because you’ll no doubt want to use the F# “native” types internally.

So, what options do we have if some of our F# types happen to leak into our public API? Luckily, lots. Let’s take a look at how some of the common F# constructs can be called from C#.

Unions

I talked about the power of discriminated unions a while back, so how do they map to C#? Interestingly the F# compiler generates a very simple OO type hierarchy, with an abstract base class that each of the cases derive from. This will mean that your calling code will need to use some casting, but don’t worry, you’re not back in the hell of runtime typing; there are a set of methods on the type that you can use to identify the case before you do the cast, so at least you get some compile time sanity checking.

Here’s a simple discriminated union that we’ll be using as an example:

    type TestDU =
        | One of int * int
        | Two of string
        | Three

If we wanted to use this type in F#, we’d do so like this:

    // A function 'f' that takes an instance 't' of our union type
    let f t = match t with One (a,b) -> printf "Got One" | _ -> ()
 
    // Apply our function with a particular TestDU union case
    let _ = f <| Two "foo"

That’s pretty natural, as you’d expect. If we exposed the function f to C#, it looks like this:

void Module1.f(Module1.TestDU t)

This means we have to create an instance of the TestDU type, but we’ll quickly discover it’s abstract. As I mentioned before, the F# compiler has generated a very simple class hierarchy, with the union cases being subtypes of TestDU, implemented as nested types (which are used quite extensively by the F# compiler). But if we try and instantiate one of these subtypes…?

The union case types have no constructors.

    var one = Module1.TestDU.NewOne(1, 2);
    var two = Module1.TestDU.NewTwo("foo");

And slightly oddly, the parameterless cases (Three in our example) use a property on the base class instead:

    var three = Module1.TestDU.Three;

So now we can create instances of the union type to pass to the F# function.

But what if we’re returned one instead? It will be of type TestDU, so how do we know what case it actually is? And how do we get to the parameters? Well, as we know it’s going to be one of the subtypes, we could use the C# ‘is’ operator to determine its identity, but the base class also provides helper methods to do it for us, prefixed with Is, e.g.:

    if (two.IsTwo)
    {
        var x = (two as Module1.TestDU.Two).Item;
        // Do something with x
    }

As you can see, Item contains the case value. For tupled types this becomes Item1, Item2 etc.

Records

Records are immutable types that contain fields, they tend to be used quite a lot in F# code, and luckily using them from C# is very straightforward. The only thing to remember is that they’re only constructor is the one taking all of the field values (because they’re immutable), e.g.:

    type TestRecord =
        {
        field1 : int
        field2 : string
        }

Can be instantiated like this in C#:

    var r = new Module1.TestRecord(1, "foo");

Functions

It’s usual in F# to use curried functions. That is, functions that can be applied one argument at a time, returning a partially applied function. It turns out that the compiler maps this type of function (when directly exposed, as a public function on a module, say) into its tupled equivalent, which makes them easily callable directly from C#. For instance:

let public adder a b = a + b

Has a native F# signature of int -> int -> int, but results in a generated CIL function taking a tuple of two ints, with signature more like int * int -> int.

But there are more complicated cases; F# functions can take functions and return them. When we’re dealing with functions in a first-class way like this, we can use the generic Microsoft.FSharp.Core.FSharpFunc type. For example:

    let public adder a b = a + b
    let public addOne = adder 1

Where addOne is a function that returns a partially applied function. Calling it from F# is a breeze:

    let x = addOne 100

But from C# it’s a bit trickier. The function returns an FSharpFunc, that we then need to apply with Invoke:

    Microsoft.FSharp.Core.FSharpFunc<int,int> f = Module1.addOne;
    var x = f.Invoke(100);

Although obviously you can combine this into a single line, and remove some of the type signatures (which I tend to do as much as possible using var when writing C# code):

    var x = Module1.addOne.Invoke(100);

Things get even nastier if you happen to expose an F# function that takes a function. For example, a function that takes another function and applies it to two arguments:

    let apply op a b = op a b

In F# interactive we can see that this has the following, nicely generalised type signature:

    val apply : ('a -> 'b -> 'c) -> 'a -> 'b -> 'c

i.e. it’s a function that takes as its first argument a function that takes an ‘a and a ‘b and returns a ‘c, it also accepts the ‘a and ‘b to pass to this function, and returns the ‘c result. Clear as. We may well want to create a version of this that uses the integer operator +, which is trivial from F#:

    let add a b = apply (+) a b

So how does it look if we try and do that from C#? In a word: messy. We have to use the Microsoft.FSharp.Core.FuncConvert.ToFSharpFunc helper functions, along with the .NET framework Converter as a way of specifying a generic, value-returning function.

    using Microsoft.FSharp.Core;
...
    var op = FuncConvert.ToFSharpFunc(new Converter<int, FSharpFunc<int, int>>((aa) =>
        {
            return FuncConvert.ToFSharpFunc(new Converter<int, int>((bb) =>
            {
                // The 'meat' of the function. Hmmm...
                return aa + bb;
            }));
        }));
    var zz = Module1.apply(op, 1, 2);

Whoa. That’s enough for now. As you can see, the various different ways of interoperating between F# and C# range from the neat to the nasty. The trick is definitely to pick your battles. Think carefully about what you need to expose and remember it’s probably best not to cross the beams if you can avoid it.

What is .NET type equivalence?

Described at a high level here, .NET 4.0 type equivalence essentially gives you a way of indicating that different .NET types represent the same underlying COM type and is most commonly used in COM interop scenarios. One of the reasons for its introduction is to save developers from having to ship large interop DLLs with their software, e.g. the multi-megabyte Microsoft.Office.Interop. Instead the compiler can inline the definition of any types used, and mark them appropriately as representing the original COM types.

The error

We noticed that whenever we built and ran an application that referenced a DLL using .NET 2.0, it worked. Doing the same thing with .NET 4.0 caused a BadImageFormatException.

Unhandled Exception: System.BadImageFormatException: Could not load file or assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies. An attempt was made to load a program with an incorrect format.
at X.Main()


Let’s dig!

So, the BadImageFormatException doesn’t actually tell us much. Let’s break out WinDbg and see what we can find. Running the faulting app we can see several C++ exceptions before the CLR exception is thrown:

(178c.790): C++ EH exception - code e06d7363 (first chance)
...
(178c.790): C++ EH exception - code e06d7363 (first chance)
(178c.790): CLR exception - code e0434352 (first chance)

I changed the exception handling settings to stop on C++ exceptions (sxe eh) then ran again to see where things were going wrong. It stopped here:

0:000> kp
ChildEBP RetAddr
0012d15c 79084c0f KERNEL32!RaiseException+0x53
0012d194 793371be MSVCR100_CLR0400!_CxxThrowException+0x48
0012d5e4 79455cae clr!EEFileLoadException::Throw+0x1a8
0012d634 794558d2 clr!CompareTypeTokens+0x200
0012d6b0 791b5c00 clr!IsTypeDefEquivalent+0x102
0012d6d4 791b2ca8 clr!MethodTableBuilder::CheckForTypeEquivalence+0x94
0012d7ac 791b27c9 clr!MethodTableBuilder::BuildMethodTableThrowing+0x60d
0012d9a4 791a4578 clr!ClassLoader::CreateTypeHandleForTypeDefThrowing+0x88e

Interesting. Notice how the call stack contains some .NET 4.0 specific methods relating to the new type equivalence feature. We’re hitting a new code path, which is consistent with the fact that running against a down-level CLR works.

After a bit more toing-and-froing, I discovered that the C++ exception is thrown when clr!MDInternalRO::IsValidToken returns an error. By disassembling the function we can see it’s just looking at various bits in the token value, and it decides that the value passed (0×02000000) isn’t valid. Looking at the output from ildasm that token doesn’t appear anywhere. And if we add a dump of the value, we can see that it indeed doesn’t look like the other tokens:

0:000> bu clr!MDInternalRO::IsValidToken "dd esp+8 L1; g"
...
0012f5a8  02000001
0012f31c  06000001
0012f2c0  02000002
0012f0f4  02000002
0012ebe4  01000001
0012e944  23000001
...
0012d5f4  02000000
(18ec.1ec8): C++ EH exception - code e06d7363 (first chance)

What’s the culprit?

So it looks pretty conclusive; the DLL contains something that the CLR isn’t expecting. But what? It’s time to break out the oldest tool in the troubleshooting box: the binary chop!

Eventually I got the referenced DLL down to only a single simple construct. Can you guess what it is? A global literal value. A real global value, one that isn’t even part of a type. Crazy huh? In IL it looks like this:

.field public static literal valuetype Test.MyEnum LiteralValue = int32(0x00000001)

It’s a literal value of an enumerated type. That’s important: using a value of a simple type (say int32) does not provoke the error.

Now, I wasn’t even sure that this is a valid IL construct, but according to the ECMA IL spec, specifically partition II, section 15, it is:

The CLI also supports global fields, which are fields declared outside of any type definition. Global fields shall be static.

So it looks like we’re not doing anything illegal, backed up by the fact that the .NET 2.0 CLR can make use of it without a problem.

Interestingly, there’s another aspect that influences whether this code path is hit. As mentioned above, type equivalence is intended for use with interop libraries. As such, it only kicks in if your referenced assembly is marked with the PrimaryInteropAssembly attribute, e.g.:


.custom instance void [mscorlib]System.Runtime.InteropServices.PrimaryInteropAssemblyAttribute::.ctor(int32,int32) = ( 01 00 01 00 00 00 00 00 00 00 00 00 )


The Fix?

The issue is currently with Microsoft product support. Let’s see what they come up with; is it too esoteric for a hotfix…?

The Repro

Here’s some code and instructions on how to repro the problem.

1. Build the IL into a DLL using ilasm.
   "c:\WINNT\Microsoft.NET\Framework\v2.0.50727\ilasm.exe" /dll Test.il /output=Test.dll
2. Build the application into a .NET 4.0 EXE that references the DLL
   "c:\winnt\Microsoft.NET\Framework\v4.0.30319\csc.exe" TestConsumer.cs /reference:Test.dll
3. Run the resulting TestConsumer.exe application and you’ll get the exception

Test.il

.assembly extern mscorlib
{
.publickeytoken = (B7 7A 5C 56 19 34 E0 89 )
.ver 2:0:0:0
}
.assembly Test
{
.custom instance void [mscorlib]System.Runtime.InteropServices.PrimaryInteropAssemblyAttribute::.ctor(int32,int32) = ( 01 00 01 00 00 00 00 00 00 00 00 00 )
.hash algorithm 0x00008004
.ver 1:0:0:0
}
.module Test.dll
.imagebase 0x00400000
.file alignment 0x00000200
.stackreserve 0x00100000
.subsystem 0x0003
.corflags 0x00000001

.class public auto ansi sealed Test.MyEnum
extends [mscorlib]System.Enum
{
.field public specialname rtspecialname int32 value__
.field public static literal valuetype Test.MyEnum Zero = int32(0x00000000)
.field public static literal valuetype Test.MyEnum One = int32(0x00000001)
}

TestConsumer.cs

class X
{
    static void Main()
    {
        var v = Test.MyEnum.Zero;
    }
}

So what tools can you use to troubleshoot problems with dynamic IL? In a word: peverify.

What is peverify?
peverify (where PE stands for portable executable, the file format used by Windows executables) takes a .NET assembly and validates the metadata – the type structure – and IL – the instructions, using a combination of rules and static analysis.

As the MSDN pages states it’s intended for use by “compiler writers and script engine developers”, but with a few small modifications to your code you can also use it with Reflection.Emit.

Here’s some example code that uses F# to emit a simple type (deriving from MarshalByRefObject) to wrap another object. The intention is that the wrapper type implements the same interfaces as the underlying object, and delegates calls to it. The key part of the code is where we emit the body of each function.

let wrapObject (obj:obj) =

    let name = "MyAssembly"

    let filename = name + ".dll"

    let ab = AppDomain.CurrentDomain.DefineDynamicAssembly(AssemblyName(name), AssemblyBuilderAccess.RunAndSave)

    let modb = ab.DefineDynamicModule(filename, filename)

    let tb = modb.DefineType("MyType", TypeAttributes.Class, typeof<MarshalByRefObject>)

    let objField : FieldBuilder = tb.DefineField("_underlyingObject", typeof<obj>, FieldAttributes.Public)

    obj.GetType().GetInterfaces() 

    |> Array.iter (fun itf ->

        tb.AddInterfaceImplementation(itf)

        itf.GetMethods()

        |> Array.iter (fun m ->

            let ptypes = m.GetParameters() |> Array.map (fun p -> p.ParameterType)

            let mb = tb.DefineMethod(m.Name, MethodAttributes.Public|||MethodAttributes.Virtual, m.ReturnType, ptypes)

            let il = mb.GetILGenerator()

            // load object reference from field

            il.Emit(OpCodes.Ldarg_0)

            il.Emit(OpCodes.Ldfld, objField)

            // cast it to appropriate type

            il.Emit(OpCodes.Castclass, itf)

            // push all args, verbatim

            m.GetParameters() |> Array.iteri (fun n p -> il.Emit(OpCodes.Ldarg,n+1))

            // call method on underlying objects

            il.Emit(OpCodes.Callvirt, m)

            il.Emit(OpCodes.Ret)

        ))

    let typ = tb.CreateType()

    ab.Save(filename)

    let o = typ.GetConstructor([||]).Invoke([||])

    typ.GetField("_underlyingObject").SetValue(o, obj)

    o

Now, in my first version of the code, I forgot that you need to load “this” onto the stack before loading a field, so the code looked like this:

I used the following code to create an instance of a type implementing IFoo, wrap it and invoke the function:

do

    let o = 

        { new MyInterfaces.IFoo with

            member this.Bar(a) = a + "!" }

    let wrappedObj = wrap o

    wrappedObj.GetType().InvokeMember("Bar", BindingFlags.Instance|||BindingFlags.Public|||BindingFlags.InvokeMethod, null, wrappedObj, [|box "Hello"|])

    ()

As you’d expect the function generated the expected InvalidProgramException:

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidProgramException: Common Language Runtime detected an invalid program.
at MyType.Bar(String )
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
at System.RuntimeMethodHandle.InvokeMethodFast(Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)


In order to use peverify with your code, the first thing you need to do is save the assembly to disk. This involves making a couple of changes to the code, because normally you’d do everything in memory and avoid the cost of the disk access. First, change Run to RunAndSave in the call to define the assembly:

And ensure you include the filename when you create the module:

Now, you should be able to save the assembly to disk by adding a call after you’re finished building the type:

Be aware that you can’t specify a path in the call to AssemblyBuilder.Save. This is presumably a security restriction; keeping generated binaries within the application’s directory tree and preventing people creating arbitrary binaries all over the file system. If you’re not running from an obvious “top level” application, e.g. you’re using a scripting environment like F# interactive, you may find the location of the file odd; when using FSI from within Visual Studio, the binary is created in “C:Program FilesMicrosoft Visual Studio 9.0Common7IDE”.

Now we can run peverify on the DLL and we get a much richer explanation of the problem:


Microsoft (R) .NET Framework PE Verifier. Version 3.5.30729.1
Copyright (c) Microsoft Corporation. All rights reserved.

[IL]: Error: [c:Program FilesMicrosoft Visual Studio 9.0Common7IDEMyAssembl
y.dll : MyType::Bar][offset 0x00000000] Stack underflow.
1 Error(s) Verifying c:Program FilesMicrosoft Visual Studio 9.0Common7IDEMy
Assembly.dll


From this error message it’s pretty clear that the first thing we’re doing (i.e. the instruction at offset 0) is underflowing the stack; calling an instruction that expects more on the stack than we’ve put there. That should give us a good idea of how to fix the problem.

Another error that I came across was the omission of a cast in the IL:

From which peverify generated the fantastically specific error message:

[IL]: Error: [c:Program FilesMicrosoft Visual Studio 9.0Common7IDEMyAssembl
y.dll : MyType::Bar][offset 0x0000000C][found ref 'System.Object'][expected ref
'MyInterfaces.IFoo'] Unexpected type on the stack.


And of course, once all your emission bugs are fixed, you should get a success report:

All Classes and Methods in c:Program FilesMicrosoft Visual Studio 9.0Common7
IDEMyAssembly.dll Verified.


The only problem with peverify as far as I can see is that there are no compiler style “error codes” to help you identify and resolve the problem. It assumes fairly intimate knowledge of the IL/CLR specification and there may be a bit of effort required to get from the error to the resolution. But one thing’s for sure: it’s definitely easier than trying to do it with just InvalidProgramException.