Work: Using F# in anger

During working hours (and sometimes beyond) I’ve been developing a large project using lots of F#, a little bit of C# and a bit too much XAML. It’s quite an interesting mix, with technologies that have a vastly different outlook on life: F# with it’s strong, compile-time type checking and XAML with its, err, not-so-strong type checking. In fact, XAML and WPF play so fast and loose with types, that errors in the binding mechanism only happen at run-time and are almost always eaten silently. As you can imagine, this makes for some fun debugging. All I can say is thank goodness for Snoop.

We have a quite a substantial amount of ViewModel code in F# (tens of thousands of lines), much of it using Async to allow us to create responsive UIs without requiring tons of boilerplate code. Generally it works nicely, but I do have some concerns about the amount of thread hopping going on, and how that will affect performance. Especially as we break things down into pretty small pieces of functionality, sometimes just a few simple operations, and each is run on the thread pool. I don’t have any concrete figures to go on here, and I’m hoping we don’t end up in a situation where we sacrifice user productivity for developer productivity. Time (and perfmon) will tell.

Debugging

I’ve done less hardcore WinDbg stuff this year. In fact, a couple of times I’ve been guilty of jumping in with the debugger where it probably would’ve been more effective to take a high-level look first. A case of having a debugger-shaped hammer, and everything looking like a nail.

Moving to 64bit

A very positive thing that happend this year was the move to 64-bit Windows 7 for development at work, meaning that Visual Studio actually stays up without crashing for more than an hour. Previously I’d been having a terrible time: the combination of a couple of large-ish F# projects and 32-bit XP caused frequent VS out-of-memory errors, which was very frustrating. Luckily the move to Windows 7 has resolved it, and I can even run more than a handful of apps at the same time; what a luxury.

Home: iOS

After a long day at work, I finally get to go home, put the kids to bed and then my feet up, and break-out Xcode for a bit of iOS hacking. This year I’ve been feeling like I’m finally getting my head around some of the things that are fundamental in iOS development; from Objective-C concepts like release/retain semantics to iOS specifics, view controllers, table views etc. Previously I’ve been guilty of some copy-and-paste coding, mostly because it’s pretty slow going getting up to speed with new tech (and the myriad iOS APIs) when you’re only getting to do it for a few hours while knackered after work. Well, that’s my excuse anyway.

I’ve been trying to finish off some of the apps that I’ve started, but not shipped. I tend to procrastinate a bit (massive understatement), and of course things aren’t helped by the development ‘fat tail’, where the last 20% – polishing and getting things ‘just right’ – takes at least 80% of the time. My 2010 post on Core Animation flip clocks continues to get a lot of traffic, and I finally bit the bullet and posted a (free!) app that uses it and it’s getting a lot of downloads, which is nice. I also had a look at iAd, which I’ll reserve judgement on for the time being, at the minute it’s not looking particularly lucrative…

As well as iOS stuff, I also took a look at OpenGL. I tend to do this every now and again, each time dipping in a little further and understanding a bit more. This time round it was looking at shaders (in OpenGL ES 2.0) and integrating with Bullet physics. My learning was certainly helped by this excellent Learning Modern 3D Graphics Programming tutorial.

Kinect and F#

Also while at home I finally got around to setting-up by my Kinect to do a bit of F# hacking. In the end I didn’t produce very much, but it was pretty good fun anyway.

Bring on 2012

So, to sum it up: more of the same really, I guess. Although this year there’s definitely been a bit more of a focus on UI development, in both WPF and iOS, which I’ve been enjoying. It does tend to feel a bit schizophrenic, switching between technologies and platforms at the end of the working day. It can be interesting to identify the common themes and significant differences in the two approaches to GUI development.

This year I’m hoping to make my blogging a bit more regular, but who knows if I’ll be able to stick to it… If there’s anything you’d particularly like me to focus on, feel free to leave a comment.

Just what do you think you're doing, Dave?

The first bit was easy: I’m “lucky” enough to have one of the older Xboxes, which meant I’d had to get a Kinect with separate power, which is the one required by the SDK. Now all I needed was a Windows machine to develop on.

For all my Visual Studio stuff at home I use a virtual machine, and unfortunately I missed the point in the readme about “Kinect for Windows applications cannot run in a virtual machine”. Doh. That would explain why, try as I might I couldn’t get VirtualBox to detect the device when I plugged it into the host. Whatever I did I ended up with a ‘resource is busy’ error.

I even tried another VM, this time from VMWare. It got further, with the guest seeing the devices, but whenever I tried to run the sample apps the API initialisation call failed. Unfortunately the samples make a bit of a rookie mistake and don’t display the error code associated with the failure; the only way to get the underlying HRESULT is to debug the app. As it turned out the error was the catch-all 80080014, which is attributed to various USB issues.

So, I finally relented and decided to once again set-up Bootcamp, which I’d stopped using a while back: why bother when VMs had done everything I needed? Again I was snookered, this time by the incompatibility of the EFI firmware in my ageing MacBook with the Win7 x64 install disc (which I’d had to actually burn onto optical media – can’t remember the last time I burnt a real physical disk)!

Eventually, I got Win7 x86, Visual Studio and the Kinect SDK installed on the metal, plugged in the sensor and – whoa – the devices were recognised and the drivers installed and… the samples ran!

From this point doing some hacking was pretty straightforward. I set about creating a project that used the skeletal tracking ability from the SDK. The project needs to reference the Microsoft.Research.Kinect assembly, then we can open some namespaces and initialise the API:

open System
open Microsoft.Research.Kinect
 
let nui = Nui.Runtime()
nui.Initialize(Nui.RuntimeOptions.UseSkeletalTracking)

The API uses .NET events to communicate back to the application that some form of data is available. Depending on the options that you specify at init time, any of the skeletal, depth frame or colour data will be returned in the event arguments. This seems like a good place to use F# Async’s event integration: Async.AwaitEvent. We can quite easily write some code that will create an Async task that will repeatedly listen for events:

    let skeleton (nui : Nui.Runtime) =
        let rec loop () =
            async { 
                let! args = Async.AwaitEvent nui.SkeletonFrameReady
                // Do something with the args
                return! loop () 
            }
        loop ()
    // Start the process of listening for events...
    skeleton nui |> Async.Start

This seems to work nicely, although I’m not sure of the overhead involved, perhaps a solution involving Rx would be more appropriate. Anyway, let’s do something with the data we get in the event, like getting all the skeletons that are being tracked, and passing them to a draw function:

                args.SkeletonFrame.Skeletons 
                |> Seq.filter (fun s -> s.TrackingState = Nui.SkeletonTrackingState.Tracked)
                |> Seq.iter draw

All that’s left is doing something cool in draw! Oh, and setting up all the GUI stuff necessary to actually get some pixels on the screen. I went for the WPF approach (as the managed samples do), which involves creating a simple object tree to display a rectangle in a canvas in a grid in a window, and doing a bit of marshalling back from the thread pool (where our Async code runs) to the main GUI thread.

Here’s all the code for possibly the most tedious thing you could do with a Kinect! But hey, it’s a starting point, right? I’m sure at some point I’ll get around to doing something cooler than a couple of small red rectangles with it…

open System
open System.Windows
open Microsoft.Research.Kinect
 
[<STAThread>]
do
    let nui = Nui.Runtime()
    nui.Initialize(Nui.RuntimeOptions.UseSkeletalTracking)
 
    // lifted straight from the sample code
    let getDisplayPosition w h (joint : Nui.Joint) =
            let mutable depthX = 0.0f
            let mutable depthY = 0.0f
            nui.SkeletonEngine.SkeletonToDepthImage(joint.Position, &depthX, &depthY)
            let depthX = depthX * 320.0f; //convert to 320, 240 space
            let depthY = depthY * 240.0f; //convert to 320, 240 space
            let mutable colorX = 0
            let mutable colorY = 0
            let iv = new Nui.ImageViewArea()
            // only ImageResolution.Resolution640x480 is supported at this point
            nui.NuiCamera.GetColorPixelCoordinatesFromDepthPixel(Nui.ImageResolution.Resolution640x480, iv, 
                (int)depthX, (int)depthY, 0s, &colorX, &colorY)
 
            // map back to visible area
            new Point((w * (float)colorX / 640.), (h * (float)colorY / 480.))
 
    // Set-up the WPF window and its contents
    let width = 1024.
    let height = 768.
    let w = Window(Width=width, Height=height)
    let g = Controls.Grid()
    let c = Controls.Canvas()
    let hd = Shapes.Rectangle(Fill=Media.Brushes.Red, Width=10., Height=10.)
    ignore <| c.Children.Add hd
    ignore <| g.Children.Add c
    w.Content <- g
 
    // We simple move the rectangle to where the head is
    let draw (s : Nui.SkeletonData) =
        let point = getDisplayPosition width height s.Joints.[Nui.JointID.Head]
        ignore <| w.Dispatcher.BeginInvoke(Threading.DispatcherPriority.Normal, Action(fun () ->    
            hd.SetValue(Controls.Canvas.TopProperty, point.Y)
            hd.SetValue(Controls.Canvas.LeftProperty, point.X)))
 
    let skeleton (nui : Nui.Runtime) =
        let rec loop () =
            async { 
                let! args = Async.AwaitEvent nui.SkeletonFrameReady
                args.SkeletonFrame.Skeletons 
                |> Seq.filter (fun s -> s.TrackingState = Nui.SkeletonTrackingState.Tracked)
                |> Seq.iter draw
                return! loop () 
            }
        loop ()
 
    skeleton nui |> Async.Start
 
    let a = System.Windows.Application()
    ignore <| a.Run(w)
 
    nui.Uninitialize()

The moral of the story? Don’t be like me, and make sure you actually read the FAQing readme, then maybe you’ll spend more time doing a decent demo and less time plugging and unplugging your Kinect!

The change

The change itself is simple: static fields can no longer be public. Static fields can still be created, but they must be private.

In pre-2.0 versions of F# it was possible to declare a static field on a type like this:

type MyType() =
    [<DefaultValue>]
    static val mutable public MyProperty : int

Resulting in a type containing a static field, as you’d expect:

    .field public static int32 MyProperty

The code now generates the compiler error:

error FS0881: Static 'val' fields in types must be mutable, private and marked with the '[<DefaultValue>]' attribute. They are initialized to the 'null' or 'zero' value for their type. Consider also using a 'static let mutable' binding in a class type.

Notice that there are already some gnarly aspects to the definition of the property; notably the use of the [<DefaultValue>] attribute, which indicates that the field is un-initialized. This gives us a hint that there might be some inherent problems.

Why do we even need them?

In a word or three: WPF dependency properties.

The recommended way of implementing dependency properties in other .NET languages is to use public static fields, e.g. in C#:

    public static readonly DependencyProperty MyPropertyProperty =
        DependencyProperty.Register( 
          "MyProperty", typeof(int), typeof(MyType));

This is one of the few places where C# is terser than F#. Here we can declare and define the value in one line (ish), whereas F# requires a separate declaration of the field (as above) then initialisation in the static constructor (static do).

Why was it removed?

I got the answer from the horse’s mouth. Don Syme said that:

We deliberately removed the ability to create public static fields in Beta2/RC, because of issues associated with initialization of the fields (i.e. ensuring the “static do” bindings are run correctly, and if they are run, then they are run for the whole file, in the right order, with no deadlocks or uninitialized access when initialization occurs from multiple threads).

You can imagine how this would be a problem; there would need to be a way of ensuring that whichever static field was accessed it caused the static constructor to run, which may itself access static fields. All pretty nasty. In fact, Don mentioned that C# suffers from much the same synchronisation issues, but just tends to be used in a way that means it’s less likely to be noticed!

The alternative

At least in theory it’s possible to create a type that uses static properties rather than static fields to store its registered DependencyProperty information.

type Foo () =
    inherit FrameworkElement()
    static mutable private _myPropertyInternal : DependencyProperty 
    static member this.MyProperty with get () = _myPropertyInternal

Whether or not this works depends very much on how calling code accesses DependencyProperty information. If it uses reflection to access the field directly it will obviously fail. But if it uses a more robust/flexible method then it should be OK. Empirically it seems that the WPF framework code itself does the latter, for instance, when it’s instantiating objects from XAML, and it works properly independently of how it’s implemented.

The conclusion

So the conclusion is “wontfix”: the behaviour is by design. Unfortunately it has the effect that it’s no longer possible to create a type with an identical IL “signature” in both F# and C#. It seems a bit of a shame, but I guess the trade-off is that we’re protected against insiduous initialisation issues, so it will probably turn out to be the right thing in the long term.

type public MyControl() =
    inherit ItemsControl()
 
    [<defaultValue>]
    static val mutable FooProperty : DependencyProperty
 
    static member OnFooChanged (dob:DependencyObject) args =
        (dob :?> MyControl).Update ()
 
    [<system.ComponentModel.Bindable(true)>]
    member public this.Foo
        with get() : string  = string (base.GetValue(MyControl.FooProperty))
        and  set(r : string) = base.SetValue(MyControl.FooProperty, r)
 
    override this.OnPropertyChanged (args) =
        match args.Property.Name with
        | "Foo" -> this.Update ()
        | _          -> ()
 
    member internal this.Update () =
        System.Diagnostics.Debug.WriteLine (sprintf "Updating %A" (base.GetValue(MyControl.FooProperty)))
 
    static do
        let metadata = PropertyMetadata(null, PropertyChangedCallback (MyControl.OnFooChanged) )
        MyControl.FooProperty <- DependencyProperty.Register("Foo", typeof<string>, typeof<MyControl>, metadata)

Constructor

We don’t have one! Well, that’s not strictly true. There’s no per-instance set-up that we need to do here, so instead we have a default, parameter-less constructor implied by the “empty brackets” syntax in the type declaration. If we wanted to execute some code in the constructor, we could add the following:

    do
        Debug.WriteLine "Constructing."

It’s also possible to add further constructors (perhaps parameterised differently) but when using WPF bear in mind that instances of your class will often be created from a XAML declaration, which generally uses the default constructor and then sets properties as required. Mutable objects: ug.

Inheritance

Our type derives from the WPF ItemsControl class using the inherit keyword. Of course, we’re still subject to the single-inheritance limit of .NET (not a bad thing, if you ask me – no more tortured object hierarchies):

type public MyControl() =
    inherit ItemsControl()

Note that we have to include the empty brackets on the inherited type name, as this will be constructed implicitly when our derived class is constructed. We can access the inherited class from elsewhere in the code using the base keyword.

Static members

Dependency properties are a WPF construct that provide external storage of property values. They allow deep trees of objects to efficiently use lots of properties where they often have the default value, which is commonly the case in WPF. In order to use them with your class you have to do a few things, including creating a static value to hold the property and its metadata. We create a mutable static member for this:

    [<defaultValue>]
    static val mutable FooProperty : DependencyProperty

Why a mutable static value? If you’ve used F# already you might be aware that it’s also possible to declare an immutable static variable and its initial value in one shot with a let:

    static let FooProperty = DependencyProperty.RegisterProperty ("Foo", typeof<string>, typeof<MyControl>)

Unfortunately, this results in your DP being private. Although the CLR property is still accessible, anything that attempts to access the DP directly – for instance, code within the WPF libraries – won’t see it. This means you have to use the mutable style, which is unfortunate.

Static methods

We’ve declared a static member function that is used to receive notifications when our DP is changed (although it’s complete overkill in this example, because we’ve already declared our DP with metadata that tells it to notify us when it changes):

    static member OnFooChanged (dob:DependencyObject) args =
        (dob :?> MyControl).Update ()

As you’d expect, there’s no this parameter on the signature, a static member doesn’t have any implicit object instance to work on. Luckily the arguments to most event functions include the DependencyObject that raised the notification. That means we can downcast dynamically to our expected type (with : ?>) and use it. Bear in mind that this is more like casting than as in C#, as it will throw an InvalidCastException at runtime rather than returning null.

Properties

In order for our dependency property to be easily accessible we can expose it as a plain old CLR property. The implementation of the getter and setter simply defers all of the actual work of storing and retrieving the value to the underlying dependency property.

    member public this.Foo
        with get() : string  = string (base.GetValue(MyControl.FooProperty))
        and  set(r : string) = base.SetValue(MyControl.FooProperty, r)

Notice how we have to cast the obj returned from GetValue into the correct type. This is another example of having to bridge the gap between the dynamically typed WPF property system and F#’s static typing.

Overridden members

As well as providing new member functions and properties, we may need to override existing ones. Member functions marked abstract in the base class can be overridden using the override keyword:

    override this.OnPropertyChanged (args) =
        match args.Property.Name with
        | "Foo" -> this.Update ()
        | _          -> ()

The F# compiler provides the same kind of checking that the C# compiler does, warning you if you inadvertantly hide a base class function by creating an override with the same name but not marking it with override.

Static constructor

As mentioned before, we use the static constructor to initialise our mutable static variables. The syntax is similar to the do syntax of a normal constructor, with the addition of the static keyword:

    static do
        let metadata = PropertyMetadata(null, PropertyChangedCallback (MyControl.OnFooChanged) )
        ...

Static constructors are run once per class, regardless of how many instances of the class you have. WPF relies quite heavily on static, class-based functionality; mostly because a lot of what’s set-up is per-class configuration – it’s not going to change during the lifetime of the application – so you may find yourself doing a fair amount of work in a static constructor.

So, there’s a quick run around some of the object-oriented features of the F# language: classes, inheritance, instance constructors, overridden member functions, static member functions and constructors. You can see how using WPF means you lose some of the benefits of the F# language; notably immutability and static typing. If you’re an experienced functional programmer getting deep into creating WPF or Silverlight custom controls you may find yourself using these OO constructs more than you’d like. Although F# makes it relatively painless in practice, mixing this heavily object oriented style of programming with a functional approach can still be a little hard to stomach at times.

I’m making use of a few F#/.NET idioms to make the process easier. For instance, sequences (.NET’s IEnumerable) are used as the source of uniformly distributed random numbers. Also, to verify that the resulting numbers are actually normally distributed, we can easily use existing WPF/silverlight controls to visualise the values direct from within Visual Studio, in a manner similar to this previous post – but without having to write the plotting code ourselves.

Implementation

So firstly, we create the random number stream:

let rr = System.Random(System.DateTime.Now.Minute)
let rands = seq { while true do yield rr.NextDouble() }

As you can see rands is an infinite sequence. As such, you should be wary of the “eager” functions from the Seq module that attempt to consume the entire sequence at once, they’ll obviously cause a problem here. Also notice that I’ve picked a somewhat dubious seed for the random number generator, as this is only for demonstration purposes.

In order to obtain normally distributed random numbers I’m using the Box-Muller transform. I started off with an implementation transcribed from VBA (from Wilmott et al) – yes, yes, I know – but of course this used mutable references. Not nice. So I reworked it a little to use Seq.pick, which takes items from the sequence while a predicate returns None:

let boxMuller _ =
    let (x,dist) = 
        rands
        |> Seq.pick (fun x ->
            let x = 2.0 * x - 1.0
            let y = 2.0 * ((Seq.take 1 rands) |> Seq.hd) - 1.0
            match (x * x + y * y) with
                | dist when dist < 1.0 -> Some (x,dist)
                | dist -> None
            ) 
    x * Math.Sqrt(-2.0 * Math.Log(dist) / dist)

It’s not particularly efficient as we’re consuming two uniform numbers to generate one normal one – hence the additional Seq.take within the pick. Of course we could change the function to return the 2 generated numbers as an extra element in the tuple.

Next, let’s generate some numbers, and get them in a suitable form to display:

let makeData _ =
    Seq.init 100000 boxMuller
    |> Seq.countBy (fun d -> 
        let d = decimal d
        Math.Round(d, 1))
    |> Seq.sortBy fst

We initialise a sequence with an arbitrary amount of numbers (100000, enough to get a good distribution, hopefully), then use Seq.countBy to do a naive grouping that allows us to see how the numbers are distributed, i.e. counting how many numbers fit in each 0.10 bucket. We then sort the output by bucket.

Visualisation

Now we can display what we’ve got. We can create a simple WPF object-model using imperative code, including the very useful chart control from the WpfToolkit. The chart control enables us to add a series and set our seq<decimal * float> directly as the series ItemsSource, which is the standard way to specify the items for a collection control. If you specify the tuple Item1 and Item2 properties for the bindings, WPF will be able to access the data from each element, and that’s all you need.

    let series = 
        ColumnSeries(
            IndependentValueBinding = Data.Binding("Item1"),
            DependentValueBinding = Data.Binding("Item2"),
            ItemsSource = makeData ())
    let chart = Chart()
    chart.Series.Add series
    Window(
        Name="Plot",
        Title="Normally distributed random numbers",
        Width=900.0,
        Height=700.0,
        Content=chart,
        Visibility=Visibility.Visible)

Here’s the resulting output:
As I’ve mentioned before, this is a fantastically immediate way of doing numerical development in Visual Studio. You can enter all of this code directly into F# interactive within VS, iterate, and quickly see the effect that your changes have. It’s just as interactive as using the graph features in Excel, and the resulting code is much more easily reusable.

XD5RRV3XDNHN