“The 25th Consumer Electronics Show… showed very clearly that one thing continues to dominate the media and electronics business in the USA. This is not the personal computer – which seems to be causing far more interest in Europe than in America – but the video game.”

So while in the UK we were busy typing in listings from games magazines, in the US Atari, Mattel were creating ready-to-play cartridge based games.

Getting compelling press coverage was made difficult by the fact that it was impossible to get decent screen shots. Think how much of the pages in modern games mags are covered with glossy, full-colour, high-resolution screen shots. Without these, you’re left with nothing but words to describe these visual wonders. Luckily, you didn’t need much of a vocabulary to describe the games of the time; they were either based on existing analogue games, or they were derived from the small pool of original video game ideas:

“Munchie, which is similar to PacMan” … “Cosmic Raiders is a fast-action Defender style games” … “Solar Conqueror is an Asteroids game.”

Of course, some of this “re-purposing” was pukka; Coleco made table-top versions of things like PacMan, Donkey Kong, and Frogger (image above). You can imagine how faithful these arcade conversions actually were…

We’ve made quite some progress in the last 30 years, both in terms of game mechanics and graphics. It’s best summed up by 2 images, I’m sure you can guess which is from 1982, and which from 2012?

Click through these images for the full scans on flickr.

In these times of financial woe, people love to complain about all the big, nasty, faceless companies avoiding paying tax. They inevitably moan about it on Facebook, Google and Twitter (and sometimes even do something about it in the real, physical, world). But what about those companies themselves? The Web 2.0 behemoths are now some of the largest companies in the world, and they’ve decided – just like the evil old companies – that they don’t like paying all that tax. Don’t be evil… and don’t pay any tax if you can help it. But how do they do it?

The Numbers

An Irish Times newspaper article caught my eye the other day. It discusses the rapid growth of Facebook’s Dublin operation in 2010, whose accounts have just been published (as they must be by law, despite FB’s apparent reluctance). The headline figure is that:

[Revenue] is up from €15.2 million to €229.1 million

That’s a pretty significant increase! But that’s only revenue, how much of it was left over as profit?

Facebook Ireland generated a pretax profit of €1.9 million

What?! Where did the other €227 million go?

According to the article it mostly disappeared into a mysterious black hole known as “administrative expenses”:

Administrative expenses increased … to €221.6 million as a result of … royalties paid to other Facebook entities for services provided during the year.

How do they do that?

Is it me or does that sound suspicious? Well, it should do, because it is.

It turns out that Ireland’s significantly lower rate of corporation tax (12% vs 26% in the UK, and one of the lowest in the world) isn’t the only reason why companies like Facebook, Google et al move there. There’s also the very useful fact that, unlike most developed economies, Ireland chose not to introduce any legislation to prevent companies siphoning off their revenue to low-or-no-tax regimes like the Cayman islands or Bermuda.

So that’s where the “missing” €221 million has ended up: it’s been paid from the Irish part of the business to the tax-haven subsidiary, leaving only a small amount subject to corporation tax in Ireland. This practise is known in tax circles as the “double Irish”, and it (along with its Dutch equivalent) seems to be fairly widely used and well understood, even having its own wikipedia entry. Of course Facebook now funnels as much as possible of the revenue from the UK (its second biggest market) through the Ireland operation to get the most out of this arrangement. That explains the large increase in the numbers since last year.

There are many high-profile tech companies that have Dublin operations and may be avoiding tax in this way, including:

• Facebook
• Twitter
• LinkedIn
• Google
• Microsoft
• Oracle

That’s quite a roll call.

The Result

The result of all these shenanigans for Facebook? It pays a ridiculously small effective tax rate of less than 1%, even better than Google’s 2.4% rate. And given Ireland’s current economic state I can’t imagine their government will be introducing more restrictive tax legislation any time soon in an attempt to crack down on it. They can’t risk these big companies abandoning their new homes in Dublin’s Silicon Dock.

Of course, you should feel free to ‘like’ this blog post on Facebook, after all, what will it cost you?

(* Disclaimer: These are solely my opinions, not those of my employer or anyone else).

Work: Using F# in anger

During working hours (and sometimes beyond) I’ve been developing a large project using lots of F#, a little bit of C# and a bit too much XAML. It’s quite an interesting mix, with technologies that have a vastly different outlook on life: F# with it’s strong, compile-time type checking and XAML with its, err, not-so-strong type checking. In fact, XAML and WPF play so fast and loose with types, that errors in the binding mechanism only happen at run-time and are almost always eaten silently. As you can imagine, this makes for some fun debugging. All I can say is thank goodness for Snoop.

We have a quite a substantial amount of ViewModel code in F# (tens of thousands of lines), much of it using Async to allow us to create responsive UIs without requiring tons of boilerplate code. Generally it works nicely, but I do have some concerns about the amount of thread hopping going on, and how that will affect performance. Especially as we break things down into pretty small pieces of functionality, sometimes just a few simple operations, and each is run on the thread pool. I don’t have any concrete figures to go on here, and I’m hoping we don’t end up in a situation where we sacrifice user productivity for developer productivity. Time (and perfmon) will tell.

Debugging

I’ve done less hardcore WinDbg stuff this year. In fact, a couple of times I’ve been guilty of jumping in with the debugger where it probably would’ve been more effective to take a high-level look first. A case of having a debugger-shaped hammer, and everything looking like a nail.

Moving to 64bit

A very positive thing that happend this year was the move to 64-bit Windows 7 for development at work, meaning that Visual Studio actually stays up without crashing for more than an hour. Previously I’d been having a terrible time: the combination of a couple of large-ish F# projects and 32-bit XP caused frequent VS out-of-memory errors, which was very frustrating. Luckily the move to Windows 7 has resolved it, and I can even run more than a handful of apps at the same time; what a luxury.

Home: iOS

After a long day at work, I finally get to go home, put the kids to bed and then my feet up, and break-out Xcode for a bit of iOS hacking. This year I’ve been feeling like I’m finally getting my head around some of the things that are fundamental in iOS development; from Objective-C concepts like release/retain semantics to iOS specifics, view controllers, table views etc. Previously I’ve been guilty of some copy-and-paste coding, mostly because it’s pretty slow going getting up to speed with new tech (and the myriad iOS APIs) when you’re only getting to do it for a few hours while knackered after work. Well, that’s my excuse anyway.

I’ve been trying to finish off some of the apps that I’ve started, but not shipped. I tend to procrastinate a bit (massive understatement), and of course things aren’t helped by the development ‘fat tail’, where the last 20% – polishing and getting things ‘just right’ – takes at least 80% of the time. My 2010 post on Core Animation flip clocks continues to get a lot of traffic, and I finally bit the bullet and posted a (free!) app that uses it and it’s getting a lot of downloads, which is nice. I also had a look at iAd, which I’ll reserve judgement on for the time being, at the minute it’s not looking particularly lucrative…

As well as iOS stuff, I also took a look at OpenGL. I tend to do this every now and again, each time dipping in a little further and understanding a bit more. This time round it was looking at shaders (in OpenGL ES 2.0) and integrating with Bullet physics. My learning was certainly helped by this excellent Learning Modern 3D Graphics Programming tutorial.

Kinect and F#

Also while at home I finally got around to setting-up by my Kinect to do a bit of F# hacking. In the end I didn’t produce very much, but it was pretty good fun anyway.

Bring on 2012

So, to sum it up: more of the same really, I guess. Although this year there’s definitely been a bit more of a focus on UI development, in both WPF and iOS, which I’ve been enjoying. It does tend to feel a bit schizophrenic, switching between technologies and platforms at the end of the working day. It can be interesting to identify the common themes and significant differences in the two approaches to GUI development.

This year I’m hoping to make my blogging a bit more regular, but who knows if I’ll be able to stick to it… If there’s anything you’d particularly like me to focus on, feel free to leave a comment.

Star and text

Drawing the basic shape

The first step is to draw the ’star’ shape. I decided to create a re-usable path, although at the minute I’m actually only drawing it once, so the benefits are limited. Doing this involves using the CGPath... versions of the Core Graphics functions, that add themselves to a mutable path objects, rather than being drawn directly on the currently active graphics context.

    CGMutablePathRef r = CGPathCreateMutable();
    // Always have to move to a start point
    CGPathMoveToPoint(r, NULL, x1, y1); 
    CGPathAddLineToPoint(r, NULL, x2, y2);
    CGPathCloseSubpath(r);

Once you’ve got created the path you can merrily set various stroke and fill colours and draw it over-and-over to your hearts content.

The star drawing is parameterised in a few ways: we can set the inner radius (r2), outer radius (r1) and the number of points, which must be divisible by 3. We draw the points in groups of three, the first on the outer radius, then the inner, then the last on the outer. In that way the sections tesselate into a complete circle.

Using it with CALayer

So now we’ve got some code to draw the shape we want, we need a way to get it onto the screen, using CALayers. There are 3 main ways of providing content for layers:

• Use a delegate that implements drawInContext (and don’t forget to call setNeedsDisplay at least once to cause it be drawn!)
• Set the content to a CGImageRef. Meh… that’s going to mean it’s a bitmap, with all the associated aliasing/scaling issues.
• Subclassing. Nah.

We’ll use the first approach; we can create a simple NSObject-derived class that can manage the layer hierarchy (more of that later) and implement the required selector on it. In that function we can switch on the layer we’re being asked to draw, and do the appropriate handling:

- (void)drawLayer:(CALayer *)theLayer
        inContext:(CGContextRef)context 
{
    if (theLayer == _textLayer) {
        // ...
    } else if (theLayer == _starLayer) {
        // ...
    }
}

Animating

We’re only going to be doing fairly basic rotation animation here, so we can use CABasicAnimation. We use the key-value coding support to specify the transform.rotation property as the target. This is an alias for rotation around the Z axis, which is pointing “out” of the screen. We rotate from 0 to 2*pi radians, repeating essentially indefinitely by specifying a large repeatCount.

    CABasicAnimation *animation = 
        [CABasicAnimation animationWithKeyPath:@"transform.rotation"];
    animation.duration=8.0;
    animation.repeatCount=HUGE_VALF;
    animation.autoreverses=NO;
    animation.fromValue=[NSNumber numberWithFloat:0.0];
    animation.toValue=[NSNumber numberWithFloat:TWOPI];
    [_starLayer addAnimation:animation forKey:@"rotation"];

Drawing the text

This was an interesting one. I originally started looking at a CATextLayer-based solution, but was surprised to find that it’s quite difficult to get vertical alignment within a rectangle. Instead, I decided to use the NSString UIKit additions that provide enough drawing and – importantly – measuring functions for us to work out exactly where we need to place the text.

One important thing to note here is that we’re potentially mixing Core Graphics and UIKit functions here. They have different expectations about how to get hold of the required graphics context; with Core Graphics it’s always passed explicitly, whereas UIKit will grab the current context. This means that if you try and call drawInRect within your drawLayer function, you’ll see errors like “Invalid context: 0×0″ on the console, and no output.

The solution is simple when you know how: tell UIKit about your explicit context, like this:

- (void)drawLayer:(CALayer *)theLayer
        inContext:(CGContextRef)context 
{
    // ...
    // Let UIKit know about this context
    UIGraphicsPushContext(context);
    // Because this function uses it internally...
    [myString drawInRect:r 
                     withFont:font
            lineBreakMode:UILineBreakModeClip 
                    alignment:UITextAlignmentCenter];
    UIGraphicsPopContext();
}

By measuring the text before we draw, we can align it centrally vertically and fill the space horizontally, letting iOS worry about the horizontal alignment. sizeWithFont takes into account a bounding rectangle and our desired breaking/clipping options:

        CGSize sz = [s sizeWithFont:font 
                 constrainedToSize:theLayer.bounds.size 
                     lineBreakMode:UILineBreakModeClip];

Setting up a layer hierarchy

Given that I wanted some parts of the thing to rotate, and others to be static, I needed to create multiple layers and put them together. This is very easily achieved by adding layers to the subLayers collection of our root layer, then we return the root layer and add that to the view.

Layer arrangement

    _star = [[StarLayer alloc] initWithRect:CGRectMake(100, 100, 100, 100)];
    [[self.view.layer] addSublayer:_star.root];

Next Steps

Here’s a red star and random text sitting a bit incongruously in a prototype of a spelling app I’m writing.

It would be good to create a whole load of stars (probably not with text in) and shoot them across the screen in a star-burst by generating a random direction/speed vector and animating their speeds, opacity and scale to make them fade out.

You can check out the code here.

OpenGL

The 3D work I’d done previously on iOS was using OpenGL ES 1.1, so as part of doing this, I also needed to get up to speed with the changes in version 2.0. The move involves replacing the fixed function pipeline approach with the use of shaders crafted in “shader language”, specifically GLSL. This approach is more flexible, but it means you take full responsibility for combining all the relevant matrices (model, projection etc), rather than letting OpenGL do it for you.

There are 2 distinct types of shader programs that operate on either vertices (vertex shaders) or pixels (fragment shaders). The steps required to create shader programs mirror the process of creating “normal” programs: they’re loaded, compiled and linked. Once you’ve got a program loaded, you can initialise it by setting variables of various kinds: attribute, uniform or varying. These are used for vertex and colour data, constants and passing data between vertex and fragment shaders respectively. If you’re interested in learning more, I suggest you check out some of the plethora of internet and dead-tree material on OpenGL. I’ll only be doing pretty basic stuff here.

Bullet

I decided to try using Bullet as the physics engine. It looks very powerful and is widely used in various games and movies; the only bad thing I can say about it is that decent documentation is a bit lacking. I found the best way to get help was googling for API names on the Bullet forum; not a very efficient way of doing things. Maybe I’ve just been spoilt by the copious Apple and MS docs (yeah, yeah).

Building it

Luckily Bullet comes with an Xcode project “in the box”, so with a bit of autoconf magic, it was soon up and building the demo apps. Although there are loads of files in there, the core library itself isn’t too large or complex. As such, I decided to create a separate static library project targeting the iOS simulator (i386) and devices (armv6/7).

API basics

My intention is to use only a small subset of the Bullet API, encompassing rigid bodies and a couple of simple constraint types. For the purposes of the demo, I’m going to create the pretty standard set-up of a large static ground-box, along with a collection of more dynamic objects.

The key part of joining up Bullet with your existing API is the btMotionState class. It enables the API to callback into your code with updated positions at each ‘tick’ of the physics engine, and you can then alter your geometry appropriately. I created a simple class that derives from btMotionState, implements the required pure virtual methods and contains a reference to my world object. Bullet calls the subclassed methods and we pass the transformations on to the object.

class QuadMotionState : public btMotionState {
    Quad *_object;
    btTransform _transform;
public:
    QuadMotionState(const btTransform &initialpos, Quad *node) {
        _object = node;
        _transform = initialpos;
    }
 
    virtual ~QuadMotionState() {
    }
 
    void setNode(Quad *node) {
        _object = node;
    }
 
    virtual void getWorldTransform(btTransform &worldTrans) const {
        worldTrans = _transform;
    }
 
    virtual void setWorldTransform(const btTransform &worldTrans) {
        // Pass the worldTrans to _object
    }
};

Debugging

Now, obviously you will write the perfect code first time, and will have no need for any kind of diagnostics. But I am not that perfect; I need to see the difference between what I’ve told the API to do, and what it’s actually doing. You can do this by creating an object implementing btIDebugDraw and passing it to the (slightly oddly named) setDebugDrawer function on your world:

btIDebugDraw *debug = new DebugDrawHelper();
m_dynamicsWorld->setDebugDrawer(debug);

The most important function on btIDebugDraw is drawLine. It will be called repeatedly to draw axes and bounding boxes in a variety of colours to indicate the object’s state. We can use the Shader we created earlier to give us an easy way of generating these debug visuals. All we need is a very simple shader, so can declare it inline:

program.loadShaders(
		"attribute vec4 position;\n"
		"uniform mat4 projection;\n"
		"void main () {\n"
		"gl_Position = position * projection;\n"
		"}\n", 
 
		"uniform highp vec3 color;\n"
		"void main () {\n"
		"gl_FragColor = vec4(color.r, color.g, color.b, 0.0);\n"
		"}");

Of course, anything more complex than this and you’ll want to put it into a separate file, but this program allows us to pass in the start and end vertex and its colour to draw the line in our implementation of DrawLine:

virtual void drawLine(const btVector3& from,const btVector3& to,const btVector3& color) {
        glUseProgram(program.getProgram());
 
        // Set the projection matrix
        GLint pu = glGetUniformLocation(program.getProgram(), "projection");
        glUniformMatrix4fv(pu, 1, 0, _projection.Pointer());
 
        // Set the colour of the line
        GLint puc = glGetUniformLocation(program.getProgram(), "color");
        vec3 v = vec3(color.getX(), color.getY(), color.getZ());
        glUniform3fv(puc, 1, v.Pointer());
 
        // Set the line vertices
        float tmp[ 6 ] = { from.getX(), from.getY(), from.getZ(),
            to.getX(), to.getY(), to.getZ() };
        glVertexAttribPointer(0, 3, GL_FLOAT, false, 0, &tmp[0]);
        glEnableVertexAttribArray(0);
        glDrawArrays( GL_LINES, 0, 2 );
}

Bullet debug output

Populating the world

So in order to have something nice to look at, we’re gonna need to put some ‘things’ into our world. And in order to make it act like the real world we’ll need to give those things some physical properties.

I created a simple class to render a textured box. This OpenGL object then needs to be represented in the physics worlds by a rigid body object, which in turn contains a collision detection object that closely matches the dimensions of the original object. In fact, in the case of a box, the collision detection object matches exactly, otherwise you’ll need to create an approximation using the more advanced classes. We use a btBoxShape to initialise our btRigidBody instance, along with setting its mass, inertia, and making a link to the state object that I mentioned above.

btRigidBody *addShape(Quad *shape, btScalar mass = 1.0f, float restitution = 0.2f)
{
	// Set the initial transform based on the current location of the object
        btTransform transform;
	transform.setIdentity();
	transform.setOrigin(btVector3(shape->getX(), shape->getY(), 0.));
 
	btCollisionShape* cshape = 
            new btBoxShape(btVector3(btScalar(shape->getWidth()/2),
                                     btScalar(shape->getHeight()/2),
                                     btScalar(shape->getDepth()/2)));
        btVector3 localInertia(0,0,0);
	if (mass != 0.0f)
            cshape->calculateLocalInertia(mass,localInertia);
 
        QuadMotionState *state = new QuadMotionState(transform, shape);
	btRigidBody::btRigidBodyConstructionInfo rbInfo(mass, state, cshape, localInertia);
	btRigidBody* body = new btRigidBody(rbInfo);
 
	body->setDamping(0.85, 0.85);
	body->setRestitution(restitution);
 
        // Add the rigid body to the Bullet physics world
	m_dynamicsWorld->addRigidBody(body);
 
        // Establish the link between our object and the physics object
	shape->setPhysicsObject(body);
	return body;
}

To make things look a bit more polished I added some textures to the quads, along with very simple normal mapping to get some lighting effects. I also added the ability to rotate the view and zoom in and out by using UIGestureRecognizers and changing the camera transform appropriately. The “finishing” touch (yeah, right), is the ability to tap to add a new quad.

The results

Here’s how the final thing looks. It’s very basic, I’ve hardly touched the surface of the Bullet API – there are no constraints in play, for instance – but it’s still kind of fun to play with. It’s running fairly slowly here, what with it being in the simulator, and being recorded. On a physical iPad it’s much snappier.
Let me know if you’d be interested to see some more coverage of some of Bullet’s constraints and hinges features.

Dark Days

We should bear in mind that in some camps, C++ never went away. The financial, scientific and games programming communities have bravely kept the fires burning.

Why? Primarily performance. In these industries performance is a feature, it’s a key differentiator, and it will help you make money. Can you say the same for an enterprise app (there I go, using that term disparagingly again) that just displays a form interface over a relational database?

However, even in the C++ bastions, the language was getting a bad press. I can only speak for the finance industry, where I’ve been working for the past 10 years, but even here the focus briefly switched away from raw performance to complex (overly-complex, dangerously-complex, as hindsight tells us) trades where getting to market first with something new was key: developer productivity was key. Flexibility and generality was key.

In that case, you don’t want to be dealing with the vagaries of subtle errors, crashes and memory leaks that can easily creep in when you’re using a language as low-level as C++. You just want to be able to release quickly. You probably want to be using a managed language.

In fact, C++ was getting a bad press all over. The functional power that was provided in languages like Haskell and F# – and being hastily added to C# – was missing or awkwardly implemented in C++. The provision of things like the .NET thread pool provided programmers on the CLR an easy way to schedule asynchronous tasks, and it was being touted as the only way to survive in the highly-parallel near-future. Of course, it was possible to do this stuff in C++, it was just more difficult.

Despite this, the programming community widely understood the power that C++ brings, but also that it has to be wielded carefully. Witness the oft-repeated jokes about the fact that when you shoot yourself in the foot with C++, you tend to blow your leg off. There were even tensions within Microsoft, with the pro-managed DevDiv vs the hardcore, native-favouring Windows division.

The New Order

So what’s changed? Herb Sutter’s recent talk covered many of the reasons why C++ is relevant again, and the most obvious reason is the emergence and importance of mobile devices and data centres.

They have a completely different set of requirements than the desktop: they focus on squeezing the best experience out of every available cycle (and corresponding minute of battery life), and the most performance out of every watt of data centre power per degree of cooling. We’re seeing the impact of the smartphone in your pocket and the data-centric web and social media reflecting back onto the languages we use to write software.

Managed languages fall down in some important areas here. They sacrifice memory efficiency to the god of garbage collection and they offer fewer opportunities for aggressive compiler optimisations, including, in the case of .NET, making use of chip-provided performance features like SIMD instructions.

Nowadays in more and more of the computing industry, it’s the hardware that’s driving the development choices, not the other way around.

With resource-constricted platforms, you want to be doing more of the heavy lifting offline, at compile time, not at run-time with a JIT compiler.

How the iPhone showed devs don’t care

While (parts of) Microsoft were busy pushing the managed languages hard, Apple was quietly producing game-changing pieces of hardware that the device-carrying public were buying in their hordes. And they weren’t just buying devices, they were buying apps to play on these devices – in their millions. Lots of developers wanted a piece of this action but – quelle horreur! – You had to use an obscure variant of C to create them.

But – surprise, surprise – developers did it anyway. Luckily, they weren’t on their own, but this time rather than a managed runtime, they had an API to use. And they had to use it; it was, and is, literally a condition of sale that your app uses only the APIs that Apple provide and in the way that they intended. Whatever your view on the fairness of this approach, having a restricted surface area and well defined set of libraries helps developers win back some of the productivity they might otherwise have sacrificed by using Objective-C.

Microsoft must’ve been looking askance at this, wondering how Apple had so effectively switched the model around. MS had been based on the idea of developer productivity being king and the hardware being utterly insignificant. Now developers were writing code in a language that was broadly derided by people in the programming community, yet they were in a virtuous circle where writing apps helped to ship hardware that increased the market for their apps.

We need libraries

Is it possible to find a good balance between the raw power of C++ and the productivity of managed languages? I think it is; and partly it comes down to having a good set of libraries. When I say good, I mean consistent, relevant, well-documented, easily available set of libraries. Can you see where C++ may have struggled here before?

For the low-level, programming task oriented basics, Boost is the answer. Seriously, Boost always seems to be the answer to questions of “how do I do X in C++?” where X is a modern programming technique. Smart pointers, lambdas, higher-order functions… they’re all there.

A few years ago if you were working in C++, you didn’t have many options apart from vanilla STL, Boost, and writing-it-yourself (again). Now, the STL and the language itself is richer and anecdotally, organisations are more receptive to the use of Boost. So hopefully, this combination gives us back access to most of the functionality that’s available to managed developers in their pre-packaged base class libraries.

C++ and Apps

But how is that going to help people use it to produce compelling apps and “experiences” (cough) and specifically, how is Microsoft going to use its developer productivity nous to keep Windows relevant in the new hardware markets?

They need something else; something like the Apple Core frameworks. It looks like this is what Microsoft is talking about with Windows 8 and the Windows Runtime. It’s suddenly not embarrassed to admit that, hey, Windows itself is written in C++, so maybe you shouldn’t have to go through a managed ‘interpretation’ of the existing APIs, one that’s exclusively available to managed languages.

Of course, Microsoft can’t do anything as radical as forcing all developers targeting a mobile or tablet platform to use C++. For them, programmers, rather than any particular hardware, are the bread-and-butter, so they tend to have a slightly less dictator-style relationship with devs than Apple does.

Judging from what I’ve seen from Build, Microsofts chosen approach is to provide language-specific “projections” (their term) from the Windows Runtime layer into a variety of languages, including C++. The native C++ types are mapped into corresponding concepts in the vernacular of the higher level languages; Javascript, C# etc. They’ve gone to some lengths, it seems, to ensure that transitions between levels are efficient. Using C++ at both levels should be the fast path, and of course you can mix it freely with code that, for example, uses Boost or other C++ libraries.

Additionally and importantly, as well as basic types, collections etc, the Windows Runtime also contains all of the rich APIs for accessing app-level functionality; things like image capture, sound and geolocation. So suddenly, all of this stuff is available in all its glory directly from C++ – and other languages too via cheap bridges. What they’ve essentially got is an identical copy of the architecture that exists in the iOS world, while still attempting to keep non C++ developers on-side.

The Return…?

So, have Microsoft managed to bring C++ in from the cold? They seem to have admitted, at least to themselves, that developers will need to use it in order to effectively target some platforms, and they’ve used their tooling experience to make it look and feel like a managed language in their IDEs, as well as leveraging their compiler technology to do some additional useful performance tricks. It feels like C++ was a fundamental concern in the redesign of the new alternative to the creaking Win32 API.

But will the transition be something to make current developers switch to C++, especially those who’ve been used to living in the cosseted managed world? Perhaps, if the Microsoft-based tablet market becomes as compelling – read, large – as Apple’s. Otherwise I don’t see any reason why developers who don’t have specific platform or other runtime requirements would move; for the majority of environments, developer time is still the dominant cost, so developer productivity will still be paramount.

The gap between managed and unmanaged languages has certainly narrowed in that area, but it still exists. C++ is still proving surprisingly relevant, despite its 30+ years, and hopefully its use in new and different areas of computing will feed back into improvements in the language and libraries for everyone’s benefit.

Just what do you think you're doing, Dave?

The first bit was easy: I’m “lucky” enough to have one of the older Xboxes, which meant I’d had to get a Kinect with separate power, which is the one required by the SDK. Now all I needed was a Windows machine to develop on.

For all my Visual Studio stuff at home I use a virtual machine, and unfortunately I missed the point in the readme about “Kinect for Windows applications cannot run in a virtual machine”. Doh. That would explain why, try as I might I couldn’t get VirtualBox to detect the device when I plugged it into the host. Whatever I did I ended up with a ‘resource is busy’ error.

I even tried another VM, this time from VMWare. It got further, with the guest seeing the devices, but whenever I tried to run the sample apps the API initialisation call failed. Unfortunately the samples make a bit of a rookie mistake and don’t display the error code associated with the failure; the only way to get the underlying HRESULT is to debug the app. As it turned out the error was the catch-all 80080014, which is attributed to various USB issues.

So, I finally relented and decided to once again set-up Bootcamp, which I’d stopped using a while back: why bother when VMs had done everything I needed? Again I was snookered, this time by the incompatibility of the EFI firmware in my ageing MacBook with the Win7 x64 install disc (which I’d had to actually burn onto optical media – can’t remember the last time I burnt a real physical disk)!

Eventually, I got Win7 x86, Visual Studio and the Kinect SDK installed on the metal, plugged in the sensor and – whoa – the devices were recognised and the drivers installed and… the samples ran!

From this point doing some hacking was pretty straightforward. I set about creating a project that used the skeletal tracking ability from the SDK. The project needs to reference the Microsoft.Research.Kinect assembly, then we can open some namespaces and initialise the API:

open System
open Microsoft.Research.Kinect
 
let nui = Nui.Runtime()
nui.Initialize(Nui.RuntimeOptions.UseSkeletalTracking)

The API uses .NET events to communicate back to the application that some form of data is available. Depending on the options that you specify at init time, any of the skeletal, depth frame or colour data will be returned in the event arguments. This seems like a good place to use F# Async’s event integration: Async.AwaitEvent. We can quite easily write some code that will create an Async task that will repeatedly listen for events:

    let skeleton (nui : Nui.Runtime) =
        let rec loop () =
            async { 
                let! args = Async.AwaitEvent nui.SkeletonFrameReady
                // Do something with the args
                return! loop () 
            }
        loop ()
    // Start the process of listening for events...
    skeleton nui |> Async.Start

This seems to work nicely, although I’m not sure of the overhead involved, perhaps a solution involving Rx would be more appropriate. Anyway, let’s do something with the data we get in the event, like getting all the skeletons that are being tracked, and passing them to a draw function:

                args.SkeletonFrame.Skeletons 
                |> Seq.filter (fun s -> s.TrackingState = Nui.SkeletonTrackingState.Tracked)
                |> Seq.iter draw

All that’s left is doing something cool in draw! Oh, and setting up all the GUI stuff necessary to actually get some pixels on the screen. I went for the WPF approach (as the managed samples do), which involves creating a simple object tree to display a rectangle in a canvas in a grid in a window, and doing a bit of marshalling back from the thread pool (where our Async code runs) to the main GUI thread.

Here’s all the code for possibly the most tedious thing you could do with a Kinect! But hey, it’s a starting point, right? I’m sure at some point I’ll get around to doing something cooler than a couple of small red rectangles with it…

open System
open System.Windows
open Microsoft.Research.Kinect
 
[<STAThread>]
do
    let nui = Nui.Runtime()
    nui.Initialize(Nui.RuntimeOptions.UseSkeletalTracking)
 
    // lifted straight from the sample code
    let getDisplayPosition w h (joint : Nui.Joint) =
            let mutable depthX = 0.0f
            let mutable depthY = 0.0f
            nui.SkeletonEngine.SkeletonToDepthImage(joint.Position, &depthX, &depthY)
            let depthX = depthX * 320.0f; //convert to 320, 240 space
            let depthY = depthY * 240.0f; //convert to 320, 240 space
            let mutable colorX = 0
            let mutable colorY = 0
            let iv = new Nui.ImageViewArea()
            // only ImageResolution.Resolution640x480 is supported at this point
            nui.NuiCamera.GetColorPixelCoordinatesFromDepthPixel(Nui.ImageResolution.Resolution640x480, iv, 
                (int)depthX, (int)depthY, 0s, &colorX, &colorY)
 
            // map back to visible area
            new Point((w * (float)colorX / 640.), (h * (float)colorY / 480.))
 
    // Set-up the WPF window and its contents
    let width = 1024.
    let height = 768.
    let w = Window(Width=width, Height=height)
    let g = Controls.Grid()
    let c = Controls.Canvas()
    let hd = Shapes.Rectangle(Fill=Media.Brushes.Red, Width=10., Height=10.)
    ignore <| c.Children.Add hd
    ignore <| g.Children.Add c
    w.Content <- g
 
    // We simple move the rectangle to where the head is
    let draw (s : Nui.SkeletonData) =
        let point = getDisplayPosition width height s.Joints.[Nui.JointID.Head]
        ignore <| w.Dispatcher.BeginInvoke(Threading.DispatcherPriority.Normal, Action(fun () ->    
            hd.SetValue(Controls.Canvas.TopProperty, point.Y)
            hd.SetValue(Controls.Canvas.LeftProperty, point.X)))
 
    let skeleton (nui : Nui.Runtime) =
        let rec loop () =
            async { 
                let! args = Async.AwaitEvent nui.SkeletonFrameReady
                args.SkeletonFrame.Skeletons 
                |> Seq.filter (fun s -> s.TrackingState = Nui.SkeletonTrackingState.Tracked)
                |> Seq.iter draw
                return! loop () 
            }
        loop ()
 
    skeleton nui |> Async.Start
 
    let a = System.Windows.Application()
    ignore <| a.Run(w)
 
    nui.Uninitialize()

The moral of the story? Don’t be like me, and make sure you actually read the FAQing readme, then maybe you’ll spend more time doing a decent demo and less time plugging and unplugging your Kinect!

Lock; literal images 'r' us

One of the functions that immediately caught my eye was LdrLockLoaderLock. I’d previously spent quite a few frustrating hours trying to figure out how to determine whether some code was being executed from DllMain, i.e. while in the loader lock, so I could avoid doing anything dodgy – or indeed, anything at all.

The case I was looking at was some logging library code that was used, amongst other things, to record the fact that DLLs were being unloaded. Unfortunately when this was called from DllMain, it sometimes caused a deadlock, for all the reasons we already know about. The library code was called from lots of DLLs, so it wasn’t feasible to fix all of the call sites, instead I had to make the logging a no-op when it’s not safe.

I’m embarrassed to say that my previous attempt to detect the lock involved some pretty heinous hackery. I worked out the memory address (the offset within ntdll.dll) where the loader lock critical section is located, cast that bit of memory to a CRITICAL_SECTION and tested it. I even had to provide the ability to change the offset based on the version of ntdll being used, in a vain attempt to reduce its fragility. Ouch. It was very nasty, and to be honest although it worked in the cases where I tested it, I was reluctant to release it.

Luckily, along comes LdrLockLoaderLock to save my blushes. It appears to give me exactly the functionality I need; you can pass a flag to tell it to return immediately if the lock’s already been taken, and there’s a status parameter that can be used to tell if you got the lock – whereupon you can call the corresponding LdrUnlockLoaderLock. Nice!

I wonder if this is what’s used by the Visual Studio loader lock managed debugging assistant to determine if your managed code is being run under the loader lock?

module Test = 
    let work i = 
        async { 
            System.Threading.Thread.Sleep(500) 
            return i+1 
        }
    let run _ = 
        [1..1000] |> List.map work 
        |> Async.Parallel 
        |> Async.RunSynchronously

(Waits for people to start FSI and paste in the code…)

My guess would’ve been something just over 500ms; each of the 1000 async tasks would surely sleep in parallel, and then the operation itself is trivial. The additional elapsed time would be dominated by the overhead of thread management, and depend on the number of threads that can physically run in parallel (I’m using an 8-core machine). But still, something close to 500ms…

The actual result? 28000ms. Yes, you read that right: 28 seconds. What on earth did we do wrong?

Surely it can’t be that Sleep, right? I mean, each task is going to run on the thread pool anyway, so what difference would that make? Well, that *is* exactly the problem. It turns out that you can’t mix “vanilla” threading calls with Async API ones, because Async works in a completely different and incompatible way. If we use raw threading functions like Sleep, the async framework has no way of knowing if we’re doing anything useful. It eventually ends up causing .NET thread pool starvation, as we’re essentially grabbing and holding each thread in a way that doesn’t cooperate with anything else. New threads will spin-up to compensate and we’ll end up getting additional memory usage as well as worse performance.

Nasty

Instead Async uses what’s called continuation passing style, a technique favoured by the functional programming community, that also happens to map quite nicely to features in the CLR, including the thread pool, tail calls and garbage collection. It has the rather nice feature that it never blocks an OS thread (unless it’s busy actually executing). Instead it queues a function (a continuation) to be called at some later point.

Nice

For all the gory details and full Async semantics, I suggest you read the paper from Microsoft Research. Tucked away in there you’ll see that they do mention this restriction:

The key facet of an asynchronous I/O primitive is that it does not block an OS thread while executing, but instead schedules the continuation of the asynchronous computation as a callback in response to an event.

So how do we “fix” our example? Simple: we replace the non-CPS-friendly explicit Sleep, with an Async-native Sleep.

module Test = 
    let work i = 
        async { 
            do! Async.Sleep(500) 
            return i+1 
        }
    let run _ = 
        [1..1000] |> List.map work 
        |> Async.Parallel 
        |> Async.RunSynchronously

The result now: 558ms. Much more like it!

What happens is that the Async.Sleep call is converted into an event that is fired after 500ms to continue the execution, i.e. actually perform the addition. In the meantime, each of the other instances of the workflow get a chance to run, and we get real parallelism.

In this case it might look screamingly obvious, but be aware if you’re doing something such as calling into a third-party, legacy, or otherwise non-Async-aware code, it could cause your otherwise highly parallel code to choke. You’ll have to wrap it up in order to make it play nicely with Async and make sure you get the full benefit of this powerful F# feature.

Simples?

Of course the idiomatic way to avoid problems at the boundary between your F# code and the outside world is to ensure that you only expose a small set of compatible types. This works pretty well if your clients are also .NET languages. For instance you can do things like exposing your collections as seq, rather than say, a native F# list, and this will mean your collections can be consumed as IEnumerable. The only problem is it means you’ve got the added burden of maintaining this mapping layer, because you’ll no doubt want to use the F# “native” types internally.

So, what options do we have if some of our F# types happen to leak into our public API? Luckily, lots. Let’s take a look at how some of the common F# constructs can be called from C#.

Unions

I talked about the power of discriminated unions a while back, so how do they map to C#? Interestingly the F# compiler generates a very simple OO type hierarchy, with an abstract base class that each of the cases derive from. This will mean that your calling code will need to use some casting, but don’t worry, you’re not back in the hell of runtime typing; there are a set of methods on the type that you can use to identify the case before you do the cast, so at least you get some compile time sanity checking.

Here’s a simple discriminated union that we’ll be using as an example:

    type TestDU =
        | One of int * int
        | Two of string
        | Three

If we wanted to use this type in F#, we’d do so like this:

    // A function 'f' that takes an instance 't' of our union type
    let f t = match t with One (a,b) -> printf "Got One" | _ -> ()
 
    // Apply our function with a particular TestDU union case
    let _ = f <| Two "foo"

That’s pretty natural, as you’d expect. If we exposed the function f to C#, it looks like this:

void Module1.f(Module1.TestDU t)

This means we have to create an instance of the TestDU type, but we’ll quickly discover it’s abstract. As I mentioned before, the F# compiler has generated a very simple class hierarchy, with the union cases being subtypes of TestDU, implemented as nested types (which are used quite extensively by the F# compiler). But if we try and instantiate one of these subtypes…?

The union case types have no constructors.

    var one = Module1.TestDU.NewOne(1, 2);
    var two = Module1.TestDU.NewTwo("foo");

And slightly oddly, the parameterless cases (Three in our example) use a property on the base class instead:

    var three = Module1.TestDU.Three;

So now we can create instances of the union type to pass to the F# function.

But what if we’re returned one instead? It will be of type TestDU, so how do we know what case it actually is? And how do we get to the parameters? Well, as we know it’s going to be one of the subtypes, we could use the C# ‘is’ operator to determine its identity, but the base class also provides helper methods to do it for us, prefixed with Is, e.g.:

    if (two.IsTwo)
    {
        var x = (two as Module1.TestDU.Two).Item;
        // Do something with x
    }

As you can see, Item contains the case value. For tupled types this becomes Item1, Item2 etc.

Records

Records are immutable types that contain fields, they tend to be used quite a lot in F# code, and luckily using them from C# is very straightforward. The only thing to remember is that they’re only constructor is the one taking all of the field values (because they’re immutable), e.g.:

    type TestRecord =
        {
        field1 : int
        field2 : string
        }

Can be instantiated like this in C#:

    var r = new Module1.TestRecord(1, "foo");

Functions

It’s usual in F# to use curried functions. That is, functions that can be applied one argument at a time, returning a partially applied function. It turns out that the compiler maps this type of function (when directly exposed, as a public function on a module, say) into its tupled equivalent, which makes them easily callable directly from C#. For instance:

let public adder a b = a + b

Has a native F# signature of int -> int -> int, but results in a generated CIL function taking a tuple of two ints, with signature more like int * int -> int.

But there are more complicated cases; F# functions can take functions and return them. When we’re dealing with functions in a first-class way like this, we can use the generic Microsoft.FSharp.Core.FSharpFunc type. For example:

    let public adder a b = a + b
    let public addOne = adder 1

Where addOne is a function that returns a partially applied function. Calling it from F# is a breeze:

    let x = addOne 100

But from C# it’s a bit trickier. The function returns an FSharpFunc, that we then need to apply with Invoke:

    Microsoft.FSharp.Core.FSharpFunc<int,int> f = Module1.addOne;
    var x = f.Invoke(100);

Although obviously you can combine this into a single line, and remove some of the type signatures (which I tend to do as much as possible using var when writing C# code):

    var x = Module1.addOne.Invoke(100);

Things get even nastier if you happen to expose an F# function that takes a function. For example, a function that takes another function and applies it to two arguments:

    let apply op a b = op a b

In F# interactive we can see that this has the following, nicely generalised type signature:

    val apply : ('a -> 'b -> 'c) -> 'a -> 'b -> 'c

i.e. it’s a function that takes as its first argument a function that takes an ‘a and a ‘b and returns a ‘c, it also accepts the ‘a and ‘b to pass to this function, and returns the ‘c result. Clear as. We may well want to create a version of this that uses the integer operator +, which is trivial from F#:

    let add a b = apply (+) a b

So how does it look if we try and do that from C#? In a word: messy. We have to use the Microsoft.FSharp.Core.FuncConvert.ToFSharpFunc helper functions, along with the .NET framework Converter as a way of specifying a generic, value-returning function.

    using Microsoft.FSharp.Core;
...
    var op = FuncConvert.ToFSharpFunc(new Converter<int, FSharpFunc<int, int>>((aa) =>
        {
            return FuncConvert.ToFSharpFunc(new Converter<int, int>((bb) =>
            {
                // The 'meat' of the function. Hmmm...
                return aa + bb;
            }));
        }));
    var zz = Module1.apply(op, 1, 2);

Whoa. That’s enough for now. As you can see, the various different ways of interoperating between F# and C# range from the neat to the nasty. The trick is definitely to pick your battles. Think carefully about what you need to expose and remember it’s probably best not to cross the beams if you can avoid it.