Star and text

Drawing the basic shape

The first step is to draw the ’star’ shape. I decided to create a re-usable path, although at the minute I’m actually only drawing it once, so the benefits are limited. Doing this involves using the CGPath... versions of the Core Graphics functions, that add themselves to a mutable path objects, rather than being drawn directly on the currently active graphics context.

    CGMutablePathRef r = CGPathCreateMutable();
    // Always have to move to a start point
    CGPathMoveToPoint(r, NULL, x1, y1); 
    CGPathAddLineToPoint(r, NULL, x2, y2);
    CGPathCloseSubpath(r);

Once you’ve got created the path you can merrily set various stroke and fill colours and draw it over-and-over to your hearts content.

The star drawing is parameterised in a few ways: we can set the inner radius (r2), outer radius (r1) and the number of points, which must be divisible by 3. We draw the points in groups of three, the first on the outer radius, then the inner, then the last on the outer. In that way the sections tesselate into a complete circle.

Using it with CALayer

So now we’ve got some code to draw the shape we want, we need a way to get it onto the screen, using CALayers. There are 3 main ways of providing content for layers:

• Use a delegate that implements drawInContext (and don’t forget to call setNeedsDisplay at least once to cause it be drawn!)
• Set the content to a CGImageRef. Meh… that’s going to mean it’s a bitmap, with all the associated aliasing/scaling issues.
• Subclassing. Nah.

We’ll use the first approach; we can create a simple NSObject-derived class that can manage the layer hierarchy (more of that later) and implement the required selector on it. In that function we can switch on the layer we’re being asked to draw, and do the appropriate handling:

- (void)drawLayer:(CALayer *)theLayer
        inContext:(CGContextRef)context 
{
    if (theLayer == _textLayer) {
        // ...
    } else if (theLayer == _starLayer) {
        // ...
    }
}

Animating

We’re only going to be doing fairly basic rotation animation here, so we can use CABasicAnimation. We use the key-value coding support to specify the transform.rotation property as the target. This is an alias for rotation around the Z axis, which is pointing “out” of the screen. We rotate from 0 to 2*pi radians, repeating essentially indefinitely by specifying a large repeatCount.

    CABasicAnimation *animation = 
        [CABasicAnimation animationWithKeyPath:@"transform.rotation"];
    animation.duration=8.0;
    animation.repeatCount=HUGE_VALF;
    animation.autoreverses=NO;
    animation.fromValue=[NSNumber numberWithFloat:0.0];
    animation.toValue=[NSNumber numberWithFloat:TWOPI];
    [_starLayer addAnimation:animation forKey:@"rotation"];

Drawing the text

This was an interesting one. I originally started looking at a CATextLayer-based solution, but was surprised to find that it’s quite difficult to get vertical alignment within a rectangle. Instead, I decided to use the NSString UIKit additions that provide enough drawing and – importantly – measuring functions for us to work out exactly where we need to place the text.

One important thing to note here is that we’re potentially mixing Core Graphics and UIKit functions here. They have different expectations about how to get hold of the required graphics context; with Core Graphics it’s always passed explicitly, whereas UIKit will grab the current context. This means that if you try and call drawInRect within your drawLayer function, you’ll see errors like “Invalid context: 0×0″ on the console, and no output.

The solution is simple when you know how: tell UIKit about your explicit context, like this:

- (void)drawLayer:(CALayer *)theLayer
        inContext:(CGContextRef)context 
{
    // ...
    // Let UIKit know about this context
    UIGraphicsPushContext(context);
    // Because this function uses it internally...
    [myString drawInRect:r 
                     withFont:font
            lineBreakMode:UILineBreakModeClip 
                    alignment:UITextAlignmentCenter];
    UIGraphicsPopContext();
}

By measuring the text before we draw, we can align it centrally vertically and fill the space horizontally, letting iOS worry about the horizontal alignment. sizeWithFont takes into account a bounding rectangle and our desired breaking/clipping options:

        CGSize sz = [s sizeWithFont:font 
                 constrainedToSize:theLayer.bounds.size 
                     lineBreakMode:UILineBreakModeClip];

Setting up a layer hierarchy

Given that I wanted some parts of the thing to rotate, and others to be static, I needed to create multiple layers and put them together. This is very easily achieved by adding layers to the subLayers collection of our root layer, then we return the root layer and add that to the view.

Layer arrangement

    _star = [[StarLayer alloc] initWithRect:CGRectMake(100, 100, 100, 100)];
    [[self.view.layer] addSublayer:_star.root];

Next Steps

Here’s a red star and random text sitting a bit incongruously in a prototype of a spelling app I’m writing.

It would be good to create a whole load of stars (probably not with text in) and shoot them across the screen in a star-burst by generating a random direction/speed vector and animating their speeds, opacity and scale to make them fade out.

You can check out the code here.

OpenGL

The 3D work I’d done previously on iOS was using OpenGL ES 1.1, so as part of doing this, I also needed to get up to speed with the changes in version 2.0. The move involves replacing the fixed function pipeline approach with the use of shaders crafted in “shader language”, specifically GLSL. This approach is more flexible, but it means you take full responsibility for combining all the relevant matrices (model, projection etc), rather than letting OpenGL do it for you.

There are 2 distinct types of shader programs that operate on either vertices (vertex shaders) or pixels (fragment shaders). The steps required to create shader programs mirror the process of creating “normal” programs: they’re loaded, compiled and linked. Once you’ve got a program loaded, you can initialise it by setting variables of various kinds: attribute, uniform or varying. These are used for vertex and colour data, constants and passing data between vertex and fragment shaders respectively. If you’re interested in learning more, I suggest you check out some of the plethora of internet and dead-tree material on OpenGL. I’ll only be doing pretty basic stuff here.

Bullet

I decided to try using Bullet as the physics engine. It looks very powerful and is widely used in various games and movies; the only bad thing I can say about it is that decent documentation is a bit lacking. I found the best way to get help was googling for API names on the Bullet forum; not a very efficient way of doing things. Maybe I’ve just been spoilt by the copious Apple and MS docs (yeah, yeah).

Building it

Luckily Bullet comes with an Xcode project “in the box”, so with a bit of autoconf magic, it was soon up and building the demo apps. Although there are loads of files in there, the core library itself isn’t too large or complex. As such, I decided to create a separate static library project targeting the iOS simulator (i386) and devices (armv6/7).

API basics

My intention is to use only a small subset of the Bullet API, encompassing rigid bodies and a couple of simple constraint types. For the purposes of the demo, I’m going to create the pretty standard set-up of a large static ground-box, along with a collection of more dynamic objects.

The key part of joining up Bullet with your existing API is the btMotionState class. It enables the API to callback into your code with updated positions at each ‘tick’ of the physics engine, and you can then alter your geometry appropriately. I created a simple class that derives from btMotionState, implements the required pure virtual methods and contains a reference to my world object. Bullet calls the subclassed methods and we pass the transformations on to the object.

class QuadMotionState : public btMotionState {
    Quad *_object;
    btTransform _transform;
public:
    QuadMotionState(const btTransform &initialpos, Quad *node) {
        _object = node;
        _transform = initialpos;
    }
 
    virtual ~QuadMotionState() {
    }
 
    void setNode(Quad *node) {
        _object = node;
    }
 
    virtual void getWorldTransform(btTransform &worldTrans) const {
        worldTrans = _transform;
    }
 
    virtual void setWorldTransform(const btTransform &worldTrans) {
        // Pass the worldTrans to _object
    }
};

Debugging

Now, obviously you will write the perfect code first time, and will have no need for any kind of diagnostics. But I am not that perfect; I need to see the difference between what I’ve told the API to do, and what it’s actually doing. You can do this by creating an object implementing btIDebugDraw and passing it to the (slightly oddly named) setDebugDrawer function on your world:

btIDebugDraw *debug = new DebugDrawHelper();
m_dynamicsWorld->setDebugDrawer(debug);

The most important function on btIDebugDraw is drawLine. It will be called repeatedly to draw axes and bounding boxes in a variety of colours to indicate the object’s state. We can use the Shader we created earlier to give us an easy way of generating these debug visuals. All we need is a very simple shader, so can declare it inline:

program.loadShaders(
		"attribute vec4 position;\n"
		"uniform mat4 projection;\n"
		"void main () {\n"
		"gl_Position = position * projection;\n"
		"}\n", 
 
		"uniform highp vec3 color;\n"
		"void main () {\n"
		"gl_FragColor = vec4(color.r, color.g, color.b, 0.0);\n"
		"}");

Of course, anything more complex than this and you’ll want to put it into a separate file, but this program allows us to pass in the start and end vertex and its colour to draw the line in our implementation of DrawLine:

virtual void drawLine(const btVector3& from,const btVector3& to,const btVector3& color) {
        glUseProgram(program.getProgram());
 
        // Set the projection matrix
        GLint pu = glGetUniformLocation(program.getProgram(), "projection");
        glUniformMatrix4fv(pu, 1, 0, _projection.Pointer());
 
        // Set the colour of the line
        GLint puc = glGetUniformLocation(program.getProgram(), "color");
        vec3 v = vec3(color.getX(), color.getY(), color.getZ());
        glUniform3fv(puc, 1, v.Pointer());
 
        // Set the line vertices
        float tmp[ 6 ] = { from.getX(), from.getY(), from.getZ(),
            to.getX(), to.getY(), to.getZ() };
        glVertexAttribPointer(0, 3, GL_FLOAT, false, 0, &tmp[0]);
        glEnableVertexAttribArray(0);
        glDrawArrays( GL_LINES, 0, 2 );
}

Bullet debug output

Populating the world

So in order to have something nice to look at, we’re gonna need to put some ‘things’ into our world. And in order to make it act like the real world we’ll need to give those things some physical properties.

I created a simple class to render a textured box. This OpenGL object then needs to be represented in the physics worlds by a rigid body object, which in turn contains a collision detection object that closely matches the dimensions of the original object. In fact, in the case of a box, the collision detection object matches exactly, otherwise you’ll need to create an approximation using the more advanced classes. We use a btBoxShape to initialise our btRigidBody instance, along with setting its mass, inertia, and making a link to the state object that I mentioned above.

btRigidBody *addShape(Quad *shape, btScalar mass = 1.0f, float restitution = 0.2f)
{
	// Set the initial transform based on the current location of the object
        btTransform transform;
	transform.setIdentity();
	transform.setOrigin(btVector3(shape->getX(), shape->getY(), 0.));
 
	btCollisionShape* cshape = 
            new btBoxShape(btVector3(btScalar(shape->getWidth()/2),
                                     btScalar(shape->getHeight()/2),
                                     btScalar(shape->getDepth()/2)));
        btVector3 localInertia(0,0,0);
	if (mass != 0.0f)
            cshape->calculateLocalInertia(mass,localInertia);
 
        QuadMotionState *state = new QuadMotionState(transform, shape);
	btRigidBody::btRigidBodyConstructionInfo rbInfo(mass, state, cshape, localInertia);
	btRigidBody* body = new btRigidBody(rbInfo);
 
	body->setDamping(0.85, 0.85);
	body->setRestitution(restitution);
 
        // Add the rigid body to the Bullet physics world
	m_dynamicsWorld->addRigidBody(body);
 
        // Establish the link between our object and the physics object
	shape->setPhysicsObject(body);
	return body;
}

To make things look a bit more polished I added some textures to the quads, along with very simple normal mapping to get some lighting effects. I also added the ability to rotate the view and zoom in and out by using UIGestureRecognizers and changing the camera transform appropriately. The “finishing” touch (yeah, right), is the ability to tap to add a new quad.

The results

Here’s how the final thing looks. It’s very basic, I’ve hardly touched the surface of the Bullet API – there are no constraints in play, for instance – but it’s still kind of fun to play with. It’s running fairly slowly here, what with it being in the simulator, and being recorded. On a physical iPad it’s much snappier.
Let me know if you’d be interested to see some more coverage of some of Bullet’s constraints and hinges features.

_toplayer = [CALayer layer];

Splitting images

Now we’ve got the layers, we need something to but into them. I didn’t want to create lots of pre-processed images for the top and bottom sections of each number, so instead I wrote a routine to split the images in code, drawing each half into an image context and grabbing a UIImage from it. This is then stored in an array for later use. We can get any of the elements from the array and set it as the content of the layer as required.

// The size of each part is half the height of the whole image
CGSize size = CGSizeMake([img size].width, [img size].height/2);
 
// Create image-based graphics context for top half
UIGraphicsBeginImageContext(size);
 
// Draw into context, bottom half is cropped off
[img drawAtPoint:CGPointMake(0.0,0.0)];
// Grab the current contents of the context as a UIImage 
// and add it to our array
UIImage *top = [UIGraphicsGetImageFromCurrentImageContext() retain];			
[_imagesTop addObject:(id)top];
 
// Now draw the image starting half way down, to get the bottom half
[img drawAtPoint:CGPointMake(0.0,-[img size].height/2)];
// And store that image in the array too
UIImage *bottom = [UIGraphicsGetImageFromCurrentImageContext() retain];			
[_imagesBottom addObject:(id)bottom];
 
UIGraphicsEndImageContext();

Now our _imagesTop and _imagesBottom arrays contains all the images we need, let’s get ‘em moving.

Applying animation

The axes and anchor point of a CALayer

_fliplayer.anchorPoint = CGPointMake(0.5, 1.0);

I use the CATransform3DMakeRotation function to create a transform around the X axis using the vector [1,0,0]. For the first part of the motion, we go from 0 radians to pi/2 radians (0 to 90 degrees), using an explicit CABasicAnimation object:

CABasicAnimation *topAnim = [CABasicAnimation animationWithKeyPath:@"transform"];
topAnim.duration=0.25;
topAnim.repeatCount=1;
topAnim.fromValue= [NSValue valueWithCATransform3D:CATransform3DMakeRotation(0.0, 1, 0, 0)];
float f = -M_PI/2;
topAnim.toValue=[NSValue valueWithCATransform3D:CATransform3DMakeRotation(f, 1, 0, 0)];
topAnim.delegate = self;
topAnim.removedOnCompletion = FALSE;
[_fliplayer addAnimation:topAnim forKey:[NSString stringWithUTF8String:k_TopAnimKey]];

Drawing layer content

Unfortunately a CALayer doesn’t have a “back”, so creating the flip layer is not as easy as it could be. We need to add a step in the process to change the image displayed by the flip layer half way through its fall. We can do this in a variety of ways, I’ve chosen to set a delegate object to perform the drawing of the layer contents:

[_fliplayer setDelegate:_layerHelper];

This means our delegate object (_layerHelper) has its drawLayer method called whenever the layer needs to display its contents. Be wary though, this may not be as often as you think! Core graphics aggressively caches the contents of the layer, and will only call your delegate when absolutely necessary. You can force it to happen by calling setNeedsDisplay, which invalidates the contents of the layer:

[_fliplayer setNeedsDisplay];

I encapsulated this in the helper object itself, by providing a custom implementation of the isTop property setter that calls setNeedsDisplay. At least that way callers don’t have to be aware of this requirement.

The drawLayer method itself simply draws one of two images into the context, depending on whether it’s in ‘top’ mode. A slight further complication is that when drawing the bottom part, the image needs to be inverted to display upside down. This is achieved by applying a translation and scale to the context before drawing the image:

CGContextSaveGState(context);
CGContextTranslateCTM(context, 0.0f, r.size.height);
CGContextScaleCTM(context, 1.0f, -1.0f);
 
CGContextDrawImage(context, r, [_imgTop CGImage]);
 
CGContextRestoreGState(context);

Avoiding implicit animation

I’ve tried to avoid using implicit animation here, instead using explicit construction of CABasicAnimation objects, because we need some control of when things happen, and the order in which they happen. Normally, it would be easier to just set some property of the view and let the OS manage animating it. Many properties have default transition that are applied whenever they’re changed, so it takes some additional effort to make them change immediately.

In our example we change the contents of the top half and show the flip layer, but we need that to happen immediately so we disable actions within an explicit transaction that we then commit. This gives us the control we need, and avoids the default behaviour of the transition occurring when the message loop next runs.

[CATransaction begin];
[CATransaction setValue:(id)kCFBooleanTrue
				forKey:kCATransactionDisableActions];
_toplayer.contents = (id)[[_imagesTop objectAtIndex:[self getNextIndex]] CGImage];
_fliplayer.hidden = NO;
[CATransaction commit];

Getting some perspective

Although the images I’ve used to illustrate this post are isometric, what we really want is a front-on perspective view, so that the falling piece seems to stick out. For this we have to set a sublayerTransform on the view’s inherent layer, by setting an individual element on a transform:

CATransform3D aTransform = CATransform3DIdentity;
float zDistance = 1000;
aTransform.m34 = 1.0 / -zDistance;	
[self layer].sublayerTransform = aTransform;

This gives us a much more realistic look.

Putting it together

Wow, so now we’ve got some appropriately sliced images, some layers and some animations. Let’s put them together to get the final effect.

The animation states (last is same as first)

- (void)animationDidStop:(CAAnimation *)oldAnimation finished:(BOOL)flag
{
	[self changeState];
}

• TopToMiddle: show flip layer, and start rotation through 90 degrees
• MiddleToBottom: change image on flip layer, start rotation through further 90 degrees
• ResetForNext: change displayed top and bottom layer contents and hide flip layer

Let’s see how it looks in action:

Form factor

The most obvious reason that this kind of interface works on the iPad is the form factor of the device itself. Think about it: an iPhone is a small (well, smallish) object which has no physical, real-world equivalent. It’s not supposed to “be” anything other than a phone. The iPad, on the other hand, is intended to mimic an actual object, or rather, a variety of objects: a book, a newspaper or an address book for example. The interface for each of these uses reflects this directly: the address book looks like a “little black book”, complete with tabs.

The iPhone uses some of the same metaphors – like the address book for contacts – but the representation is abstract, whereas the iPad UI is literal, using an actual image of a book. The abstraction is necessary on the iPhone in order for it to remain usable with a smaller screen. Compare the iPad and iPhone Contacts interfaces:

The realistic looking address book application on the iPad

iPhone contacts interface

The abstract to realistic continuum has been explored extensively in the world of icons (explained brilliantly here), but less so in the interface as a whole. Obviously some apps don’t have any form of real world equivalent (think System Preferences.app), so they’re likely to stay in a more abstract form.

Display fidelity

Of course, it’s always been possible to represent real objects on the screen, but without a display capable of a decent resolution it probably won’t look real. So another enabler for naturalistic UI is display technology: the more pixels and colour depth the better when it comes to rendering physical things. Realism often relies on subtle lighting and rich textures to achieve the desired effect. Anyone who’s ever attempted to render wood grain using 8-bit colour will appreciate how much easier it is with a 32-bit 1024×768 display.

The need for display quality goes hand-in-hand with a need for more processing power to push the pixels. If the processing “oomph” required to render your interface with sufficient quality means that it’s not responsive enough, then you’ve failed by any measure. The iPad seems to have reached a sweet spot where display capabilities, raw power and form-factor meet to make naturalistic UIs a feasible approach.

Direct manipulation

Up until the relatively recent introduction of direct manipulation technologies like touch-senstive displays, there’s a been a forced physical separation between users and their computer in the form of a mouse and keyboard. A drag operation using a mouse involves disconnected, abstract operations like pointing and clicking, which bear no relation to the object being acted upon. As such, there’s little use in having an extremely realistic representation of something to interact with, because the suspension-of-disbelief has already been broken.

But in the world of touch-based devices, our interactions are much closer to the physical actions we’d make, so having a closer visual representation to go along with it is beneficial. For instance, turning a page in an eBook and actually seeing the page flick over seems consistent when using a natural gesture like flicking a finger across the screen. Otherwise if you had to click a corner of the page and drag it across in a poor imitation of the physical action the visual aspects can quickly start to feel superfluous.

Perhaps given the iPad’s larger interaction surface we can expect to see a wider and more expressive range of gestures supported?

History

Of course, naturalistic UI isn’t really a new thing. It’s in use in quite a few applications already, albeit mostly in desktop apps, where capable hardware is more readily available. In fact, there was much a-twittering (not all of it from Will Shipley) about the similarity between the demo’d iPad apps and the interface of the fantastic Mac application Delicious Library. I’m sure of course – how do they say it in the movie credits – the similarity is entirely coincidental…

Delicious Library UI

iBooks UI

One of the first widespread uses of naturalistic UI elements was in software synths. Guys like Propellerheads and Native Instruments set about creating exact software duplicates of well-known and well-loved bits of mid-80’s audio hardware. The UIs included virtual versions of a huge number of physical elements that had never been used before: including things like sliders, LEDs, toggle buttons, rotary knobs and rocker switches. These interface elements were probably first seen in software-based mixing desks, but the techniques used to make them feel real have gone on to be used in many different ways. Even the lowly push button in both Windows and MacOS got an upgrade by application of some of the 3D effects that they originated.

Where Now?

It will be interesting to see what other types of app are given the iPad realism makeover. It certainly looks like all of the standard Apple apps are being changed, including Notes, iCal, iBooks and Contacts, which were shown in the launch presentation, but how will 3rd-party developers manage? It undoubtedly takes extra effort to make something look convincing when it can be compared to the physical object itself.

Apple has previously managed to get a good level of consistency in its abstract UIs by providing the various interface elements ready-made, but will that approach scale to this kind of UI? No doubt the developers who take great pride in the interfaces now will continue to strive to create apps that sit comfortably alongside Apple’s own, but perhaps there’ll just be a more obvious visual distinction between those guys and the less polished application user interfaces than there has been in the past.

Pegs vs Triads

1. Minimise cost of entry/exit – Instant on, instant off

If there’s one thing that most characterises the mobile gaming experience, it’s the difference in playing cycle frequency and duration. As this highly unscientific graph shows:

Gaming intensity over an 18 hour period

Handheld games need to be able to be picked-up and played, and similarly, put-down, in a flash. It’s just not feasible to insist on finishing a mission, returning to a safe-house etc, before the game can be safely stopped. You don’t want to lose the last few minutes progress just because you’ve arrived at your destination. Peggle has a few wrinkles here, the only times it crashed on me was after unlocking the phone to continue a game, but generally it’s perfectly paced for snatched play; each ball lasts a few seconds, and each game only a few minutes. Furthermore when resuming a game you can instantly see the number of blue and orange pegs remaining and therefore gauge game progress at a glance. Having a free-roaming city to explore is a fantastic experience, but keeping track of where you are requires more mental effort/investment.

2. Support restricted input – One-handed operation

Personally, I get a few good hours a week of gaming time on the tube. Unfortunately most of that is while hanging from the hand-rails amongst a crowd of other commuters. This one-handed playing style is completely incompatible with GTA’s control model; the driving requires one hand for the accelerator, brake etc, and another for the directional controls. The overall pacing of the game is also important; twitch gaming doesn’t fit well with being buffeted around on the underground, but board-game style pacing is largely unaffected.

3. Degrade output gracefully – Shhhhhh!

GTA is most definitely a multi-sensory experience. The radio soundtrack that kicks in as you jump into a ‘jacked motor. The ambient sounds of the city as you stalk the sidewalks looking for unsuspecting victims to eviscerate. Peggle has its own rousing soundtrack of course, Ode to orange peg Joy, but it’s much more of an ancillary aspect of the game. You can safely turn it off and it’s not going to reduce your ability to play, or alter the play experience. The importance of this becomes apparent whilst enjoying a surreptitious few minutes of fun in, say, just for example, err, the toilets at work. Not that I would ever do that, obviously…

4. Work with device restrictions – Physicality

One of the things I found most jarring about the translation of the GTA experience to the iPhone is the very thing that makes the iPhone what it is: the touch screen. It’s just odd not having the haptic feedback on a stick, especially when you’re virtually pushing a car round a tight corner, or trying to chase someone down the street. I just felt like I needed to feel the limits of the controls. Part of the reason for this is the fact that you’re essentially controlling physical objects directly, whereas Peggle is much more abstract – albeit fundamentally based on the concrete physical behaviour of a falling ball.

…but which is best?

The answer? Who knows! The fact is they’re completely different experiences, so it depends what you’re looking for. The thing about smartphones, portable gaming devices etc, is that people use them in a vast array of different environments and in very different ways. Personally I don’t want a console replacement, but I do enjoy console-like experiences on the go every once in a while. I’m willing to change the way I use the phone – at least temporarily – to accommodate that. But I also love a game that fits perfectly with the way I already use my device; especially if it happens to be incredibly addictive, with simple but emergent game-play. These days I’m a real casual gamer, having to grab what time I can between work and the family, so having something that I can enjoy in 30 second segments on my own terms is a real pleasure.

It will be interesting to see how developers, gaming and otherwise, deal with the different usability requirements of these emerging platforms. These factors could quickly become important differentiators in a crowded market.

Well, I’ve done it: I’ve got my first game live on the AppStore. It’s been an interesting journey. I’m terribly bad at getting my hands on devkits and SDKs, having a play with them and then not doing anything constructive. This dates way back to things like the Playstation NetYaroze, which was pretty expensive, and with which I failed to produce anything concrete. So this time around all the pieces were in place: shiny new “gaming” kit, interesting SDK, low cost of entry. I was determined to create!

I’ll be making a series of posts on the process and details of creating it, in the interest of sharing the fun. In the meantime, you can find out more about the game here.

One of the things that seemed a little odd about the SDK is it’s use of CAF-format audio files, detailed here. I got hold of a few very nice audio samples from the freesound site, but needed to convert them from WAVs to CAFs.

I thought ffmpeg might be up to the job, but the version I had didn’t list it as an avaliable output format using ffmpeg -formats. However after a bit of digging I discovered that it is supported by libsndfile, so set about installing it using MacPorts:

sudo port install libsndfile

Then I used the included libsndfile-convert app to convert my file:

libsndfile-convert file.wav file.caf

The output format is inferred from the file extension, so you don’t have to specify it. However, when I rebuilt and ran my iPhone app using the new file, it didn’t play back. I suspected there may be something wrong with the format of the file, so I took a look to see what file reported. For the original WAV file I got

file.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, mono 44100 Hz

Unfortunately file doesn’t work on .CAF files, but you can open them using QuickTime Player, and using the Movie Inspector window you can see that the file has the following format:

16-bit Integer (Big Endian), Mono, 44.100 kHz

So it looks like the problem may be libsndfile-convert changing the endian-ness of the file contents, from the x86-style little-endian to Motorola-ish (i.e. pre-x86 Mac) big-endian, which is a bit of a pain. According to the docs, the libsndfile API supports endian-ness manipulation, so it’s probably just the case that the helper app is doing the wrong thing automatically. I’ll look at putting together a small command line app to use the API directly and enable me to batch process .WAV files correctly.