Welcome to the real world: Hands-on with Microsoft HoloLens

Your coffee table is now your kingdom

On our way out of the Build conference's secretive ("leave your devices at the door") HoloLens session, we were accosted by a Microsoft marketing chap.

"I have a question to ask you," he said.

"Go on," we prompted, cautiously.

"I thought I heard you say in there that this was ‘better than you'd expected'. What did you mean? Did you believe everything people wrote? About the smoke and mirrors and all that stuff?"

The man became quite animated. Almost angry.

"Those claims that this stuff didn't exist?"

We'll leave our reply (and, thus, conclusion) to the end, but this emotional outburst seems to sum up Microsoft's attitude towards its HoloLens project; it's something the company is clearly fiercely proud of, and extremely keen to prove has very real potential.

Throughout Build 2015, the hardware - and the Windows Holographic software that drives it - has been flagged up time and again as the jewel in the crown of the upcoming Windows 10 and its Universal App multi-platform ecosystem.

HoloLens is a real step into the left field for Microsoft but also, it turns out, something it may actually not be over-hyping.

Like us, you've probably seen the videos by now: Minecraft castles on coffee tables, medical students disassembling human bodies and architects building virtual structures that fit straight into the real world. We didn't know if all of that is attainable to the scale and quality the promotional material suggests and in truth, we still don't. But we can definitely see how it's possible.

HoloLens, despite Microsoft's flagrant rebranding of the typical meaning of the word "hologram", is basically another attempt at an AR (augmented reality) wearable - a device through which you view the real world, but with overlaid computer graphics and audio, and the user's voice and bodily gestures utilised to interact with the added features in meaningful ways.

We've seen such techniques in anything from simple "marked up" mapping apps in smartphones, to strafing and killing a dragon emerging from your sofa through the low-resolution stereo cameras of Nintendo's 3DS games console.

Google Glass also had a stab at using visual overlays in combination with facial and vocal gestures, but failed to win over the public due to privacy and technological issues.

The HoloLens session we were invited to was set up as a coding "academy" for a small group of journalists. We were led through a 90-minute session showing how the HoloLens hardware interfaces with programming environments. In this case, it was popular visual game development engine Unity, with Microsoft Visual Studio consulted at the end to compile and launch a program straight into the headset, via a USB cable.

The headset supplied was no early prototype with a battery pack slung round the user's neck, as has been reportedly doing the rounds since January. This was stylish, lightweight, and fully adjustable to the wearer's head (even down to the space between the eyes, which was an inputtable software setting).

In reality, the project had been largely pre-made in Unity, and we were merely reactivating various features in order to see how the various functions of HoloLens worked together.

But it was still easy to appreciate how simple the set-up in Unity actually was - an instantly recognisable basic graphics project comprised of a flat polygonal base surface upon which sat two cubes, a "paper" ball hovering in the air over the top of each one, and a couple of origami aeroplane shapes sloping from the upper surface of each cube down to the flat base surface.

Removing Unity's "main camera" object and replacing it with a short, bespoke piece of code was all it took to make HoloLens (and thus the user) the project's camera, and then after exporting to Visual Studio and then into the headset, the entire graphical arrangement appeared "in the real world" right before our eyes.

The viewing area to actually see the image through HoloLens's translucent visor isn't quite as generous as one might like, with obvious letterboxing at the top and bottom, and Computing found the level of visual glare a little distracting too, but overall, this was a hugely convincing, high-resolution 3D render that hung unwaveringly in space and caused that classic brainflip where you'd walk carefully around the origami objects, unable to resist reaching out to touch what doesn't exist.

Over the next few minutes we added basic Unity physics, "gaze" (in this case a head-directed cursor) and gesture input, allowing a quick "tap" in the air to unseat a ball and cause it to tumble down the paper plane slide.

It was impressive enough, but when the balls fell off the polygonal base, they clipped through space and disappeared forever, just as they would in the Unity project on a normal 2D screen. It felt just like viewing a standard Unity beginner's project, but in 3D space.

Then we enabled the "Spatial Mapping" script, and the power of HoloLens really started to make sense.

The hardware - again, with barely a few lines of code plugged into Unity - is able to fully appreciate and map the shape of the real world around the user in real time. Moving our head around, we saw HoloLens draw a white polygonal mesh over everything it detected in the room.

People, furniture, ceilings, doors. A chair could be moved, and seconds later HoloLens would update the change in the mesh.

Reactivating the origami objects, paper balls were now rolling all over the real floor of the room, they were dropping from the ceiling onto coffee tables, and they were teetering on the edge of sofas. And in our ears, spatial sound oriented the object in the room by playing a softly undulating tune mapped to its relative position to the headset.

Then we unlocked a surreal hell portal, visible through a crack in the floor, full of lurid green mountains and a flying dragon. The polystyrene balls fell in, and plunged 50 feet below a cloud layer to settle next to a winding stream.

Computing found itself crawling around on our hands and knees to try and see more of this world under the split lip of the floor. This world that didn't even exist, but actually - genuinely - now did, such was the combination of real-world information it utilised to form the illusion.

And so to the marketing man we said this:

"This is better than Google Glass. This seems to genuinely match the potential your promotional materials convey. We are excited to see it develop further, and fall into the hands of enterprise, where it seems even a small amount of imagination and implementation will pay huge dividends. Please can we take one home? Oh, please?"

Be aware: nobody would tell us anything about the hardware in Microsoft HoloLens. Nothing about the specifications, the battery life, or the cost. Lips were sealed.

As with anything new, exciting and prototypical, there are still more questions than answers with HoloLens, but it's a unique combination of tried-and-tested concepts, one or two new ones, and a savvy and robust coding solution. Even with our (very) limited Unity expertise, our imagination is running away.

In the hands of a talented end user, Microsoft HoloLens could be very special indeed.