Augmented and mixed reality started as a lofty promise that's just now taking form, but with several companies taking somewhat different approaches, it's hard to understand what's what. Let's take a look at the three big players and what they're doing: HoloLens, Meta, and Magic Leap.
Of all the holographic mixed reality headsets, Microsoft's HoloLens is the only one you can actually use right now—if you're a developer who managed to get one already, that is.
The device looks like a halo and a pair of sunglasses birthed an overweight child, but it is comfortable to wear once you figure out the right position. Once on, you can view a holographic world inside whatever indoor space you are in.
The headset can be used to browse the web, take a tour of the distant world, or play an immersive game. It takes some time to understand how to use it, but you only need to learn two gestures to learn how to interact with its holographic interface. With those tools you have the freedom to move around at will.
Although an unfinished product, HoloLens works remarkably well and understands its limitations. Experiences are built in the Unity development platform traditionally used for games. It suits holographic experience development quite well as the tools for rendering a false environment on your television don't differ much from those you need to project that experience into the real world.
The hardware inside the HoloLens isn't much more powerful than what you'd find in a modern smartphone, at least, from a CPU/GPU standpoint anyway. Paring down the hardware helps the headset stay cool which, in turn, keeps your brain uncooked and your hair from singeing on your head.
Virtual reality (VR) headsets can avoid heat issues by being physically connected to more robust hardware, but because mixed reality (MR) headsets like the HoloLens require you to move around your environment, they can't just plug into a desktop computer.
However, the HoloLens also contains a custom chip they designed in-house called the HPU ("holographic processing unit") that processes terabytes of data in real time to handle the environmental understanding and build spatial map meshes. It also provides real-time micro-corrections to positional tracking, upscaling the GPU output from 60 fps in full color to 240 fps (4x single color) with each frame, adjusting to any of the micro-movements one's head makes.
If you didn't know that the HoloLens' sports a comparably slow Atom processor and minimal 2 GB of RAM, you'd never notice. Like most smartphones, it runs smoothly and renders a quality image. A line of cameras facing outward from the headset watch your movements and see the space you're in, while others measure depth. A plethora of sensors measure your movement and let the device play tricks on your eyes, making the holograms incredibly realistic.
The HoloLens also has a limited battery life of about three hours, and you probably wouldn't want to wear it for much longer than that. It charges quickly with a common micro-USB cable. That means if you want more time, you could theoretically plug it in during use (bad idea) or connect it to an external battery for a likely awkward but longer session (somewhat less bad idea).
Microsoft's approach might be the most traditional. While the interface is novel, you're still running Windows and using it feels, somewhat, like you're in a computer. Whether evolving out of our 2D screens into a familiar 3D environment makes for the best experience remains to be seen, but either way they've kept it simple and it really works.
Meta's latest augmented reality headset (more specifically called the Meta 2) seemingly takes a similar approach to Microsoft's. It projects holograms onto glass eyewear attached to your head and you interact with those holograms with your hands. But Meta's approach to the interface is notably different.
Meta doesn't want to create a new computing environment like the ones we use today, so they turned to neuroscience for answers. How will our brains understand holographic interfaces and interact with them naturally? Meta wants their operating system to create an environment our brains simply understand, without require you to learn to use it.
If you watch the TED talk above, given by Meta's founder Meron Gribetz, you'll see an interface that might look familiar if you've seen the Iron Man franchise or Minority Report. If you want to interact with a hologram, you treat it like a physical object. Grab it if you want to pick it up. Turn it over if you want a different view.
Some of these movements come naturally, but this interface lets you do much more with holograms, including open them up, change colors, and manipulate them in whatever ways the developers design.
The big question is, how will our brains react if we don't have a frame of reference for these actions? Can neuroscience accurately predict how each of us will interact with digital and holographic information? Meta thinks so, but how it'll work in reality remains to be seen.
Complex technology always has a learning curve and Meta likely won't get it exactly right on the first try, but they believe the hardware they need won't be available for another five years or so anyway. They're working with large headsets like everyone else, but are aiming to turn their current design—which looks a bit like a bike helmet—into a small strip of glass. That takes time.
Like all of these devices, Meta has a series of advantages and trade-offs. Where Microsoft's HoloLens opts for a small screen to put the entire computer on your head, Meta chose a notably different path by offering a larger, more immersive screen with a high resolution of 2560x1440.
That's a far cry from the 8K displays necessary to make the pixels disappear, but impressive nevertheless. Meta also includes plenty of impressive motion sensors and cameras. You'll find the full specifications on their pre-order page.
What's most impressive, however, is you can purchase a Meta 2 developer kit for $949—a third of the price of the HoloLens. The catch? You still have to tether it to a computer with a nine-foot cable.
Perhaps this tether is why Meta has focuses strongly on practical applications and pretty much ignored gaming altogether. Try running around in a holographic first-person shooter without tripping on your computer leash. Falling flat on your face might be the most real, immersive experience you could ask for in a headset, but probably not the kind anyone wants.
Magic Leap might provide an insanely great mixed reality experience, if their early demo is to be believed, but they haven't shared much about their upcoming goggle-style headset with the public. We probably won't see a consumer headset until next year, but a very small number of developers will get their hands on the preliminary devices very soon.
While we've seen a few demonstrations and heard a bit of talk, we still don't know very much about the hardware and how it'll work. Nevertheless, we can piece a few things together.
We know Magic Leap is a holographic headset that creates mixed reality experiences suited for both games and practical applications.
You can seemingly place information around the room, such as weather forecasts, charts, webpages, and online shopping. Instead of everything existing on a computer screen, you can just walk around the room and interact with holographic projections as you would with real objects.
HoloLens takes this approach, but utilizes a set of tap and bloom gestures for interaction. Based on the demos (such as the one above), Magic Leap seems to understand your hand movements more intelligently, allowing for more natural interactions. Whether or not that actually works outside of a controlled environment remains to be seen.
We already know holographic projections struggle with lights, windows, and mirrors, and very specific methods of interaction help the HoloLens overcome those visual distractions in most cases. If Magic Leap can overcome those issues and understand natural movements, we'll have an impressive piece of technology to look forward to. A quick view of their website will tell you they don't have any doubts about reaching these goals.
While we can speculate endlessly, we still don't know much about the form Magic Leap's first headset will take. For all we know it could plug into a fanny pack computer for extra processing power and a larger viewing area, offering a notable alternative to HoloLens' small screen. Or it could work similarly.
Thanks to CBS, we do know it's currently tethered to a computer like the Meta 2, but we know the consumer version will, somehow, escape that limitation. Despite the secrecy surrounding its development, when the hardware starts making its way to developers we'll likely get more details.
As you may have noted, these three devices take somewhat different approaches to creating the best mixed reality experience possible right now. But technical limitations definitely limit how far the first generation can go. The chart below shows the most distinct differences at a glance so you can get a clear picture.
The differences are actually pretty straightforward:
- HoloLens opts for portability over processing power, giving you a small viewing area but the ability to use the device in practically any room.
- The Meta 2 opts for higher quality graphics and a larger viewing area, but requires you to tether it to a computer.
- Magic Leap seems to want to offer the best of both these devices, but we don't know if they've figured out how to provide the experience of the Meta 2 with the portability of the HoloLens.
It's impossible to know which will offer the best, most immersive experience, but these three companies have the main paths covered, and we are excited to get our hands on them.