If you ask to call Alex Kipman the most significant advance in the brand-new version of HoloLens, Microsoft's mixed-reality headset, he says the answer is yes . It's not a bypass of the question – it's a testament to his excitement.
Kipman, Microsoft's Technical Assistant for AI and Mixed Reality, is passionate about "all things" in the HoloLens 2. However, when pressed, he comes down to the three main enhancements: it's more comfortable, immersive, and more Out-of-box value as the first HoloLens. Kipman often shared that mantra – "comfort, submersion, out-of-the-box value" – during my full-time visit to Microsoft headquarters last month, like someone who had been well trained by his communications staff. Later, when I was asked by an editor what was new after the new HoloLens 2, I realized that the mantra still rattled in my brain as though it had been transmitted through the headgear.
The new HoloLens 2 is more comfortable than the first headset and more immersive. The diagonal field of view has more than doubled, and Microsoft has a novel patented imaging technology. It has an AI processing unit and now connects to Azure, Microsoft's cloud service.
Out-of-box value is a call to commercial customers. This is not a headset that you use to play or send interactive poop emoji to friends, or that a regular consumer will ever wear. It's not for "knowledge workers" like me and Kipman, people who sit at their desks all day, he says. It's meant for people whose jobs are changing digitally-people who are in design or manufacturing, who make gear shifts and work on rigs and military personnel.
Try to forget that the HoloLens is a headset. Kipman thinks rather as a full-fledged computer for a futuristic world of remote employees who need expertise. And Microsoft is determined to make it one of the most advanced mixed-reality computers. This is clear, even if not all applications have yet crystallized.
To understand the meaning of HoloLens 2, it helps to determine the origin. The first seeds for HoloLens were planted 11 years ago. It was crafted from Kinect, the Xbox peripheral product that uses depth sensors to calculate depth maps and recognize people within its field of view. Kipman is credited with inventing Kinect, and in 2010 he began channeling some of the Kinect technologies into a head-mounted holographic computer. It was then known as Project Baraboo, but later became HoloLens.
When HoloLens was officially launched in 2016, it was a 1.3-pound head-mounted display with depth-finding cameras and a projection optical system that directly lit holographic images in your eyes. While wearing one, you may see everything from a floating web browser to a comic fish in the bath tub to a three-dimensional motorcycle – and still see the real world around you. Or you see a remote technician in your eye-frame and show you how to fix a light switch. It is not a terminal now, and certainly not then, but Microsoft tried to present a wide range of applications that ordinary people can easily understand.
The HoloLens were only available to developers first, as Microsoft wanted to drive the development of new apps. (No AR or VR headset is worth the money without compelling apps, which was true then and is still true today.) Later that year, a version of HoloLens was launched that was delivered to every consumer in the US or Canada Spent $ 3,000. 19659003] The first HoloLens was not a "success" in how you could describe the success of other technology products, whether it's sales, ecosystem lock-in, or a pure seal of approval. In a sense, it was not that was a blockbuster hit in public. But it was the first mixed-reality wearable that ran on a holographic-specific operating system – and it was not lightweight smart glasses. It was an untethered Windows 10 headset, which meant it was indeed a working-face computer.
Earlier clients, however, had their complaints: it was heavy, it was cumbersome, it did not feel deep enough. And Microsoft heard them loud and clear.
Put Your Heads Together
One of the most obvious updates to HoloLens 2 is the build. The first HoloLens was heavy at the front, a whole series of components that were loaded on your forehead. In this new version, Microsoft has divided the parts, the lenses and some computing power positioned in front and pushed the rest to the back.
Carl Ledbetter, the chief designer of Microsoft, calls this a split-architecture design. It faced its own technical challenges as cables had to run between the front and back of the headset. These are now integrated in the arms of HoloLens 2. According to Ledbetter, this new form factor was crucial to achieving some comfort and the balance of the new model. "With the first version of HoloLens, there were many things we did not know we did not know about," says Ledbetter as he guides me through Microsoft's Human Factors lab. "But luckily we've talked to many customers for three years."
The Human Factors Laboratory is a hollow-headed room with as many mannequin heads as human ones. The latter are bent over the desks and work with the latest designs. There are also earmolds, gesture control bracelets and custom eye depth gauges. For the last three and a half years, Lebetter and his team have used these tools to develop a new HoloLens headset that fits 95 percent of heads, regardless of gender, race or age. Ledbetter is not just about finding the right fit, but also empathy for the wearer. Once he gives me a deliberately oversized Xbox gaming controller. "There," he says. "You are five years old."
Ledbetter and his team scanned over 600 human heads in the Human Factors lab. Hundreds of other people were subjected to stress tests using HoloLens 2 prototypes. They were asked to watch a long movie, play Jenga, or talk to other people. The goal was that people forget that they carry it, ideally up to two hours. In some cases "we got more than two hours and people did not take it off at all," says Ledbetter. Some tests included sensors attached to the neck of the subjects that measured muscle fatigue or fatigue. Ledbetter claims based on this data that the new HoloLens are three times as comfortable as the old ones.
I wore HoloLens 2 for some short demos during my visit to Microsoft and is undoubtedly more comfortable than the first version. It also weighs less, albeit in grams. The Click Wheel on the back of the headset, which loosens or tightens HoloLens around your face, is less clicky than the first one. Microsoft says the battery life should match the first HoloLens, about three and a half hours. Kipman says he's looking forward to the day when the battery life of the HoloLens is running low, which means they've been wearing it for such a long session.
There are also material upgrades. The front housing is made of carbon fiber, which should keep it cool and light. The unit has anodized aluminum cooling channels that dissipate heat from the processor of the headset. The silicone back pad, the part now attached to the back of the head like a gray piece of toast, has a microtexture that provides just the right grip without tearing out the hair.
The Thing That Could Lead The biggest difference, at least for the Microsoft audience, is an old-fashioned trick applied to a new headset: the front case can now be unfolded like a cool dude clip-on pair of sunglasses become. If you work on the field or on an assembly line and need to quickly switch between holographic instructions and conversations with a real person, you can simply lift the lens. Kipman likes to show it by pulling the "visor" up and pulling it down again. "Holograms everywhere!" He says as the lens housing assumes its lower position.
These new features – the split architecture, their cooling mechanisms and the hinge that enabled the visor mode – were completed in the run-up to the look of HoloLens 2, says Ledbetter. But the optics makes holograms possible. The look is by far the most interesting part of this new HoloLens.
Beam in Your Eye
Last summer, Microsoft announced that it had filed a patent with the United States Patent and Trademark Office in 2016, in which the extension described field of view on a display using MEMS laser scanning technology , MEMS refers to microelectromechanical systems that include miniaturized electrical and mechanical components. According to academic journals, lasers have been part of MEMS research and application for decades. This part of the Microsoft patent application was not new. New was the method proposed by Microsoft to modulate MEMS mirrors so that the laser is directed in a direction that creates larger angles and thus a wider field of view.
The field of view on the original HoloLens – this virtual eye box through which you see holographic content – was not very large. Ever. The material you viewed was often trimmed or milled out of the image if the object was too big for the window or if you moved your head in a particular direction. Jeremy Bailenson, founding director of the Virtual Human Interaction Lab at Stanford University, wrote a cowrote on the social impact of "virtual people" in a 2017 white paper, using AR and VR headsets as part of the study. In it, the authors painfully describe the limitations of HoloLens' narrow field of view.
"From an empirical point of view, we know that the field of view is crucial," says Bailenson to WIRED. "As a result, people generally have a better experience because they can naturally move their heads."
This was obviously one of the aspects of HoloLens that Microsoft needed to improve. And it did. The first HoloLens had a FOV of 34 degrees; The field of vision of the new headset has "more than doubled," says Kipman to a field of 52 degrees. (Microsoft declined to give exact measurements for this new Eyebox and said that the X – axis and the Y – axis are not the best way to think about the improvements of the FOV, but most extensions were in the vertical dimension .)
The HoloLens optics team was also able to maintain a resolution of 47 pixels per degree and expand the box. This means that while the first HoloLens had the equivalent of two 720p displays, one for each eye, this new face computer has the equivalent of a 2K display for each eye. And the lens stack has been reduced, from three lens plates to two.
After my own experience with HoloLens 2, I still came across the edges of the eye box. A hologram of a woman named Hannah, who gave me an overview of Microsoft's campus construction project in a recorded holographic video, was still going head-down or footless when I got too close to her. The same goes for the windmill tops, which were part of a topography demo later in the day. One where I could use my (real) hands to pinch and zoom the holograms in a new way. Although the field of view has been improved and the content is clearer, we are not quite at the point of holograms, not yet interrupted.
Both Kipman and Zulfi Alam, who heads up Microsoft's optics engineering team, acknowledge that the visual experience on HoloLens 2 is not yet complete. It is the mechanical method that has increased the field of view, but they seem to be the most excited. The MEMS mirrors that Microsoft uses are the "biggest little mirrors in the world," says Alam. The mirror looked like a rubbish spot on a conference table; When I looked up to look at it, I could see that it was a tiny reflective disk on my fingertip.
Normally, a DLP, LCD, or LCoS projector spits out light particles in a headset. Bouncing off lenses and beaming into their eyes, they essentially made them see holograms. (The original HoloLens used an LCoS projector.) And HoloLens has enough sensors to detect the position of the head in space, so he knows where to shine those images in your pupils to convince you see something. With the MEMS mirrors flashing 54,000 times per second, HoloLens 2 now splits the light and reconstructs it at each pixel. "It replicates your student several times," says Alam.
The advantage, if you do it that way, he says, if you want to increase the field of view, just change the angles of the mechanical system. You do not need to build a larger backplane to create a larger field of view that would increase the overall size of the product. As with the physical redesign of HoloLen, this innovation also presents new challenges, such as: For example, developing the software to make everything work properly. "The control logic gets very complicated," says Alam.
On the way to a new reality
That's all inside. Outwardly, HoloLens needs to be useful to the corporate customers who will use this thing in the wild. Microsoft has also done some work there.
For example, HoloLens supports 2 advanced gesture controls. Previously, you could use your finger in a Redrum-like way to select holographic tiles that were displayed in front of your eyes. You can also use a flower movement, a kind of hand-panning, to return to the Start menu on your holographic desktop. If you've looked at a holographic app icon long enough, you can highlight it.
Now, in HoloLens 2, you can access a virtual object and manipulate, rotate, shrink, even punch or punch it with your hands. Thanks to the headset's new eye-tracking technology, you can read a news story in a holographic browser, and the page will go through for you. See, Ma, no hands. All previous gestures still work, but it's these new types of interactions that Microsoft believes will lead us to a reality where mixed reality feels more natural.
Microsoft also addresses new cloud-based "spatial anchors" that allow access to holographic app functions, even if they do not carry HoloLens. Let's say I'm wearing a HoloLens 2, but you're not on an iPhone or Android smartphone. We should both be able to view this holographic representation of Microsoft's campus construction project simultaneously, provided the app developer created the app that way.
Because app developers are still a critical component of the HoloLens ecosystem, Microsoft believes it introduces the so-called Dynamics 365 Guides, a premade set of software features that can be directly integrated into HoloLens learning applications. Would you like to teach somebody how to fix the gearshift on an ATV? Kipman says it only takes a few minutes to build the HoloLens app.
But Kipman, who has been with Microsoft for 18 years, regards HoloLens as something much bigger than just a headset running hologram apps. For him, it is part of a technological revolution that takes place every 30 years. In the fifties there was the CPU; in the 1980s the GPU. Everyone was responsible for dealing with a certain amount of computation.
"Thirty years later, pay attention to the pattern," he says. "You can name it any, and we call it the holographic processing unit, but the devices of the future will all have a CPU, a GPU, and a kind of AI unit." HPU 1.0 was the first instance of Microsoft's Holographic Processing Unit. The HPU 2.0 in the new HoloLens is "ideal for algorithms and machine learning," says Kipman. "We've also created deep neural network cores optimized for the execution of these models."
HoloLens 2 now also connects directly to Microsoft's Azure cloud service, which makes Kipman's headgear "differently lit" This means that certain AI tasks are outsourced to the cloud. This results in more accuracy – such as the difference between a one centimeter spatial assignment and a one millimeter spatial allocation – but it can take a few seconds for the headset to process anything. Kipman insists that certain corporate customers agree with this latency.
"I think if you're talking about Vision Picking, which is hot news in the logistics industry, where frontline employees are processing packages without a scanner in hand, you may be looking for something lighter," says JC Kuang, an analyst at Greenlight Industries that covers AR and MR closely. "If you choose Google Glass or an older Vuzix model, however, for example, when you look at architectural design constructions to look at data on a job site, this is a much more complicated process. So, using HoloLens while running the AI operations in the cloud is an advantage.
It makes sense that Microsoft would use Azure in every possible way. "In a vacuum, without talking about augmented reality, Azure is becoming an increasingly important source of revenue for Microsoft," he says.
Connecting HoloLens to Azure could also be part of a broader strategy: one that allows Microsoft to avoid the "hype cycle," as Kipman says. There are products, he says, that everyone believes they want to take over the universe overnight, which leads to a "disillusionment deduction" because it does not. Some products make it to the other side of the abyss; some find their place in a niche market. But they will not take the arithmetic.
"Then there are things that are transformative," says Kipman. "They really live alongside other computer periods, driving democratization and innovation on a scale. I think Mixed Reality is that. But, you know, we do not have it – and we will not overdo it. "Kipman, temporarily forgetting his mantra, was suddenly crystal clear.
More Great WIRED Stories