Ask me to repair a broken car or motorcycle, and you would be out of luck.
But that's exactly what Microsoft wanted from me. And the tool he gave me was the new mixed reality headset HoloLens 2 .
After plugging it in, a series of digital instructions appeared over a broken ATV set up in a fake workshop at Microsoft headquarters. On my right side were buckets of bolts, to the left of tools. Then neon-blue, cartoon-like arrows appeared, indicating which gears I needed to move and where I needed to move them to begin the repair.
Next, I was instructed to have parts of containers in the room. The blue arrow pointed me with the bolt I needed to the trash can, then with the washer and then with the right wrench to a table.
Which screw size? What kind of wrench? I did not have to worry because the HoloLens 2 took me every step of the way. After about eight minutes, I unknowingly looked for a broken ATV and was amazed at my first job. As Keanu Reeves says in the science-fiction epic The Matrix, after downloading martial arts directly to his brain, "I know, Kung Fu."
This is the promise: a computer that can do pretty much anything for which you would use your hands. This is done by identifying where it is in a building and then putting computer images on the real world that you actually see. Unlike the original HoloLens from Microsoft, which was announced four years ago, this new headset is more comfortable and user-friendly.
This makes working with HoloLens all the more compelling when you look at the holograms it creates. And that can be anything from a huge movie screen to a model of a new building. Or a series of instructions to help you bake a few cookies or repair an aircraft engine. With a pair of cameras, the headset can also be a friend of the phone so that anyone can see through the eyes of the one wearing the headset. Think about it. Your friends can also use an app to orbit things they see when they wear the headset.
"The goal is that these things will change people," said HoloLens boss Alex Kipmanat Microsoft headquarters in Redmond, Washington. "They will empower people and organizations to do things they did not do before."
Microsoft is not crazy if you think about the potential of these headsets. The brightest minds in the tech industry are already investing billions of dollars to develop a mixed reality (or, as some call it, augmented reality). They all bet that the way we use computers will change over the next few years. Statista estimates that the AR equipment market is expected to skyrocket from under $ 6 billion last year to nearly $ 200 billion by 2025.
"We do not have to imagine that, this future is here," said Microsoftat an event that announced the device on Sunday at the Mobile World Congress (MWC) in Barcelona. "Together, these advances are the next phase of innovation."
A high-profile startup called Magic Leap began delivering a $ 2,295 headset last August after spending seven years working on it, with $ 2.3 billion invested by backers, including AT & T. Google and Warner Bros. Facebook, which sells a fully immersive virtual reality headset called Oculus Rift, is working with 399 euros on a HoloLens competitor. Even Apple secretly builds one that, according to CNET, arrived in 2020.
Microsoft opens pre-orders for HoloLens 2 on February 24 for $ 3,500. The company plans to ship the device later this year.
HoloLens (the name is inspired by holograms) sounds like the first demo directly from the science fiction world. "We're not talking about putting you in virtual worlds," said Kipman when he introduced the device in 2015. "We dream about virtual worlds, screens and pixels."
But HoloLens had his problems. The original headset was hard to put on. It felt heavy at almost 1.3 pounds, which is a bit heavier than an iPad Air. The HoloLens also pushed the heads of some people. It did not always fit over your glasses. At $ 5,000, it was expensive. And the software that ran the program often required additional settings and adjustments.
The worst part was that the holograms appeared in an area in front of you that was only about the size of a deck of cards. (In the AR language, this area is called the field of view.) In practical terms, this usually means that you have only seen a small part of a hologram.
"The field of vision of the previous headset was the biggest drawback," said Scott Stein, CNET chief review reviewer, who tried more headsets than I knew. "Virtual objects must be placed exactly in a row, and the ejection breaks the illusion and adds a little tiredness to finding things again."
So Microsoft set about solving these problems. Kipman invited Stein, the editor-in-chief of CNET Español, Gabriel Sama, and myself to the corporate office to see what happened to HoloLens 2. And to share the promise of HoloLens 3 and beyond.
Bottom line, says Kipman: "We have one. We still have a lot of work ahead of us."
Imagine going through a store and not looking for that mustard bottle. Arrows like the one that helped me repair the ATV guide you through the maze of aisles and straight to the ingredients on your grocery list.
Or they could show directions when you go down the street.
Instead of arrows, a HoloLens 2 can show you how new furniture would look in your living room. Or help to read a speech without a teleprompter.
For Kipman, this technology is a holy grail of computers. Over the past three decades, as we watched a screen interact with a keyboard, mouse, or controller. Even smartphones are just smaller screens.
These types of computers are good enough for a few things, such as editing audio and video, coding an app, or writing a story. But for almost everything else, Microsoft sees the mixed reality as a better alternative.
That's because researchers discover that HoloLens can also change the way we communicate. In an experiment in the Virtual Human Interaction Lab at Stanford University, a person was taken to an empty room, but could speak through the HoloLens with a photorealistic AR character. The avatar maintained eye contact, interacting naturally and gesticulating as if he were a real person.
"It was breathtaking," said Jeremy Bailenson, communications professor who runs the lab.
Mixed-reality systems such as HoloLens and Magic Leap are not the only technology that revolutionizes the use of computers. There is also a virtual reality where headsets keep a screen so close to your eyes that you think you are in the computer-generated world. After decades of VR development, Facebook boss Mark Zuckerberg helped legitimize the technology by spending more than $ 3 billion on then-startup Oculus.
Although the market is still emerging, people use VR for intense training (for example, showing a soldier how it is in a war zone), watching movies, and playing video games. You can buy an Oculus Rift for $ 349, a HTC Vive for $ 499 or a Sony PlayStation VR for $ 299. They're pretty much alike, though each offers screens of different quality or slightly different controllers.
MR headsets are now fundamentally different from company to company. The Magic Leap One system consists of three devices, including a headset called Lightwear, which uses a technology similar to that of HoloLens to overlay computer-generated images with the real world. Unlike HoloLens, however, Lightwear is powered by a hockey puck computer called Lightpack, which has enough computing power to produce realistic-looking 3D images. The computer is connected by a cord to the headset and has a slot in the middle so that it can span your bag and allow air to circulate and cool around it. You interact with the Magic Leap through a third gadget: a portable remote control called Control.
We know that Apple is secretly developing a headset, but we do not know much about the design except that it's powered by Apple chips, it talks wirelessly to a computer and uses super-sharp displays.
The only thing MR companies agree on is that they do not sell to ordinary people – at least not yet. The Magic Leap device has been released for developers, with a consumer version for the future in the not-too-distant future promised.
Microsoft says it focuses on business applications, although HoloLens was originally used as a way to work and play games such as the world-famous Minecraft phenomenon. Now the company says HoloLens is an easy way to transport people around the world. Or a way for a worker to do something complex without having to learn the process first.
Kipman describes it as a superpower of humans. "This is a concept that has been in our dreams," he says.
Finding a Better Fit
HoloLens 2 might not be ready for you and me, but it's a much better device than its predecessor.
The first was HoloLens bulky. All of the computing power was focused on the front, along with the mix of cameras, motion sensors, and microphones that allow you to understand where you are, what you are doing with your hands, and what you are saying. It huddled upside down, and wearing glasses like stone or me made it uncomfortable to wear for more than a few minutes.
To solve this problem, Microsoft turned to the Human Factors Engineering Lab. The group, housed in a bland building that was built on the Microsoft campus in 2015 and funded with $ 44 million, was founded to show the technology giant how to build equipment for most people.
Carl Ledbetter, Senior Design Engineer For the Microsoft Device Design Team, I began my tour of the lab by picking up an oversized Xbox video game controller sitting on a table at the entrance. It was heavier than the Xbox controller I had used, and I found it harder to reach all the buttons.
"You," he said, "are five years old."
The oversized controller is a tool that engineers can use to understand all users who use their devices, including children. Microsoft took a similar approach with HoloLens 2 and tried to equip it for as many people as possible.
To improve the original design, the team decided to evenly distribute the weight of HoloLens 2 over its head by moving the battery and other nuts backwards in a pack the size of a can of Altoids. That helped to make it comfortable – it was like slipping on a baseball cap.
Not all our heads are alike. For example, Microsoft spent a year scanning more than 600 heads, using a device with 36 connected cameras around a space to capture each form. Ultimately, these scans helped Microsoft understand different body types for men and women, from small to tall, big to small. Some people have higher cheeks, deeper noses or larger bumps in the back of their heads. Microsoft wanted the HoloLens 2 to fit 95 percent. (Sorry, Hodor.)
"It's not just about scaling head size," said Ledbetter. "Every head is different."
The HoloLens 2 also has sensors that scan your eyes. So you can quickly see who you are so that you can share a headset with more than one person, but set it up each time you set it up. The device also monitors where you are looking so that an app can help you read it by moving text along with it or simply responding to what you are looking at.
And if you have to quickly lose the HoloLens to chat with someone In the real world, unfold it like putting glasses on your forehead.
"Once the first model was built, it was like," Wow, we have to do that, "http://www.cnet.com/" Ledbetter said, "It was a lighter."
A New View
The most dramatic change in HoloLens 2, however, is the field of view or how much hologram you see. If you move your head too much in any direction in the first HoloLens, the holograms will disappear.
At HoloLens 2, Microsoft more than doubled the area where you can see them. This was done by creating a new hologram technology.
In HoloLens 1, holograms were created by reflecting images from a tiny screen of the headset into custom lenses for red, blue, and green lightwaves. These light waves were then blasted into the back of your eyes where your brain would produce the final image.
In HoloLens 2, the tiny screen has been replaced by a mirror called MEMS, which moves fast enough to create the illusion of a screen in space. The MEMS creates 120 of these screens per second, which are filtered by new, slimmer lenses in the headset to your eye. The result is smooth movements, brighter animations and a quick response as you move your head.
Most importantly, the holograms offer much more space.
Before Microsoft used MEMS technology, some executives believed it would be many years before the company could improve HoloLens' field of vision. "It was a major engineering problem," said Zulfi Alam, general manager, who worked on HoloLens' displays and screens.
Kipman called the new approach a "miracle".
When I repaired the ATV, I did not notice that the holograms were not in all areas that I could see. There was just enough that I stopped thinking about it.
One of the reasons why Microsoft says that HoloLens 2 is not yet good enough for everyday people is that there is still not enough for us. Some people believe that the killer app or the must-have feature of HoloLens and Magic Leap will be the case if they replace all screens in our homes. We would just tap a headset and look at it as holograms.
Kipman is not convinced. It's like imagining a car as a "horseless pram," he says. Instead, HoloLens is a monitor for these times and a keyboard does not work. This can be the case if you examine the schematics of an oil pipeline in the field or put a patient's x-ray over the body.
"If you can make a worker $ 150,000 and 40 percent more efficient, that's $ 60,000 a year," said Jim Heppelmann, CEO of business software maker PTC, whose software collaborates with HoloLens Place to help Use schematics to B. to repair defective machines. "I will buy 5,000 devices all day."
To further enhance these relationships, Microsoft has made some improvements to the software that drives HoloLens. Underneath, the company has developed software that uses the sensors on the headset to map the buildings they're in, so anyone using an iPhone instead of a holo lens can see the holograms in the room. HoloLens also works with Microsoft Windows PC software technologies that make it easy to encode apps and connect to the Internet.
Despite the enthusiasm, this could all be broken. Do you remember Google Glass?
When Glass was announced in 2012, people predicted that smart glasses would do similar things as expected for HoloLens. Google abandoned most of the project in 2015 after considerable criticism of its usefulness and privacy concerns raised by the connected camera.
Intel also had an intelligent eyewear initiative launched in 2013, but the company closed the project last year after trying to find partner companies. And the Osterhout Design Group, whose Smart Glasses 2017 was talked about by CES, was closed and the patents were sold last month.
These setbacks do not worry Paul Bettner . He is one of the people behind the successful mobile game Words With Friends, which later developed a well-known VR game called Lucky's Tale. Finally, Bettner and other techies say that devices like the HoloLens will shrink to the size of a normal pair of glasses, then they become part of everyday life.
But to get there, companies need to move forward gradually. just like smartphones. "They could not switch from nothing to the iPhone," he said. "They had to make flip phones first, then Palm Treos and then everything else to get there."
HoloLens is not Kipman's first attempt to change the way we interact with computers. This was Project Natal, an accessory for Microsoft's Xbox 360 video game console that lets you track your body as you move in front of it. This was done by covering a room with thousands of infrared points and then using a camera to determine where they land.
The Kinect, as it was later named, was so popular that it sold 8 million units in the first 60 days, earning it a place in the Guinness Book of World Records as the hitherto fastest-selling consumer gadget ( Apple's iPad picked up the 2011 title.
"The inspiration was really that we could actually integrate a human into the gameplay," Kipman said at the time. "It really allows us to come in and play, it really inspires us."
Microsoft ultimately sold 35 million Kinects, but the device seethed out when developers had trouble finding entertaining ways in their games. Nevertheless, the technology has helped inspire Face ID and Animoji on Apple's iPhones, Face-Unlock for Windows PCs, and of course HoloLens.
Today, Microsoft's enthusiasm for MR is contagious. Investments in AR and MR consumer software companies are expected to total $ 270.8 million this year. This is close to double the investment in companies producing similar VR software, according to Nielsen's SuperData Research. In the business world, companies like Chevron have bought hundreds of HoloLens headsets as they begin to use them widely among their employees.
Read: Microsoft's HoloLens 2: Why is Everything Really about the Cloud (by Mary Jo Foley of ZDNet)
As Microsoft prepares for the sale of HoloLens 2, Kipman looks a few years into the future when he will demonstrate HoloLens 3.
He complained when asked for details except to promise it is more comfortable, easier to use and has more features now available. Whatever the details are, Kipman said they had led the US Army to invest $ 480 million in more than 100,000 headsets to "increase the lethality by showing the ability to recognize the enemy, To decide and fight against the enemy ", according to a government proposal. (However, the contract has provoked opposition in the Redmond series, which Nadellarejected.)
HoloLens will not be the only device looking for space on the head. There is still Magic Leap and the unannounced headset from Apple. By 2022, "tens of millions" of it will have been sold, said Tim Merel, managing director of market watcher Digi-Capital.
For the moment, Kipman plans to keep HoloLens inaccessible to you and me. "I have no interest in outsmarting these products," he said. "There is a point where there is enough immersion, sufficient comfort and enough out-of-box value that I would like to announce a consumer product."
Until then you just have to wait. Unless you need to repair an ATV.
Scott Stein and Gabriel Sama of CNET contributed to this article.
First release on February 24 at 9:20 pm PT.
Updated on February 24 at 10:45 PT : Adds more details on the military contract and other technical details; Updated on February 25 at 6:45 pm PT : Adds more information about the HoloLens hardware. Corrects the misspelling of the name of PTC's CEO. Updated on February 25 at 3:09 pm PT : Adds Nadella's response to Pushback on HoloLen's military contract.