By the end of this year, Google's WayMo self-driving program plan on expanding its public self-driving ride-sharing program in Arizona, and about 46 companies working to do the same. I'm not sure I'm in a self-driving car.
( Full disclosure: FLIR wanted to see how its thermal imaging technology could be incorporated into self-driving cars. )
FLIR, which claims to be in the business of thermal imaging technology self-driving game, has already been offered automotive applications for the last few years in the form of visual driver assistance systems in BMW, Mercedes, Audi and others. In this form, a display in the car uses the thermal imaging as a form of night vision to help animals and pedestrians in the road at night.
What else is the concept for self-driving technologies, except instead of serving as a visual aid for a human driver? in real time and react to the environment around the car.
Here's a video that FLIR paid Jalopnik columnist Alex Roy to do it all:
In an effort to advertise thermal imaging's self-driving capability, FLIR has developed its own visual awareness software and demonstration camera. The way it works is very straightforward.
The camera is no larger than a GoPro, just outside the car, just behind the windshield in the middle of the roof which is where a lot of new cars Placing cameras anyway. The application I witnessed was very simple, with a FLIR camera and a standard camera mounted side-by-side, connected to a relatively powerful gaming laptop via USB cables. In this application, some of the capabilities of the setup are limited by the computer hardware and the fact that we were using USB connections.
On the demonstration screen inside the car, I saw a side-by-side view of what the standard camera was seeing, and a synced-up photo of what the thermal camera was doing in black and white, in real time. Since it was dark outside, the thermal image was picking up a bit more than the standard camera.
While in our demo, the thermal feed was in black and white-black
Pedestrian figures standing in the image as moving, glowing white human-shaped figures, with a clear outline that any Person watching could instantly recognize, which is a good sign for any computer program trying to do the same. The way the computer works over 14,000 pre-programmed annotated visuals.
Basically, FLIR has taken recorded footage and gone through and annotated, or marked down, which shapes are cars, which shapes are bicycles, animals, people, etc. and so on track similar-looking objects in real time.
I watched as we drove a couple of blocks around the Jalopnik office in Manhattan. Thanks to Benjamin Franklin or George Hudson or whichever brain-genius came up with daylight savings time, it was nearly pitch black by 5:30 pm, and the standard camera was picking up next to zero pedestrians and vehicles, despite street and ambient light ,
But the thermal imaging camera is picking up a constant stream of cyclists and people on sidewalks, in crosswalks, jaywalkers, people in the back of box trucks, and even someone's head poking around a tree.
I also watched as the FLIR camera and annotation software highlighting every parked car on the side of the road, every car passing through in the intersection ahead. A New York City bus. The New York City Bus. A FLIR representative did not see the bus because they did not have enough data from their test drives in California. Presumably because California does not believe in public transit, though I've been assured of the Silicon Valley is going to invent the bus any day now.
It's important to note that this thermal imaging technology is not meant to replace traditional cameras, radar or lidar, the laser imaging system already found on most self-driving cars that does the three-dimensional mapping work. Instead, it is meant to be a passive layer supplementing the recognition software in the car.
The largest issue with thermal imaging is the prohibitively expensive cost of thermal imaging cameras, which can be over $ 1,000 per unit, which may be redundant. $ 1,000 as more companies incorporate the technology into their own driving cars. The other main issue with the tech is its limited functionality as a backup, additional passive layer to imaging system.
Thermal imaging does not pick up street light colors, road markings, lane markings, and a lot of three-dimensional information is lost in the middle-toned grays of the surroundings. But what matters-moving cars, cyclists, animals and people-stands out remarkably better than a standard video camera. And to my eyes, objects can be reliably tracked by the software, especially at night.
FLIR even went through the trouble of re-enacting the fatal self-driving incident, where a Volvo XC90 running Uber's self-driving system and hardware struck and killed a jaywalking pedestrian pushing a bicycle across the street at night, while over employee in the driver's seat watching television on their phone. In FLIR's recreation, the thermal imaging software recognized the pedestrian nearly 80 meters away, and around 30 seconds before they were clearly visible to me on the normal camera.
The Autonomous Car Had Six Seconds To Prevent Fatal Crash But Failed Act: Feds
The self-driving car owned by Uber that fatally struck a woman in March detected her in the road …
Read more Read
But beyond the self-driving aspects, there's something about watching the system in real time, something companies should keep in mind as they try to convince the Masses to hop in the back seat of a driverless car.
When it comes to self-driving cars, however, the more safeguards and redundancy, the better.