Nauto aims to protect distracted drivers with in-vehicle AI cameras pointing inside

Dashboard cameras have captured videos of car crashes, crimes and other momentous events. But what about cameras that also point at drivers?

Nauto, a company with AI-powered safety cameras, has created a way for commercial fleet operators and others to capture problems with distracted or sleepy drivers and warn them in real time. Nauto has a dual-facing camera that points at the road to detect hazards and at the driver to discover drowsiness or distraction in real-time traffic situations. It can send alerts that immediately capture the driver’s attention.

The AI helps prevent accidents — and costly liability verdicts — using advanced risk detection and alerting, including for pedestrians, bicyclists, and motorcyclists, said Stefan Heck, CEO of Palo Alto, California-based Nauto, in an interview with VentureBeat.

It can also capture performance analytics. The result is that it can improve driver performance in just two weeks. Nauto is already been used by nearly 1,000 fleets worldwide and customers across multiple industries have said it reduces collisions by 40% to 80%. While most of the fleets use an after-market camera from Nauto, its cameras are also being designed into cars. To date, Nauto has helped avoid about 30,000 collisions.

Stellantis Ventures, the venture arm of carmaker Stellantis, invested in Nauto, which was started in 2015. To date, Nauto has raised $215 million.

I took a ride in a car with a Nauto camera in it with Kevin Van De Leur, software engineer at Nauto. He drove us around for a good 40 minutes and showed me how the camera works when it comes to tracking vehicles (such as the location of a truck in a commercial fleet), car utilization, misuse and maintenance planning. He showed me the alert that happens when tailgating. Fortunately, we didn’t test the system in dire situations. But it was good to see that the camera didn’t “cry wolf” at every opportunity.

Nauto’s latest offering delivers telematics alongside AI-powered driver and vehicle safety capabilities, all on a single device and a single software platform inside the car. It’s an example of AI technology at the edge, and it could help save a lot of lives.

Dangers of distracted driving

The U.S. National Highway Traffic Safety Administration reported distracted driving fatalities are likely in excess of 10,000 per year (upgraded from a previous report that said 3,100 a year), accounting for 29% of all traffic fatalities. And driver distraction is responsible for more than 58% of teen crashes.

The prime reasons for distraction are manual, visual and cognitive. Manual distractions are those where you move your hands from the wheel. Visual distractions are those where you focus your eyes away from the road. And a cognitive distraction is when you’re mind wanders away from the task of driving. Text messaging involves all three types of distraction, and cell phone users are 5.36 times more likely to get into an accident than undistracted drivers.

“Ever since smartphones, it’s an epidemic and it’s just not getting the level of attention that it should,” Heck said. “When you’re talking to somebody in the car, that’s already pretty dangerous as it’s nine times more dangerous than normal driving. If you start texting and driving with your knee, now you’re like 30 times more dangerous than normal driving. The risks just explode.”

He added, “We actually see distracted drivers are disproportionately contributing to the most severe accidents.”

A sobering statistic from the Centers for Disease Control indicates that in 2021 one in six people who died in crashes were pedestrians while the average size of the verdict in the U.S. trucking industry has gone up by 867% from 2010 to 2018 according to the US Chamber of Commerce Institute of Legal Reform.

Modern commercial fleets face a multitude of safety and operational challenges, including heightened collision-related losses, a surge in fatalities and injuries, elevated driver turnover rates, diminished productivity, escalated fuel, and insurance costs, increasing operational costs, and the growing expenses associated with technology management.

Nauto’s cameras, which have been available for a half-dozen years in various forms, are in North America, Japan and parts of Europe. The company is expanding to more of Europe and Latin America. The company have about 150 people. They improve the algorithms every few weeks.

How it works

Nauto’s cameras point at the road and at the driver, hanging from a rear-view mirror.

Nauto’s cameras have AI built into them with detectors that are designed to provide distinct alerts that differentiate between pedestrians, bicyclists, and motorcyclists, when there is a risk detected of colliding with one of these vulnerable road users.

Nauto uses sounds and other means to alert the driver inside the vehicle for drowsiness, cell phone use, seat belt policy violation and smoking. These all leads to the No. 1 cause of collision: different forms of inattention. During our drive, Van De Leur occasionally showed me how the alerts happen. Only once did it appear to notify us of a hazard that wasn’t really there.

Many modern cars come with these kinds of alerts. I have driven multiple new cars around in the past year that signal when there is a serious hazard — like a car ahead stepping on its brakes while I am moving at a fast speed. Too often the warnings are false alarms.

How the AI kicks in

Nauto has to make the same kind of decision using its in-car AI computer that is built into the camera. It calculates the seconds to possible impact and then delivers the warning as early as it can without lulling the driver to complacence with lots of false alarms, Van De Leur said. All of the Nauto alerts that I saw seemed appropriate.

When it comes to issuing a valid alert, the camera analyzes whether you are distracted while some external event is happening in front of the car. If it detects distraction, it will issue the alert sooner. I heard a speeding alert as well, in case the car goes over the posted speed limit.

The camera captures telematics such as acceleration, braking, cornering, and more. It has collision warnings for tailgating, pedestrians in the car’s path, and more, Van De Leur said. On top of that, it detects distractions like smartphone usage, smoking violations, not wearing a seatbelt, drowsiness, or otherwise taking your eyes of the road. If a driver puts tape on the camera so it can’t see, the camera can report that as well.

When the car is going above five miles per hour, it scans for external hazards. The aftermarket camera does not depend on other sensors in the car, nor does it do livestreaming for driver privacy reasons. The camera can differentiate between a pedestrian, a bicyclist or a motorcle rider.

The goal is really to give the driver the critical extra time to hit the brakes, swerve, pull over or make any other appropriate corrections when a risk is detected.

The NHTSA recommends having about four seconds for response time the following distance. If the gap can be closed in just one second, then that is dangerous tailgating. If you’re tailgating someone for six consecutive seconds, the alert will be triggered. If you’re distracted for 2.5 seconds or longer, it will send an alert. That’s in the interest of keeping the alerts to a minimum so that you don’t start to ignore them.

“The big thing is to focus on really prescriptive alerts. You don’t want to tell somebody that they’re doing something wrong. You just want to get them to focus on driving or just pull over to use a phone,” Van De Leur said.

When it sends video or an accident report, it uses an LTE connection in the camera.

“We do our calculations on the edge device,” Van De Leur said. “Everything is processed on the device.”

The camera has to rely on computing power within the device because the latency in going to a data center is just too long when milliseconds matter in a driver’s response time. Rather than plug into an existing computer in a car, the Nauto camera plugs into the car for electricity, with the cord being taped along the top of the windshield to the charging port so that it doesn’t obstruct the driver’s view.

The data is encrypted on the camera. If someone stole it, the data is all encrypted and each fleet has its own encryption. Fleets can set the level of video recording. Some will keep just the last hour of driving video and other data for privacy reasons. The camera doesn’t record audio at all.

Origins

The name Nauto is a combination of network and auto. It’s also related to “naut,” which is the Greek root for navigation. Heck was a Stanford University professor who often rode his bike to work from his home near campus. Once or twice a week, someone would nearly run him over. This made him very familiar with the causes of accidents and he was shocked to see the numbers getting worse due to the rise of smart phone distraction. Distractions of all kinds account for the bulk of accidents.

Heck has a doctorate in computer networking and has studied AI for a long time. He took note when lots of autonomous vehicle companies were starting. He did some calculations on how much testing it would take to make an autonomous car safe. He estimated that you would have to test drive a car for 200 billion miles to say that the car covers 99.9% of driving situations.

“There isn’t a company on the planet today that is planning to do autonomous vehicle testing for 200 billion miles before rolling it out,” Heck said.

It made him skeptical of achieving fully autonomous cars, even if they get help from car driving simulations. But it motivated him to think about how to make human drivers as safe as possible.

In Europe, car cameras will be required in the cars for the 2025 model year. Tesla cars have them in the U.S. Heck’s goal is to reduce the number of collisions by 90% or more. But he doesn’t think we’re going to get to 100% anytime soon.

Nauto can deliver information to the fleet so a driver can receive coaching based on the driving safety record.

“When you really look at bad driving, most of its unconscious,” he said. “Yeah, you have the crazy drunk teenager who’s trying to deliberately take risks. But for most of us, we’re not trying to get ourselves killed or deliberately be bad drivers. The risks we take are all things we didn’t see.”

How safety pays off

Nauto said its solution delivers rapid return on investment for all types of commercial vehicles from sedans to pickups to heavy trucks. It can help improve driver safety performance, reduce collision losses and save lives. It’s a driver-friendly solution that preserves the driver’s privacy, as it doesn’t record and send video all of the time.

Rather, when there is a “critical safety” incident, the camera turns on and Nauto compiles a full accident with video and sends that over to the company. In that way, it keeps most of the data at the edge and uses it to deliver warnings to the driver. Recording a video for the Nauto solution is purely optional; AI still works including distraction and drowsiness algorithms, without video.

Most of Nauto’s $500 cameras in place now are aftermarket fleet products. But the company’s specialty is software, and it doesn’t want to be a hardware maker. So it can embed its software in the fleets of cars that auto manufacturers are deploying and designing themselves. It’s a big market, as there are 32 million commercial vehicles on the road in the U.S. Those commercial drivers are twice as good as ordinary drivers, but they still get into wrecks and the insurance liability is huge. Given the low cost, the cameras can pay for themselves quickly in fleets if they reduce two-thirds of all collisions.

He noted that in 120 years of automotive history, the best safety technology was invented in the 1950s: the seatbelt. It eliminated about 20% of serious injuries. Heck wondered what would happen if AI could beat the seatbelt. Now he believes Nauto’s own record shows the crash reduction is much better.

“We can basically go into any fleet anywhere on the planet. And in a month, Nauto can eliminate two thirds of collisions. That’s pretty phenomenal,” he said.