My guest for this episode is Matt Moore, SVP of the Highway Loss Data Institute. At HLDI Matt manages the research function for HLDI's massive insurance and automobile dataset. In this episode we leap to the cutting edge of AI in automobiles and learn about how well some key safety features perform, such as:
In addition to rattling through those features and how effective they are we touch on all sorts of related topics: what the different levels of AI are, how safe the roads can get, the mix of hardware and software in modern vehicle safety systems and much, much more!
See show notes and more more at notunreasonable.com/podcast
Twitter: @davecwright
Surprise, It's Insurance mailing list
Linkedin
Social Science of Insurance Essays
My guest for this episode is Matt Moore, SVP of the Highway Loss Data Institute. At HLDI Matt manages the research function for HLDI's massive insurance and automobile dataset. In this episode we leap to the cutting edge of AI in automobiles and learn about how well some key safety features perform, such as:
In addition to rattling through those features and how effective they are we touch on all sorts of related topics: what the different levels of AI are, how safe the roads can get, the mix of hardware and software in modern vehicle safety systems and much, much more!
See show notes and more more at notunreasonable.com/podcast
Twitter: @davecwright
Surprise, It's Insurance mailing list
Linkedin
Social Science of Insurance Essays
My guest today is Matt Moore. Matt is a senior vice president of the Highway Loss Data Institute where he oversees much of HLDI's research. Mr. Moore joined HLDI in 1999 as a programmer. And before that, worked in higher education. He is an expert in evaluating how technological and social trends affect highway safety. Matt, welcome to the show.
Matt:Thank you much.
David:First question for today is, how safe can highways actually get and will we ever get there?
Matt:That's a tricky one. It depends on how we want to define safe highways. Are we referring to the roads themselves or the laws around those roads or the vehicles or the people that use them? In terms of actual highways, they're the safest roads we have. They're designed to not have crashes. And as a consequence, there's not a lot of crashes that happen there. But when they do happen, they tend to be very severe because they're high speed, which gets us to a problem today in that we have been continuing to increase speed limits on our highways.
David:I believe. I have this feeling that they've been going down for some reason. It can't be, no. They're higher. And so at one highway, same highway x number of years ago had a lower speed limit and now it's a higher speed limit.
Matt:So back in the 70s, after the oil crisis, basically the national highway traffic safety administration took some action. And as a consequence for many years, all states had a maximum speed limit of 55 miles per hour. And now in some places there are speed limits in excess of 80 miles an hour. And if you think about your high school physics for sequels, mass times acceleration, the acceleration goes up by the square of the speed. So a little bit more speed, gets you a lot more energy in the crash. We estimate over a 20-year period, we've lost over 30,000 additional people, that who likely would not have died had it not been for the increased speed limits. So the speed's a big problem. Vehicles are getting more crash-worthy, but at the same time they go a lot faster. And then we've passed laws that enable people to drive faster. And we're also more distracted than we've ever been before. So, we've legalized marijuana for recreational use. So, and same time we fitted vehicles with advanced driver assistance systems to stop cars from crashing. So things are simultaneously getting better and worse.
David:And overall though, in researching this, I saw some stats that show that mortality has define safety that way, right as I guess the compliment to mortality, right? The survivor probability of being on a highway is going up. So fewer people are dying as a proportion of drivers on the road. You look at the early days, say the 1920s, 30s, 40s, pretty terrifying from the perspective of today. And it really falls off over time. So the concern for the driver safety should be going down over time. Do you agree with that?
Matt:No.
David:We don't. Okay.
Matt:So on a per mile basis for the most part since the early 70s, fatalities have been trending down. But they likely would have trended downward even more were it not for some of these problems that I'm describing, like speed and distraction. And there are things where we could make additional ground. Like for example, we still have a hardcore group of people that refuse to wear their seatbelts. Approximately 15% of fatal crashes are still folks that aren't wearing their seatbelts.
David:So that would have been a pretty big innovation. Seatbelts sometime in the 70s, 60s, where were seatbelts?
Matt:At one point in time, manufacturers were allowed to fit seatbelts interlocks. You couldn't drive your car unless you were buckled up. Technology was a little bit sketchy. I think the public was not ready for them. And as a consequence, laws were passed. It prohibited manufacturers from using that technology. But we could save a lot of lives if people would just buckle up. I think we have this perception that far fewer people are drinking and driving and that's certainly the case. But if you look at fatal crashes, there's still a significant percentage of fatal crashes where folks are under the influence of alcohol.
David:One of the things I was wondering about as I was reading some of the literature that HLDI produces and reading some of your other work was whether you're an optimist or a pessimist about highway safety. I'm detecting in the conversation so far that you're a bit of a pessimist. What do you think?
Matt:I'm an optimistic realist.
David:Okay. So define that for me.
Matt:We've got to have hope for the future. I think in order to continue to do the type of work I do and that is to research a big piece of that, is researching innovations and innovations that hopefully will save lives. And I don't know that I could do that day in and day out without being optimistic and without the hope that some of these things that I'm working on will benefit drivers and mankind. On the flip side, I don't think I could be a good researcher without being a pessimist.
David:And somebody has to be the boring person who's going to raise their hand and say,"Did we look at the numbers and figure out that this stuff really works?" Because there's going to be so much optimism. Let's just call it hype around new technology because everybody wants to go this and wants us to go in one direction, right? Nobody wants less highway safety. So inevitably you'll help people just overselling it. And I guess it's your job to unspike the punch bowl a little bit.
Matt:Part of our DNA, it goes back to one of our first presidents, Dr Will Haddon, who was the original head of what we would now call the National Highway Traffic Safety Administration. And Haddon was infamous for going places where people were speaking and testifying and talking about highway safety and heckling them and yelling,"Where's your data?" And this was in the late 60s and early 70s. And it was under Haddon's watch that HLDI was founded. And these days, there are many, many, infinite number of companies that have the word data in their name. But if we could somehow do a internet search that was restricted to entities that existed in the early 70s, how many groups or companies had the word data in their name then. And then we were founded in'72,'73 and we were set up to do data and database analysis. And that's long been one of our bedrock principles is we say nothing that isn't based on data and science.
David:Well, what were some, again, I don't know if you're aware of this, would've been before your time, but I'm just curious of what were some maybe early wins that HLDI might have a racked up back in those days. What was it studying productively?
Matt:So, as I understand it, one of the first studies that we did was looking at collision losses by vehicle size and class. This was in the early 70s. This was at a point in time when small cars were just being introduced into the vehicle fleet. And the notion was that small cars cost less and as a consequence they should be discounted. And when we did our first report, what we showed was that the losses on these small cars were actually higher than the larger cars.
David:Do we know why? Why isn't really what you do guys do it so much, right? You're just really describing what?
Matt:A lot of what we do is what. And then we hypothesize about the why. And certainly some of the why was that there was a bias. A lot of young people were buying these small cars because they were cheap.
David:It's control variables that matter, I guess, right.
Matt:Control variables matter, but there's still some leftover.
David:Well, one of the things that we're going to come to, I'm going to list a bunch of technologies that I'm just interested in and I know I'm hitting you with this, but hopefully you remember enough of some of the results to talk about them. But before we do that, I'm just curious of whether you know which of the, I think you call them covariates in the papers, matter the most? So I'll read out a few of these. So there is calendar year, there's model year, there is the driver gender, marital status, garaging state, vehicle, density of the area. Why is that's an interesting one? Deductible range and you get something called risk, which I don't quite know what that is. You can define that. What is risk?
Matt:Risk is just standard versus non standard. Yes, of course, today there are many, many tiers of risks.
David:It's an insurance term. It's just not a vehicle or it's not like a fact of the person or vehicle.
Matt:It's an insurance term. Back in the day when we first started getting that variable in many if not most companies, that was just standard risk and non-standard.
David:Okay. Interesting.
Matt:And so we began getting that variable.
David:Do you document how because different people might define that differently?
Matt:Everybody defines it differently.
David:Do you try to standardize that in some way or you don't. Probably a good choice to be honest.
Matt:In some companies, it's the underwriting company. Other companies, an actuary sits down and looks at their 10 or 20 or 100 scores and says, everybody above this line is non-standard or below this line is nonstandard. But in the end, when we look in our data, if we're looking at say collision claim frequencies, the non-standard folks have a collision claim frequencies that are about 50% higher than the folks who were scored standard risk.
David:It's not exactly a qualitative judgment, but I suppose there's no concrete interpretation of them, really. But this is like the insurance industry's view of this.
Matt:The thing I often say when I'm questioned about it in front of highway safety audiences, I'll say,"Okay, how does somebody get identified as non-standard risk? Get arrested for DUI."
David:Sure. It's probably a lot of intuitive things.
Matt:Too many speeding tickets file a bunch of insurance claims.
David:So of those covariates, what's the elephant in the room with this kind of highway safety stuff? Is there one? Is there one?
Matt:I think the one that's just evergreen and it resonates with me because I've got a 16-year-old daughter now, it's just the age factor. And it's there. It's real. And I've got a son who's 14 and he's going to drive and just--
David:You're a bit terrified of that.
Matt:And then the other end of the spectrum my father's 86. And I don't know what makes me more nervous driving with my father or driving with my daughter who has a learners permit. There's some commonalities to the way both of them currently drive.
David:Well, I don't know if I've told this story on the show before, but I probably have. I'll probably tell it again. Within one week of getting my solo driver permit. I'm from Ontario. You have to have six months or eight months, I think it was of driving with somebody else in the car. As soon as I could drive on my own, within one week, I had crashed both my parent's cars. Not that it was that big of an accident, it could've been, it could've been.
Matt:For me I think it was four months after having my license, I crashed my brother's girlfriend's car. And then a few months after that I backed my father's pickup truck into somebody's vehicle.
David:And I think back to those two accidents that I had and what was going on in there, it was pure and simple inexperience, 100%. Both of those situations that I was in there, I wouldn't have gotten into an accident now certainly or probably at some point in between just to know what I was doing yet.
Matt:Failure to execute.
David:Make the wrong decision. Misidentify the situation. So any other insight into what causes accidents out there?
Matt:That's a great question. It's human beings. NHTSA produced a study that indicates that 94% of crashes are caused by human error. And when you think about, it's like, well, duh.
David:Who else is it going to be?
Matt:Who else is it going to be? So it's not a huge revelation, but knowing that number is important. But we often hear that number thrown around in terms of automated driving and advanced driver assistance. So it's not a huge revelation, but knowing that number's important, uh, but we often hear that number thrown around in terms of automated driving and advanced driver assistance.
David:Get rid of the human.
Matt:Get rid of the human. And all of a sudden, cars aren't going to crash. And it's a bit of a fallacy. We crashed because our ability to perceive the environment around us is constrained by our eyesight's, by the lighting conditions, by the curvature of the road. And vehicles that use sensing technologies are subject to similar limitations. Some of the crashes that we cause are not because we can't see or perceive what's occurring by the road environment in front of us, but we make bad judgments. And not to pick on Uber, but if you read the NTSB statement about the Uber fatality, one of the things that the NTSB pointed out was the Uber vehicle reclassified the potential crash partner, which was this pedestrian on several different times. It evaluated in several different ways. And although the vehicle reduced its speed, it didn't significantly reduce its speed. And so that was to some extent an error of judgment on the part of the vehicle. And there's some good technological reasons. I think there are a number of things we could point to and say, okay, this situation might have happened for any number of reasons. But all that boils down to the machines aren't necessarily much better at sensing the environment than we are. Although they can't get distracted, their ability to make sense of their environment is just as constrained as ours is. And one of the things I think about when I think about this problem is, if you think about looking at hurricane forecast maps, think about how huge the hurricanes are. Think about how slow moving they are. Think about how much computing power is brought to bear to try to predict the path of a hurricane. And then they show three, four, five, six different tracks all with widely different tracks. And then think about a vehicle needing to make similar decisions about other things in the road environment and then go walk in Times Square. And then you've got vehicles going multiple directions and multiple points of attack and skateboarders and bicyclists and pedestrians and horses. And just how a vehicle is to make sense of that driving environment, and that's an extreme example. But much of Manhattan is pretty complex driving environment. And you think about a machine attempting to perceive and make sense of that environment and drive a vehicle. And it's a monumental task.
David:Well, one thing that really struck me when I was reading one of your papers that, I think this was a survey of autonomous vehicles of some sort. So it was like let's just examine a few real world situations, bad outcomes. So there is a crash, there was an Uber crash as you mentioned a few moments ago. There is a Tesla crash as well. And it was going through, and this was a probably a classic hill, do you document debunking a lot of the hype and saying,"Hang on a minute guys, timeout. This stuff's not ready." And there was one story in there. Some of that just makes me laugh just because you're imagine there's a vehicle as stupid"entity" and it was misidentifying tree shadows and slowing down all of a sudden. And it kind of like jarring the people in the car because it thinks the shadow is a real thing. Who knows what it thought it was. State of AI as a technology pretty young.
Matt:Exactly. The science is in its infancy. If you think about the way radar had been used from its inception detecting large aircraft in the sky, there's not a lot going on in the sky. So that was one of the original uses for radar. And then we started using it for weather. But both of those things are incredibly different than mounting it on a car and trying to make sense of a very, very crowded road environment. And the manufacturers have had a lot of challenges tuning the radar to properly interpret what's happening in the road environment. And one of the cases that I'm aware of because some folks on my staff had these vehicles, there were some vehicles that had front automatic emergency braking systems, which were enabled by radar systems. And those systems were getting fooled by bridges and railroad tracks. And so you're on a road, on approach to a railroad crossing, and the elevation of the road that you're on and the elevation of the railroad tracks were just right. The vehicle with the AEB system interprets the railroad track that the metal in the railroad track as a stationary vehicle. And then the car drops anchor. And then take what I just said, but think about a bridge. You're on approach to a bridge and you're on an incline and some bridges have leading steel plates on them, and the AEB equipped vehicle would misinterpret those steel plates as a stationary vehicle and then the car would just drop anchor.
David:One of the things that puzzled me about some of these stories is you see the car response to certain, let's call it stimuli out there and it would just slow down. So think about, let's say the tree shadow was some physical obstruction. Slowing down doesn't really help. I guess it reduces the intensity to your point way earlier about the speed. But it seems like a pretty mild response to what it thinks of is as imminent danger. But maybe that's just why is it, it's almost as though it's a probability adjusted assessment saying, well, we've got a 40% chance that this is something that's going to kill us, so we'll drop speed by 40% and you'd expect it. But that's how expected values work, I wouldn't think. Am I right in my assessment of how this works? How much do you know about these algorithms in the autonomous vehicles? How much have you had to learn about that?
Speaker 2:I don't know as much as I would like to.
David:Have you asked? Have you tried it? Have they withheld it or is it changing so fast? So like there's no point.
Matt:I believe it's changing as we speak. Just yet another example, uou're talking about shadows one of those situations that I've encountered is some vehicles with adaptive cruise control systems will attenuate their speed based on posted speed limits. So you've got your adaptive cruise control set at 65. You're rolling down the highway, it sees a 55 mile per hour speed limit sign, so it drops the speed to 55. Now, some of these systems are not domain constraint, meaning the manufacturer says use this only on a highway, but there's nothing stopping you from using it elsewhere. And I'm guilty of using these systems in domains where they're not supposed to be used. And in one situation I was on a four lane road, but it was a residential road. Had the ACC set to the posted speed limit. I was rolling along through a school zone and the posted speed limit in the school zone was 20 miles per hour. But the vehicle couldn't contextualize that school zone marking. All it knew was there was this sign that said 20 miles per hour. And it dropped my speed from 40 to 20 and I was not expecting it. Fortunately, no one was behind me because they certainly wouldn't have been expecting it. So these vehicles are amazing in what they can currently do.
David:Relative to a vehicle that can do nothing.
Matt:Relative to a vehicle that can do nothing. But in some instances I worry that in the context of what they can do and the fact that we provide vehicle users no training or no moderation of their expectations. So,"Oh wow, yeah, I just wrote a hundred miles with my adaptive cruise control system on it. Everything worked just perfectly." Well, you're in the middle of the desert. And I think some of the problems we're encountering now with these systems is that they do some things very well and there's a lot of things they can't do. And drivers are still a bit naive about what the limitations the existing systems have.
David:Maybe as we're talking a bit about the methods that you use. So one of the things that you definitely do is pure data analysis where you get a whole bunch of data sets from insurers and maybe others. And you've tried to dig out from the data what's going on but you also do some testing, direct testing of technology. So maybe talk about the process and those are two very different things of course.
Matt:So, important distinction. So the Highway Loss Data Institute is part of a sister organization to the Insurance Institute for Highway Safety. So in the context of these advanced driver assistance systems, HLDI's studies were not only the first at the IIHS but also to some extent globally we had the first real world results in the performance of these systems. And based on the strength of our findings with, for example, front crash prevention, then the IIHS began doing on track testing and figured out, what were the key attributes to the systems that make a difference and then develop two tests in order to launch a consumer information program on that particular test.
David:So that's where they buy a car or a bunch of cars, run them on a track, smash them up, whatever.
Matt:So in this case, fortunately we have inflatable vehicles. So in this case, fortunately we have inflatable vehicles.
David:You do? Okay.
Matt:And the inflatable.
David:What?
Matt:Which is crazy, right? You think about it and like I owned vehicles that cost much less than these inflatable cars. Is these inflatable cars--
David:What are they made of?
Matt:It's like a moon bounce. Like moon bounce material?
David:Yes, sure.
Matt:So like heavy duty, like a vinyl plastic kind of thing. But they have to have a shape.
David:Rigidity a little bit to it to hold the right.
Matt:Rigidity. So it maintains the shape of the vehicle. It has to be fidelic enough. It has to look enough like a car to trick the optical sensors on a vehicle, the camera based sensors. And then it also has to have enough metal and other things in it to fool the ultrasonic sensors and the radar based sensors and the lasers. So they're expensive. But the great thing is you can hit them a hundred or a thousand times and you won't damage the inflatable vehicles. And for the most part, you'll won't do much other than cosmetic damage to the vehicle you're testing.
David:Just earn the details here. Who gives you their legit vehicle? So let's say as a Tesla model, whatever it is free or something, and then you're going to be,"Let's just take this sucker out for a test." And we got all these blow up vehicles everywhere. And then who gave you the Tesla?
Matt:We buy them.
David:You buy them.
Matt:We buy them all.
David:And then what do you do with them after?
Matt:So, some of them we keep and do repeated testing. So, uh, some of them we keep and do repeated testing.
David:Sure. Some of them break. That's the end of that.
Matt:But then we do crash worthiness testing, which is we crash them.
David:You send to the slaughterhouse.
Matt:We send them to the slaughterhouse.
David:The glue factor. Let's smash this one up and then it's sold for scrap.
Matt:And when we do testing on a brand new vehicle design, one that's never been produced before, something we've never tested before, that's five or six destructive tests.
David:Okay. And how many of these tests do you do? Is this like ongoing? Hundreds of cars go through this place really?
Matt:Yes.
David:Wow.
Matt:So we do testing for front automatic emergency braking. We do headlight testing. At some point we should circle back to headlights because it's a pretty cool story. And then we do a variety of destructive tests on the vehicle to looking for how well does it protect drivers in front seat, passengers in frontal crashes, side impact, rollovers. We do a lot of testing.
David:Okay. Well let's hit the technology list and I've got a list here. So we'll go through them. And headlights happen to be number two, so we'll come to that next. But front automatic emergency brakes. So what are these? And what they do? And how good are they?
Matt:Sure. Vehicles fitted with these systems use some type of front looking sensor. And there's three primary classes of sensing technologies used. There's radar, there's lasers, there's cameras and then any combination of the three of those.
David:This is Lidar, which has lasers and radar together, I guess?
Matt:Lidars, actually that's just laser. And then radar is radar. And then cameras are cameras. And then you'll hear some folks talking about sensor fusion, which is some combination of those things. And oftentimes the camera's in the mix because if you're going to do some form of self-driving, you need a camera to read the lane lines and to look for an index and identify potential crash patterns.
David:So front automatic emergency brakes use, which ones of those did you use for those ones, that has radar on it or it has all three?
Matt:It can have all three.. And we see some really big benefits there in the insurance data. And when we look in police reported crashes and when we were able to dial in and look at the crash is specific to those systems, so front to rear crashes, big, big benefits like 65% reduction in the incidents of injury producing front to rear crashes.
David:Holy Cow. That's amazing. And well, so what's the denominator in that and what's the numerator in that calculation? That's so huge. And um, well, so what's the denominator in that and what's the numerator in that calculation?
Matt:So in that one, we use insurance data as the exposure. Insurance exposure as the denominator and that was on a per state basis. And then the numerator was injury producing police reported front to rear crashes.
David:And so that ratio shrinks by 65%. That's amazing. That's so huge.
Matt:Now for like front AEB for property damage liability, the reduction is 13%, which is also a big reduction. Now that's 13% of all PDL claims because in our data, we don't have the ability to drill in on just front to rear. But 13% that's a big number.
David:Right, and so 65% that was on front or rear only, is that an estimate? Right. And so 65% one that was on front to rear only.
Matt:Front to rear only police reported injury crash.
David:I see. So the big dataset is that 13 number which you can't really pick apart from the front to rear only one. And then you have this subset which is police reported crashes? Okay. So that's a win. How long have those been in place?
Matt:The first systems that we studied, were as I mentioned earlier, the Mercedes Distronic System and they'd been in the US market like 2009, 2010. And then the first system that was in the US in big numbers was the Volvo City Safety System, which was fitted to the XC60 in 2011, 2012, somewhere in that timeframe.
David:I have a Chevy Tahoe from 2016 or 2017 gosh, can't remember, I think 2016 And there are some of these features on it. The front automatic braking one, I don't think of it as stopping the car, but it alerts and it'll jerk me sometimes if things come up. Is that what we're talking about or is there something else going on in that?
Matt:That's what we're talking about. And some manufacturers will attempt to bring the vehicle to a full-stop. Others will only do mitigation braking.
David:You can brake check. Just taps it for you.
Matt:And then some vehicles, like for example, the original vehicles with Volvo vehicles with City Safety, they were using that Lidar based sensor. That Lidar sensor is relatively short range. So that system really was only functional up to about 18 miles per hour. But the reductions were really dramatic because if you think about insurance claims, the lion share of them are low severity front to rear crashes. And that was the heart of what that system was aimed to prevent.
David:And one of the things that amazes me, a little Segway here generally to the LD is that you name names. Like you'd go through the brands and the makes, and you give these assessments on each one. So is there any differentiation between the different brands on their quality of the front AEB?
Matt:So, that's a generalization I'm not going to make. What I will say at least in terms of testing, the differences are really vehicle to vehicle. You could have two manufacturers that have a vehicle that gets our top score in front crash prevention and then one that doesn't get a top score. It just depends on the implementation.
David:And one more to aggression. Well, how good are those JD Power Scores? Is that just marketing fluff or how good is that stuff?
Matt:Not going to comment. I haven't looked at them.
David:Great. Car adaptive headlights, how are those?
Matt:We don't see well at night. And what's interesting is when we first started looking at all of these technologies, we were very surprised by the results for the car adaptive headlights. And it was actually one sitting in a conference room with an engineer that had designed one of these lights. He said to me,"I don't believe these results." He said,"I knew they looked cool and they did what they were designed to do, which was rotate in response to the steering wheel input." But he said,"I really wasn't expecting any safety benefit." And we found benefits for car adaptive headlights but also for automatic high beams. And we scratched our heads about these results for a long time. And then the IIHS conducted a lot of human factors research to try to understand where all these benefits were coming from.
David:What were the benefits? Can you quantify them? Do you have those numbers? So maybe vaguely like them?
Matt:I was afraid you'd ask me for them?
David:Yes, sorry.
Matt:In the neighborhood of like 5%, 6% on PDL.
David:Reduction and frequency.
Matt:Reduction in frequency. I guess someone recording here, so I can't say, don't quote me on that, but if the number's wrong, don't play me.
David:Well, I can post them some show notes that will have the real numbers. I didn't actually bring the papers in here.
Matt:Got it. What we found was that a lot of lights are named in the right place. So although the vehicles today are fitted with these incredibly bright lights that throw enough light and heat to melt the MacAdam.
David:They're amazing.
Matt:They're just not aimed in the right place. And some of that was some looseness in the manufacturing tolerances. And what we found was that in almost every case, the adaptive headlights were compensating for the bad aim of the other lights. And so we launched these testing program to help ensure that these vehicles that have these incredibly expensive lights, that the lights were aimed in a place where drivers can benefit from them. And if you think about front name tagged airbags or any sort of airbag, most of us will never thankfully experience an airbag activation. Most of us will very rarely experience an electronic stability control activation.
David:What is that?
Matt:ABS-Generation 2.0. All vehicles had been required to have it since about 2007 and it uses the rotational sensors from the ABS system to detect and also your sensor and a steering wheel input sensor. And basically if the vehicle decides based on those inputs that the vehicle isn't going where you want it to go, it'll either--
David:You're spinning out or something or sliding.
Matt:Sliding. If the vehicle senses that you're doing something you don't want to do, it can either fire or release each of the brakes independently to bring you back into the intended line of travel.
David:Wow. It's pretty sophisticated.
Matt:Good stuff. But most of us will never experience one of those activations. By contrast headlights, every time we get in the car and it's dark, we either benefit from or are negatively impacted by the quality of our headlights. And what's even more impactful is if you think about the fact that these vehicles are using camera based sensors to make sense of the road environment. And those cameras can only see what you and I can see. And if you look at on our website, we have these silhouettes of the range of the light beams for the various ratings. And there is an enormous contrast between the length of the beam from a poor rated vehicle versus the length of the beam from a good rate of vehicle. So these lights confer a primary benefit in terms of helping us see when we're driving at night. But in addition to that, they help support the onboard camera based sensors and improve the quality of the benefits from systems like frontal AEB.
David:Unintended benefit. Do you study every technology that hits a car? So if the manufacturer didn't expect that one to turn up a positive relationship, who did? How did it hit your radar screen? How did you know to test it?
Matt:So one of the first manufacturers that we were evaluating had them on some of their vehicles and they were willing and able to provide us with the data. So one of the first manufacturers that we were evaluating had them on some of their vehicles and they were willing and able to provide us with the data.
David:So somebody else had the hypothesis and they said this is probably happening and so you guys dig into it. Next one, lane departure warning.
Matt:LDW. I've lost a lot of sleep over that one. LDW I've lost a lot of sleep over that one.
David:Why? How?
Matt:So if you look at the insurance results, there's no benefit. None. And that was one of the technologies early on. The very, very first thing we did at the IIHS in terms of really trying to impose any rational thought on these systems was to say what crash modes are relevant to these technologies. How many crashes could be prevented. And single vehicle run off road crash, that's a particularly injurious type of crash mode. A lot of people die that way.
David:So single vehicle, so I lose control of my vehicle for some reason.
Matt:You're going too fast on a highway. You're going around a bend, you lose control and off you go. And so these systems are supposed to help prevent that. But we've never been able to find a benefit for these systems in the insurance data. Now, when my colleagues in the IIHS were able to look at police reported crashes and drill in on those crashes that are relevant to that system, what they found was about a 20% reduction in injury crashes, but that estimate isn't statistically significant. So the data's just too thin. In terms of all police reported crashes, the benefit is about 10%.
David:And that's statistically speaking insignificant benefit of 10%.
Matt:The 10% is significant, the 20% for injury is not. But in the insurance data, we've not yet found a benefit under any coverage for these systems. And then one of the things we found out in subsequent studies, we'd stationed research assistants at car dealerships looking to see which of these technologies were turned on and which were turned off. Front crash prevention was left on over 90% of the time. Whereas, lane departure warning systems or other lane maintenance systems, were only left on about 50% of the time. So lots and lots of people are shutting them off.
David:I didn't even know. I didn't know how to turn those things off on my own car. For the better, I'm sure maybe worth another detour here into the idea of statistical significance because one of the things that surprised me as I was digging through some of the research was sometimes there isn't very much efficacy, there's a big error bands on some of the results. And maybe you can comment on that about how important it is to have statistical significance. You do publish the results anyway. So that was a risk factor that you give somebody the wrong view.
Matt:So anytime we publish our results where if the finding isn't statistically significant, we always make certain to draw attention to that fact. And oftentimes if we have a pattern of results, like with lane departure warning with multiple studies, we're getting the same answer but it's not statistically significant, that's significant if I'm making sense.
David:And just maybe to define it too, I'm not sure the whole audience would get it though but they probably do. But that means there just isn't enough data to narrow the range of results that you're receiving. And so you're not sure because you're still seeing enough variability. And overall, some of these studies have more and less significance. What influences the significance levels that you have to cut the data down too far or what tends to, it'll push one way or another.
Matt:So it's two primary factors. It's the volume of data that we have and then how much signal is coming from whatever it is we're attempting to research. So if we've got a technology fitted to a vehicle that doesn't sell in small or in large numbers and the take rate on the technology is relatively small, and the benefits of the technology are small, it's going to be hard to achieve statistical significance even with the database of our size but contrast almost anything we would look at. For Honda Accord, for example, given the high volume of sales there, and the size of our dataset ends up being statistically significant.
David:How big is the dataset?
Matt:Depends on how you want to define it. We've got lost data on over 400 million vehicles give or take. Someone's likely going to say, but there's not 400 million vehicles on the road in the US today but we have the lost records of those vehicles when they were on the road. And so depending on what we're looking at studying, sometimes we use some part or all of that dataset. So it's fair to say we've got over 400 million vehicles.
David:Big number.
Matt:Big number.
David:Let's move into the next one. Blind spot warning. Blind spot warning.
Matt:Good stuff.
David:It works.
Matt:We expected it to work. The insurance data says it works. The police reported data says it works. It works. I've got nothing bad to say about it at this point in time.
David:Magnitudes, haven't they one of those magnitudes numbers.
Matt:It's another one in the 5% or 6% range depending on the coverage side.
David:Parking sensor.
Matt:Can we talk about parking systems in general?
David:Let's do it.
Matt:So the interesting thing there is for both backing sensors and parking cameras, we see a reduction in property damage liability of about 2% to 3%. The interesting thing is for--
David:I hit somebody when I'm parking.
Matt:Exactly. Parking sensors are associated with about a 1% reduction in collision frequencies and when you consider just how many collision claims are for below three grand. And I just looked at some tests we did today on rear automatic emergency braking. And at three miles per hour on a Cadillac SUV, we generated about$3,000 worth of damage. And on a Subaru Outback, it was about$1,800 worth of damage. So you can do a lot of damage at three miles per hour. So there's a lot of claims--
David:That's mostly paint some bodywork I guess.
Matt:And then bumpers, lift gate that's like the test we did on the Cadillac was into a pole.
David:So making the biggest damage. just gets damage.
Matt:And or the floorpan depending. So it can add up very quickly. So we were very disappointed in those results. Sensors are associated with a 1% reduction in collision frequency, whereas the backing cameras are associated with a 1% increase.
David:In frequency.
Matt:In frequency.
David:People are just going to close to the edge.
Matt:At this point I can't say that we, we know. It's rational to hypothesize that this is a system that changes the driving task. Like your airbag doesn't change the way you drive. And as a consequence, you'll never get me to say that I think people drive differently because their cars have airbags or electronic stability control. All these things make you safer but doesn't change the way you interact with the vehicle. Therefore you don't drive any differently as a consequence of having those systems. Having a backing system though that provides you with that visual, has the potential for you to, you know,"I'm not sure if I want to buy a park in this space. Well you know, I've got the park assist system now, I'll take a shot at it." And so people might be attempting to back in spot parking places they wouldn't otherwise. But the other thing is, we know from another study that we do, if you have test subjects in vehicles with backing cameras, they spend over 20% of their time looking at the monitors, which means they're not spending around in the mirrors over their shoulders and around them. And on a lot of vehicles, there's a lot of area around the vehicle that's not visible in that camera. So while you're looking at the camera monitor, it's providing good, useful information about one of the most critical areas. But there's still a lot of crash critical area around the vehicle where a situation could emerge that's potentially risky that you've got reduced awareness of because you're hyper focused on that monitor.
David:You're looking down into the right instead of around you. Well, I feel like when I'm backing up with the rear camera in my cars, that there's quite a lot I can't see without the camera. I don't know what it's like in other cars, but I find that there's a lot less visibility, particularly in large SUVs. If you're looking around your shoulder or even through the mirror, then you really would get with a car or maybe you do once upon a time.
Matt:Does your SUV have a 360 camera or just a rear camera?
David:Just a rear camera.
Matt:Got It.
David:Which I really like. It gives me more confidence which is maybe the problem. I guess it's the point. So that's a mixed bag or maybe just negative. Rear automatic emergency brakes.
Matt:Really good stuff. And what's interesting there is, it's similar to me in the contrast between forward collision warning, which is good in front automatic emergency braking, which is really good. If we look at the parking sensors, the backing cameras, minimal benefits. But if you add in automatic emergency braking, you stop that car from backing into something. Big, big benefits.
David:So that's my list that I have. Any safety features that I haven't touched on that you like or don't like? Maybe a different question. What's the most overrated and underrated feature?
Matt:Overrated and underrated?
David:People think it's great, but I maybe that's the lane departure warning.
Matt:I don't know, given that 50% of people turn it off, I think--
David:Fair enough.
Matt:Underrated. I think the underrated technology today are headlights. And even not necessarily adaptive headlights, but just having a good set of headlights can impact your safety in everyday driving. I would have to say in terms of technologies, lights are underrated. I don't know that I would say that any of them are overrated. I would say that lane departure warning systems are not living up to expectations. I think a lot of it it's a matter of the manufacturers and the human factor scientists figuring out what's the right way to have the driver interact with the vehicle or maybe we get away from lane departure warning systems and just shift to active steering systems where the vehicle is just correcting on an ongoing basis.
David:There's another idea that could pop up a couple of times in some of the papers that was associated with increased repair costs, potentially associated with some of these. Maybe you can talk a bit about that. One of them that just jumped out at me was an interesting twist to it, but the glass costs going up. Maybe if that's a good one and maybe if there were others too you want to talk about.
Matt:When we first studied Distronic, the Mercedes Distronic system, that radar unit was a$2,000. The radar unit that enabled that technology was mounted behind the Mercedes logo. And that radar unit could be damaged in any sort of crash regardless of whether it was on a--
David:The first thing that gets touched almost.
Matt:It was associated with big increases in severity.
David:Because it's, replace the hardware.
Matt:You had to replace the hardware. And in some instances I've been told that, the radar unit had a glass facing on it. And in some instances people were breaking, like having a road debris break those. And so they were being repaired under no deductible glass under comp. And nobody expected that. And so when the first camera based systems came out, we thought, this is great. We're moving that expensive gear away from the front end of the vehicle and end of the occupant compartment where it should be safer. And then next thing glass loss has started to take off. And so you've got a more expensive windshield. But there's other stuff in the windshield that's contributing to glass losses. But there's remounting and recalibration costs associated with these windshield mounted sensing systems. So it is a problem. And there's no standard calibration procedures. Some manufacturers have much longer calibration procedures than others. So it's an issue we struggle with.
David:I'd like to just pause on that for a second because I just found that point. I just sat back and thought, wow. So the hardware is dirt cheap, right? It's probably like smart phone camera or something like that. So probably an OEM part that they use in android phones or something like that, right that gets plugged onto the car. But it's the calibration of the software that actually costs the money. So you've got to bring somebody in, pay him$400 to calibrate the stupid camera. So it was not even the stuff that costs money, it's the labor associated with the lack of standardization and bad software.
Matt:Well, and there is the labor hours associated with mounting the camera as well, and reconnecting the technology. Not that that's necessarily a longer, very involved process, but it's time that we didn't have to spend before to fix a windshield. And then yes, you're absolutely right. In some instances these vehicles have to be driven some number of miles before the calibration is complete. Some of them involve special measuring devices and jigs and things. I had to replace the windshield on an older vehicle of mine. And the technician was able to just come out and do it in the parking lot. If that vehicle had had a camera based sensor on it, probably wouldn't have been able to do the repair that way.
David:Any other technologies that contribute to the excess costs?
Matt:So front mounted radar as a contributor and advanced lighting systems. Some lights can cost as much as$3,500 a piece.
David:Whoa.
Matt:And if you were to remove a bumper cover on a lot of vehicles, what you'll find is that the bumper bar or the bumper beam often stops at the frame rails. So there's no solid structural member under the bumper cover under the headlights. So it's an unprotected location. And you've got one of the most expensive components in the front of the vehicle unprotected and mounted in one of the most vulnerable positions of the vehicle. But then there's interesting signal we get from some systems. For example, the rear automatic emergency braking systems are associated with increases in severity. And that was a head scratcher because those sensors aren't all that expensive. And when we decompose the data, what we found was that it was an issue of mean shifting as a problem.
David:Sure. The smaller ones are going away.
Matt:A lot of smaller ones are going away.
David:Get smaller denominator. The bigger ones are still there. And I see that one a lot. One thing they really interest me in this and this puzzle's partly solved by that point about the labor of reinstalling that device is vehicle costs aren't really going up, right? So if overall repair costs they're going up.
Matt:They're exploding.
David:Are they?
Matt:Cost, sorry. It's insane.
David:Really?
Matt:Yes.
David:Let me come back to that in a sec. Actually a couple of hit on that for a second. There's some data I found online that just said new vehicle costs and this is the CPI index. So consumer price index shows that the new vehicle cost index is not really increasing. It hasn't been the last, I don't know, 15 years or something like that. You think this is wrong, I got in the wrong place?
Matt:So, average car prices have been exploding. And in used car prices have been also taking off. And the other thing that's happening is the price gap between the minimum price and the maximum price on vehicles has just exploded. So you can get, and since this is being recorded, I'm hesitant to throw out the names and numbers.
David:Well, we can put some links up to this stuff afterwards.
Matt:So roughly you can buy a base Honda Accord for 25,000, give or take. And you can easily option an Accord up to$40,000. And so the range of prices that you can pay for vehicles is just exploding. And the other thing is content that at one point in time would have been thought of as the stuff of luxury vehicles in a Honda Civic, it's--
David:Going under the base model.
Matt:You don't have to go too far up the foot chain on a Civic to get leather seats in a Civic or a alloy wheels as opposed to rim or steel wheels with hubcaps.
David:Well, so that actually reassures me in some ways so an explanation and that I'm going to come up with. I'm going to look into it before I publish. And we'll put up some links to really resolve this. And maybe we'll exchange some emails about it too. Is that, depending on how, let's say my observation is right and didn't just screw up the Google search and the CPI data doesn't show necessarily an increase in new vehicle costs over time, that could be very well modeled dependent, right? So that might be, I can pick the base model of some simple car and that maybe it hasn't changed too much, but nobody buys that. People are actually buying something a bit different and they're getting more upgrades. And so the cost as realized is increasing.
Matt:Well, thinking about your own situation. So number one, 10 or 20 years ago, vehicles like the Tahoe did not sell in any numbers. And so one thing that's happened is just the mix of the fleet has changed radically. And while the percentage of the fleet that is comprised of pickup trucks hasn't changed much, nobody drives our grandfather's pickup trucks anymore. And if you think about the old Ford, the old Chevy two doors with a long bed and vinyl seats that's just no more. The vast majority of pickup trucks that are sold are of the four door variety. And it's not uncommon to pay$50,000,$60,000 for a pickup truck.
David:So it shouldn't be too much of a surprise then that repair costs are increasing collision damage. It should be a lot of inflation in those numbers over time. And that's what you're saying.
Matt:Yes.
David:So one thing I wanted to also touch on here was, what are all these features add up to and that is kinds of AI we're talking here, right? And so there's a scale of levels of autonomy that they rate cars on and you make reference in a few of the papers to level two autonomy. And that at least I think that that's the more what we think of as the real driverless car, the beginnings of the driverless car, right? Autopilot kinds of things. Do you happen maybe if you can help us what are the levels of autonomy one through, I think there's five, do you know what they are? One is maybe this sort of lane departure warning, would that be right?
Matt:Depends on how you want to think about it. So level two, the key considerations for the levels are, is it capable of handling acceleration and deceleration? Is the vehicle capable of steering? And so for a vehicle to be a level two system, it has to be capable of sustained acceleration and deceleration control and also handling the steering sustained control. And then it relates to who is responsible in the situation of a fault. And in the case of the level two vehicles, control reverts to the driver.
David:The driver can just yank it out of its control spiral.
Matt:Or if the vehicle it's having trouble sensing the environment.
David:Driver decks back.
Matt:The driver's responsible. Like with the highest levels of autonomy level four, level five, the vehicle itself is capable of sustained control. The vehicle is responsible if the system goes into fault. And the difference between four and five is that, for this domain constraint, meaning that the vehicle, like for example--
David:Highway versus city.
Matt:Like Cadillac Super cruise system. That is highway constraint. Some systems are going to be only low speeds. So they're only operational to like 25 miles per hour. So they would be level four systems because although they're capable of full autonomy and fault control, they're constrained to a domain. Whereas five it's all car all the time, anywhere.
David:One other thought that I had here and we're running out of time, so maybe we can close on this topic or one just related to it, is hardware versus software. So big, big, big picture, technological development. There's been a shift, maybe more the last period of time that a lot of the value that's being added by the tech industry's on software. Software which scale, software to get pushed down. Famously, Elon Musk downloaded an update on the Tesla and suddenly the braking system worked better. I don't know if you're familiar with that story. So how much of this development is hardware versus software? Where are we right now in the frontier because you think like the early days of HLDI that was all hardware, right? So at some point the software barrier was breached. Now there's some software, quite a lot of this stuff is software base, right?
Matt:Yes. And to the extent that as you pointed out, Tesla, improved the braking of the vehicle, they can improve the acceleration of vehicles. So we're at a point now where radical changes in the way of a vehicle can perform, can be done just with software.
David:That's amazing. Isn't it?
Matt:Yes, it is. And if you think about the level of anxiety you have when you take a major update on your phone--
David:And you jump in your car and tell it to drive for you.
Matt:I can't imagine the anxiety that I would have if I was updating the software on my vehicle now granted I haven't had big problems with software updates on my phone in a long time.
David:You have to get better at it. They are getting better at it. Technology matures. It does.
Matt:And it's been a long time since I've had problems with an update on my laptop. This morning I was about to do a presentation, went to shut my machine down and it said it was going to apply an update and I wasn't all that nervous about it, which three or four years ago, that would not have been the case.
David:So are you an optimist on this then?
Matt:I'm optimistically realistic or pessimistic.
David:How long before this becomes, let's say you're ridic at this prognostication.
Matt:So in terms of how to define this, and I think to me this would be a vehicle at a price point of, say the Honda Accord or the Toyota Camry, being capable of going anywhere in any time using any route without the driver having to take control. I'm just going to throw out and say at the earliest, 30 years.
David:You didn't say infinity, which is a reasonable answer to that question.
Matt:And I think depending on what our tolerance might be for domain constraints and or for modifying driving environments, like just thinking about the example of the tolling and the most congested places in London, if we were to designate environs in cities, whether it be certain roads or certain blocks within certain radius where vehicle can drive itself, we can make a lot more progress a lot faster. But without concessions to modifying the domains and having this requirement that, which is sort of in my mind, very American anywhere, anytime, any way, that's a harder problem to solve.
David:And that's a legal regulatory problem, am I understanding you right there?
Matt:To a large extent, it depends. I think that the city would have the, you know, just like a city can designate a bike lane or a city can designate a road as a toll road or a one-way street. Cities likely would have it within their purview to designate lanes or specific roads for use by autonomous vehicles. And if we were to do that, then we could have increased signage and other things, cautioning pedestrians to say,"Hey, there's autonomous vehicles operating here. You really want to think twice before you jaywalk."
David:Humans don't come here. And then the autonomous vehicles can communicate with each other. They don't have to use the radar. At least not for detecting each other. Maybe they have a network, who knows, right?
Matt:Exactly.
David:So one last question and then you've got to go. What do you think is the thing that's holding back development that maybe we can control an autonomous vehicle? Is there anything in particular that you think you'd change to make this improve faster?
Matt:As a safety advocate, no.
David:Okay. Do you think it's going just fine?
Matt:I think it's going--
David:Maybe too fast.
Matt:Just fine if not a little. We've lost a life already.
David:But how many people have died from regular car accidents in the meantime? Can you attribute it to that? Was that person not have died if they've been in their car. What probability, right? Here's the question, I guess. Do we think of it differently when somebody dies because of an autonomous vehicle than when somebody dies at the hands of a human or not. Does it matter? Does it make a difference? Accidents happen.
Matt:I hesitate to use the word accident, but I think that's exactly it. We as humans are tolerant for the most part of our faults and fallacy.
David:But you can download the report from the AI and say that was a dumb decision, but you don't know what the hell was going through the head of the guy that--
Matt:And then I think we're more accepting of accidents caused by people than machines because in the end, an accident caused by a machine is a problem caused by a human.
David:My guest today is Matt Moore. Matt, thank you very much.
Matt:Thank you, sir.