Full Transcript

Full Transcript

Transcript

Nathan Labenz: (0:00) Hello and welcome to the Cognitive Revolution where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathanlabenz joined by my cohost Erik Torenberg. Hello and welcome back to the Cognitive Revolution. Today, I'm excited to share my conversation with Timothy b Lee, noted autonomous vehicle industry analyst and author of the Understanding AI newsletter online at understandingai.org. Tim has been covering the self driving car industry since 2016, and his in-depth analysis of the various technology stacks, business models, and regulatory issues that have shaped the space make him the perfect person to answer all of my many questions on this topic. Over the next 2 hours, we characterize the current state of the art in autonomous vehicles, starting with our personal experiences with both Waymo and Tesla. We unpack the contrasting approaches of these industry leaders, including their different hardware stacks, data acquisition methodologies, and deployment strategies. We examine the technical hurdles that are still facing self driving cars from handling edge cases like emergency scenes to expanding beyond well mapped areas. We interrogate company published safety statistics and explain why it remains difficult to definitively prove that autonomous vehicles are indeed safer than human drivers, though it does seem like Waymo is getting close. And finally, we explore the regulatory environment, starting with the very notable fact that government oversight has not been nearly as restrictive as many might have feared. Along the way, Tim also explains why some previous predictions have proven overly optimistic and shares his current expectations on timelines for widespread adoption. We even explore the potential for new vehicle form factors once human drivers are no longer necessary. Throughout, Tim provides a very balanced view. He's clearly very excited about the fact that today's leaders are finally getting to the point where their products really work. Today, Tesla allows human drivers to take their hands off the steering wheel, and Waymo is beginning to scale truly driverless cars. And yet, at the same time, Tim is realistic about the fact that it probably will still take a few years before self driving is very broadly available. For my part, driving is the most dangerous thing that I personally regularly do, and many people spend a lot of time doing it. So I really can't wait for AI driving systems to become the new normal. As always, if you're finding value in the show, we'd appreciate it if you take a moment to share it with friends or to leave a review on Apple Podcasts or Spotify. You can also DM me on your favorite social network, and we continue to invite your resumes if you're an aspiring AI engineer or AI advisor via our website, cognitiverevolution.ai. Now I hope you enjoy this conversation about all things self driving with Timothy b Lee, author of understandingai.org. Timothy b Lee, author of the Understanding AI newsletter at understandingai.org. Welcome to the cognitive revolution.

Timothy B. Lee: (3:09) Thanks for having me on. I've been listening to the podcast pretty much since the beginning, so it's exciting to be part of it.

Nathan Labenz: (3:14) Thank you. That's flattering. And, I'm excited about this as well. I've been a follower of your analysis of the self driving car industry, both through your public analysis and through many 1 on 1 questions that you've been gracious enough to answer. So what I'm excited to do today is just try to lay out for listeners kind of beginning to end, like, where are we in this self driving car journey? Ultimately, when do I get my car that I can sleep in on the way to my grandmother's house? And there's obviously a lot to cover between here and there. So, you know, kind of all angles is the goal, and you're the man for the job. You wanna start off with just a kind of simple overview of the levels. I think people will be familiar with the at least I've heard this before, but it's probably a good way to introduce, and I think you have a little bit of a contrarian take on it too. So you can give people your sense of what really matters in terms of the levels of capabilities in self driving.

Timothy B. Lee: (4:06) Sure. So there's these standard SAE levels, levels 1 through 5. And level 1 is just a regular car with maybe like regular cruise control. Level 2 is driver assistance. Level 4 is a full self driving car like Waymo, but in a limited kind of area. And then you have a level 3 of somewhere between those where it's like some kind of shared responsibility between the car and the driver. And a level 5 is supposedly like a car that can go anywhere. But I don't find these levels to be super, super helpful because especially that level 3, we've never been totally clear. I guess the idea is a car where you use it in the driver's seat, but the car mostly drives itself. But then if it gets a problem, it hands over control to you. And that just seems like a kind of user experience nightmare and a safety nightmare because if the car suddenly has a problem that you haven't been paying attention, it's hard to be sure. Are you gonna have the contacts to make the safe decision? Or what if you're you're turned around or you're putting your kid in a car seat or something? I think a much simpler way to think about it is there's 2 kinds of vehicles. There is driver assistance, which is any of the Tesla or any other automaker has lane keeping driver assist kind of things. And that's the system where the driver is still responsible. The car might help. The car might do a lot of work, but you are still supposed to have your hands on the wheel eyes on the road. And then there is fourth, like, fully driverless vehicles where you're not on the steering wheel. The car is legally responsible. And so far at least, that always comes with a geofence where there's some places where it'll go and some places it won't go. I don't think that level 5 is really a useful concept where there's no geofence because there's always gonna be a service area. There's always gonna be countries or types of roads or types of off roading that the vehicle doesn't do. I think about a little bit as cell phone service. Right? To first approximation, like, could use your cell phone anywhere. Right? But that's not really true. There's areas that don't have cell phone towers, and nobody says, oh, that's not like a real cell phone because it doesn't, you know, go in in rural Alaska or something. And so I think self driving cars are similar for the foreseeable future. Vehicles that are driverless are going to have a service area. And the question would be like, what is that service area rather than can this go anywhere or just some places?

Nathan Labenz: (6:01) It might be interesting as we get deeper into the technology stack and the different approaches that different companies have taken to trying to understand better to what degree that limitation will be a matter of, like, connectivity in the same way that it is with a cell phone or, you know, having a map or not having a map versus if it's a self contained system that's all onboard on the car. Is it something where the car is just saying, sorry. This looks too, you know, crazy for me to go any further.

Timothy B. Lee: (6:30) I don't think it's about about connectivity. It's partly about the map, but it's also about things like regulatory questions. Like, in the future, assume you'll need a license to operate a taxi service in certain areas. And also, if there's no driver and the car gets stuck and there's nobody in it, like, somebody's gonna have to come service it. And so I think if you think about it as a service as opposed to a just a car, any company that offers a service has a certain set of areas where they offer the service in areas where they don't, the car is only gonna operate where the company has service. Now there might be a feature where the company literally has a footprint everywhere and so can pretty much offer service everywhere, but I think that's quite far away.

Nathan Labenz: (7:02) Let's maybe talk for a second about the 2 canonical companies that are that most people have experienced. I think still most listeners probably have not experienced either of these technologies. I think you and I are both in the the lucky few, and I have to get credit you for getting me a connection at Waymo that got me off the wait list and allowed me to actually get into the Waymo a couple times last time I was in San Francisco. So thank you for that. The 2 are Tesla and Waymo, and we can get into other, you know, companies as well. But Tesla, it sounds like you put into the level 2 category. Last time I used it, you can add on to this with your own reflections on the experience, was for me about a year ago. It was last summer. I took an 8 hour round trip to actually take my grandmother home. It was both my and her first FSD experience together. And I would say it was very strict about enforcing that you are paying attention. If I understand correctly, it has a camera on you from the rearview mirror that's monitoring to make sure you're alert and engaged. And you also had to have hand on the wheel, with a little bit of torque, which was something that took a little getting used to for me in and of itself. Otherwise, would say, you you're not paying attention and execute, like, a pullover and a shutdown of the self driving experience. So that was, like, pretty short leash on the human in the seat. I didn't find the experience to be overall very good. Like, I came away extremely impressed. I probably let it go a little too fast on the highway. Honestly, my neighbor who lent me the car had a default setting of 20% over the speed limit, which I just naively followed. And then when I got back, he's, oh, no. I I turn that thing up and down all the time. It just depends on where we're going. So I was going 84 in a 70 at night in the rain on the way home, which as I look back, I was like, maybe that wasn't so smart. But I was definitely paying close attention. And overall, it did really well. Certainly, at those high speeds, I thought it was, like, amazingly good when it was the the only kind of areas where I found that it messed up a bit was in ambiguous situations. They're like, where's there's a stop sign that could be interpreted as facing 1 of 2 ways. And I've personally been a little bit confused by some of those situations too. That was 1 where it was probably the most, like, risky thing that it got wrong was stopping on the off ramp of a highway where it it definitely should not have. But overall, I thought that was really good. I think you've used it more recently than I have. The hands are now free. What else is kind of new in that experience? And is it still 2 at that? Would you still put that in? I you're not a

Timothy B. Lee: (9:40) lover of the levels. This is why I emphasize liability. I think the fundamental question is if a car crashes, you get to sue the company, or is it whoever you hit sue you? Because ultimately, there's a technological design decision of like how strict is it gonna be about enforcing the requirement to do it. But even if the car is very advanced in the Tesla system, they're still gonna blame you. You crash the car, they're gonna say you should have been paying attention. And so that's the distinction I I would draw there. So my experience was back in March. I spent about 45 minutes behind the wheel in San Francisco. And so I was not testing the strictness of the driver monitoring because San Francisco is a pretty hair raising place to drive. I was happy to do it in San Francisco both because it gives it a good challenge for the vehicle and also because that's also where I'd taken some Waymo tests. Right? So it's little bit of an apples to apples comparison. So I did not try taking my hands off the wheel or taking my eyes off the road because I was, like, wanted to make sure I didn't crash the the vehicle I was borrowing. So don't know about the driver monitoring, but my experience I had, like I said, I did it for about 45 minutes. There were 2 times when I, took over during that 45 minute period. There was 1 time I was making a turn onto a road that had a bike lane with some plastic separators, and it was about to hit 1 of those plastic bollards or whatever you call it. And the other time it got in the wrong lane and then was trying to cut over into a lane that was, like, full. And so there are cars behind me honking because I was like so that wasn't, like, technically a safety issue, but it was definitely a mistake and something that I was annoying people on the road. Yeah. So that was my most recent experience with it.

Nathan Labenz: (11:06) And it was this do you know what version that was? We're now

Timothy B. Lee: (11:09) on 12 5. Okay. Yeah.

Nathan Labenz: (11:11) So that was this was after the big streamlining to neural networks, before the the very latest.

Timothy B. Lee: (11:17) Exactly.

Nathan Labenz: (11:18) Okay. On the Waymo side, I think what people most envision when they dream about a self driving car future, and it it is amazing that it's here today. It is in a limited geographical area. Like, I I wanted to take it to the Oakland Airport as I was leaving town. Can't do that. Can't cross the bridge. Can't get out of the San Francisco city limits. Yeah. But it was within the city really an amazing experience. I mean, you use the app. The app is basically an Uber like app. I have some questions about what this may mean for the future of Uber as well. Yeah. Some in the car, pretty quick response time. They seem to have liquidity in that San Francisco service area. It pulls up. It pulls over. I think it shows your name on the top of the thing, which is cool. Honestly, probably the hardest part of that experience was figuring out, at as a first timer, that I needed to unlock the door in the app before I could get into the car. That was a maybe a little UX refinement still to be done there. But then once you're in the car, it's wow. This thing there's nobody in the driver's seat. There's just a sign there that says don't mess with the controls. And you ride along, and it takes you where you wanna go. And I think if there was anything that was most striking about it, it was how ease or how quickly I adjusted to it and started to become bored. Like, I'm somebody who's very enthused about this technology. I was like, this is a a privilege to be in here, but then we'd stop at a red light. I'd start to look at my phone. And next thing we're going again, I'm still looking at my phone. And I'm

Timothy B. Lee: (12:45) like Yeah.

Nathan Labenz: (12:46) Wait a second. Get get your eyes off the phone. Like, actually pay attention to what's happening. It it was just amazingly comfortable, normal. And in my not super high experience, but probably an hour over a couple trips, about as close to flawless as I could say. I would say it was a more comfortable ride than an average Uber. Maybe not quite at the level of skill of the very most skilled drivers who might have made that extra light or whatever. But also, I'm that's not exactly what they're trying to get. They're not emphasizing making the light. So I was super impressed. I know you're super impressed with with Waymo too. What what would you add to that?

Timothy B. Lee: (13:22) Yeah. So I've I've done about 3 hours of driving over a couple trips to San Francisco this year. And similar experience side over those 9 rides. Never saw it make it even like a minor mistake. Definitely nothing no safety issues, but also just the ride was very smooth. It never got confused or got in the wrong lane or anything like that. And, yeah, like you, it was first couple of rides I was paying close attention. I think I took a video of the first 1. But by the fifth or sixth ride, was like reading a book or looking at my phone. It's just it's it's just when they're that consistently good, it's just not interesting. Like nothing nothing interesting is happening. So yeah, I'm like very impressed overall with the the quality of the driving.

Nathan Labenz: (14:01) So we've got these kind of 2 apparent leaders, and they have quite different strategies in a number of ways. With the highest level, it's like Tesla is a car that you own. It is available, quote, unquote, everywhere. You can refine that, but it's it's nationwide anyway. Mhmm. And you still have to be engaged, and there are some mistakes that we've each seen that we've had to catch. Although, I would never say I felt, like, radically unsafe. Like, I definitely did have to do stuff. Mhmm. On the Waymo side, it's a robotaxi. It is very narrow footprint, but it seems to be more polished at least within its kind of designated service area. So Tesla has to close the gap on whatever that last, probably even less than a tenth of a percent now, and Waymo has to expand its footprint to cover everybody if they really wanna, you know, bring in the dream of the self driving future. Yeah. What makes this hard? Is it the same thing for people obviously have a sense that, wow, there's a big world. There's a lot of random things can happen. Is it the same kind of hard for each of them still, or is it a sort of different fundamental challenge that they each have to solve to really get over that hump at this point?

Timothy B. Lee: (15:15) Maybe this shows my bias, but I think of it being sort of different stages. So I think some like history is just weird. So I've been covering this industry since 02/1637. And back then, Wimal was just this Google self driving car, project. And they've had to file annual reports about their disengagements with the California regulators. And in 2017, the February 2017, they reported that they that during 2016, their vehicles, their safety drivers had to take over once every 5,000 miles. Now there's probably some cherry picking. They would go back and look at them and see if this would really cause a safety issue. You can, like, question that. But way back then, they were, I think, in a similar position to where Tesla is now, where the cars pretty much always work. There was occasional kind of weird edge cases, and they really felt like they were ready to do the driverless thing. And they put in orders for 60,000 vehicles in 2018 from, yeah, Chrysler and from Jaguar. They thought think they like sincerely thought that they were on the verge of having this like nationwide taxi service. And then it turned out to be much harder than they thought. And the service they launched in 2018 ended up having safety drivers about 2 years before they went driverless. And then it was very small scale in 2020 and 2021. And it was only a year or 2 ago that they really started to ramp up in scale. And it's not totally clear to me why, but I think it's on some level, it's just about their the edge cases. Some of them were really hard. And the 1 that I've been thinking about the most recently, and I think you've seen a lot of news reporting on in San Francisco last year, is emergency scenes, either a car crash or a fire scene where the stuff that happens there is just very different from the typical situation. So in a normal situation, there are lanes, the car has to stay in the lane, it rolls about you turn here and you don't turn here, you stop at this light and so forth. When you get to to a car crash, a police officer will come and they'll start drafting traffic in a totally different way. And every car behaves differently, and so that you have to reason about for this car to get go here, I have to stop or I have to back up or this police officer wants there's just all these it's just a totally different problem. And that might only happen once every 10,000, 100,000 miles. But if you have a and if you have a safety driver, it's not a big deal because a safety driver can take over if the car gets confused. But if you have a driverless vehicle, the driverless vehicle has to handle that somehow. And I think there's just enough of those situations that if you have a driverless vehicle that only does 99.9% of the time, it's still getting stuck all the time or maybe even running at people sometimes. And, that's just not good enough. And so the last 5 years has been Waymo going from 99.9% to what they are now, which I don't it's a little hard to tell exactly what the kind of number is now, but now they handle enough of those edge cases that it's seems to be a kind of viable service or close to it. And I think Tesla is just at the start of that state that process. My guess is that what if a Tesla comes up to a seat of a car crash, it'll just signal the it it'll either get stuck or it'll signal the driver to take over. I've I've never tried it myself, but I would assume it's not ready to do that the way a Weibo does pretty well.

*

_📢 Sponsor Message_ (18:10)

Hey. We'll continue our interview in a moment after a word from our sponsors. I don't have any direct info on what a Tesla does in that situation either. I did ask my neighbor who drives his a lot or has it drive him a lot. And he said, yeah. It doesn't really take sort of unconventional moves to handle strange situations. 1 I specifically asked about was, like, if somebody is coming up on you and appears like they might be about to rear end you, would the car move forward to give the the car that's coming in hot more space? Mhmm. And he was like, no. I don't think so. It's probably not gonna be that creative in a situation like that. Yeah. So I guess there's is there is this 1 thing or is it 2 things? The my instinct as somebody who loves to test AI products in all different formats around the hand signals thing would be like, I sort of bet that, like, the latest Gemini model, then we're talking on the first full day when Gemini tops the LMSYS leaderboard with their latest August 1 release. And it can handle multimodal. You can put video into it. My guess is that it would, like, out of the box, be, like, pretty able to interpret a scene like that and say, what does the offer officer appear to be signaling in this situation? Obviously, you have to be, like, more reliable than probably Gemini is gonna be. But is it just ultimately just the volume of these unfamiliar? I feel my I guess my gut is saying that still feels, like, common enough that it doesn't seem like the kind of thing that should be that far. You know, if it can do all these things that it can already do, like, why won't it be able to learn the the hand signals? Or are the hand signals really that, you know, never seen before that you couldn't expect similar performance there?

\>

*

Timothy B. Lee: (20:03) I think there's a couple of things. The main thing is just as different from what the main training data is. So the fact that Tesla can do can stay on your lane in the highway just doesn't tell you much about whether it's gonna do those other things. It's pretty rare. So it's rare enough that you're gonna need a lot of miles before you have a training data. If if you're gonna do a kind of learned approach, you're gonna need a lot of training data. Now you could take a car on a test track and have somebody pretend to be a police officer, but it's like hard to think to do it realistically. Like a real crash scene has like dozens and dozens of cars in kind of random configurations. It would be expensive and hard to do that in kind of a fake situation. But then I also think like the kind of reasoning is pretty different. Like the when you're on a regular road, there's like a rule you can learn that says you always stay in this lane. Whereas in a crash situation, you really have to reason counterfactual. You have to anticipate what are the other drivers gonna do to a much greater extent. Like this vehicle is trying to get out of this lane, and so I better not get in the rain because then we'll have it. So I'm definitely not saying it's impossible for AI systems to do it. I mean, I think it seems like Waymo is getting there and I'm sure Tesla will get there eventually. Oh, and 1 of the things I think that with Tesla's current approach, I think that most Tesla owners instincts will be to switch off FSD when as they're approaching it, which means you're not necessarily going to get good training data. Like ideally what Tesla would like is for the human driver to allow the FSD to try it until the point where it actually makes a mistake and then take over because then you have data on this is, you know, how it works in a real situation. But if the human driver is like worried about annoying the cop or whatever and just turns it off before it gets there, then you don't have as much you you don't have good training data. So, yeah, I'm not saying it's impossible. It's just empirically from looking what's happened with Weibo, it seems like a much harder problem than the other. Like, I don't think the people at Weibo and Cruise are idiots. Like, I think they sort of knew this was a problem. I'm sure they made some effort to prepare for it, but it turns out to be just empirically watching how it's gone for them, it seems to be a very difficult thing to deal with.

Nathan Labenz: (21:55) Yeah. That's a good transition to 1 dimension, and I wanna go dimension by dimension here of how these companies have approached this problem, what their strategies have been, the strengths and weaknesses of those. Mhmm. This is a good, entry point to data acquisition and, I guess, like, overall learning strategy. The sort of Tesla bull case, as I understand it, is they've got by far the biggest deployed base. And I understand that they collect data on human driving. So when you say if the person is a little bit nervous or whatever and and just takes over, would that mean that Tesla wouldn't get that data? Because that my naive understanding would just be like, if they can continue to record what the human does, then isn't that sort of, in a sense, like labeled data or not gold standard, but there's some sort of ground truth there of, like, how the human navigated that situation. So am I misunderstanding? Do they not collect as much as I thought they did?

Timothy B. Lee: (22:54) So I don't think we know exactly how they do it, but I don't necessarily think I I think it's quite possible they are able to collect the data as the human being is driving. I mean, I think it's just less useful. You can try to go back in simulation and simulate what our vehicle have done in this case. But because it's a interactive kind of thing, like, it's not often obvious in the moment. The question is not did our car hit the other car? It's would this car's decision have led to a situation 3 seconds later where everybody's jammed up because the car went the wrong way and got in somebody else's way, stuff like that. That data is I think that training data is definitely useful, but it's not, I think, as useful as if you had a professional safety driver who was a little more aggressive about letting the vehicle do its thing until it got into a mistake. But Absolutely. So Tesla does have the ability to gather large amounts of data from its vehicles, from its fleet. And as I understand it, it has some pretty sophisticated tools where it can say, okay. We're trying to train on cars with bicycles strapped to the back, and they can send some kind of query out where it says, we're looking for this kind of image, and then the car will run some sort of local search through its kind of database and send back candidate images. And so they have a very good ability. I do think this is 1 of the advantages Tesla have. They have a good ability to both collect a lot of data and also focus a little bit on here's the kind of data that we're looking for. Yeah. I think that is plausibly 1 of the advantages they have over other companies that are trying to do this.

Nathan Labenz: (24:13) They would get data from all cars or all cars that haven't opted out. You don't need to be an FSD customer to get to be, like, relaying back camera data to load.

Timothy B. Lee: (24:24) Closely, but, yeah, I don't think it's limited FSD. I think anybody I think by default, probably, like, when you when your car is close to a Wi Fi, at point, the Tesla can suck down images or telemetry from recent trips you've taken.

Nathan Labenz: (24:36) Yeah. I would assume the Elon terms of service are pretty, friendly in terms of sharing data back with the company.

Timothy B. Lee: (24:43) Yeah.

Nathan Labenz: (24:44) Okay. So there's like a would seem to be a big data advantage, but I guess to the degree or the degree to which that advantage matters probably depends a lot on, like, other aspects of the technology stack. So when I'm hearing your analysis of would the car have done something which 3 seconds later would create a challenge, that opens up a number of questions for me. Guess the first 1 is, what is the cycle time at which these things are running and making adjustments? Because if it is multi second, then it's yeah. You you're doing a a significant amount of modeling and predicting what other people are gonna do in response. It seems to get complicated. I could also imagine a stack where the sort of next move you know, the next token time, if you will yeah. I'm used to language models. Right? So these things are measured in tokens per second. And now we're often getting into hundreds of tokens a second. If you could run an inference cycle at that same frequency, then it would seem like you could maybe be a little less, like, worried about the butterfly effect of if I did something slightly different, what would happen downstream? And more just focus on imitation learning of, can I learn to do exactly what the human is doing on, like, a hundredth of a second or a tenth of a second sort of basis? So I guess first question there is you can maybe just take zoom out if you want and take this from the beginning. But frequency is what I'm stuck at the moment.

Timothy B. Lee: (26:08) I I think the so let me, like, walk through the the standard architecture. So there's I think it has 3 layers. So there's perception layer, which is the layer that takes the sensor data and builds a world model. It says, here's all the objects that I can see with my sensors, their locations, their velocities, the types of objects they are. So that's the first layer. Then there's a prediction layer, which has given all this data I have, what is the world going to look like 1, 2, 3 seconds ahead? And that's important, obviously, because if there's a car coming from the opposite way at an intersection and it's not currently in front of you, but it's a certain velocity and it's going to be in front of you, you need to be like playing that out. Then there is planning where given this set of current states and future states, what's the path that gets me where I want to go without causing any collisions, etcetera. And so I think that just a very naive, like short term imitation learning approach, I think is not really going to work because what the human is doing is the human has a prediction kind of engine in their head. You're thinking about in 3 seconds, this car is going be in this location. And so if I'm also in their location, there'll be a crash. And so in the same way that like language models, when you train a large language model, it builds, it learns higher level concepts that then allow it to predict the next token in a way that's maybe not explicitly planning what the future tokens are, but it's able to do higher level reasoning. If you had a model that was able to learn the implicit model that humans have, I think that could work in theory. But I think what and actually, I think that it's evolving in that direction. So 5 to 10 years ago, it really was true that, like, Waymo and Cruise stacks were a bunch of pretty hand coded rules where if you're in this kind of situation, do this, that kind of thing. And over time, I think they have replaced more and more of that with neural networks that learn, but I think it's still probably a single digit number of there's probably 1 or a few perception networks. There's probably 1 or a few planning networks. So they published papers about a few of these. So yeah, so I think the trend has been towards more learn neural network, deep learning kind of things, but they're not yet to the point where there's just 1 big neural network that just has sensor data and pedal and steering wheel positions out. And I'm not sure that they're going to get there or that it would make sense to get there because you do want to have you want a much higher level of reliability than you see with the work language model. And I think there's a lot you can do. You at least want to have visibility into the what he calls implicit knowledge, of the world so you can verify, yes, it's seeing the things that really are there and yes, it's making reasonable predictions. Like you could I think some of the training you can do, you can train the perception and planning and predictions layers separately because you have ground truth there. You say, I actually know where this car was and so I can see it was my prediction to where the car go accurate. So, yeah, I think that's the direction it's headed. It's gonna be a small number of networks, but not 1. Tesla has talked about having an end to end network, and I don't think they've been super transparent about exactly what that looks like. I'm a little skeptical, but it's literally just 1 end to end network that takes sensitive data in and and outputs like a driving direction, but I'm not sure exactly.

*

_📢 Sponsor Message_ (29:04)

Hey. We'll continue our interview in a moment after a word from our sponsors. I think there is at least a they display for you in both of these examples. They display for you on the dash a world model. So you you can see that there is something else happening besides just straight to actuation on the controls. Mhmm. Those are both striking. I that's, like, an incredible thing to watch unto itself. Honestly, if you have never sat in 1 of these cars, even if you just forgot about the fact that it can drive itself for a second and you just looked at how well it is understanding the scene around you and labeling those things and knowing what's a person and what's a vehicle and that in of itself is just, like, pretty impressive. We just did an episode not long ago with 2 authors, Nora and Ben, of, a paper called Guaranteed Safe AI, which is a proposal for a framework that seems like it's almost, you know, perfect for self driving. It first, it kind of has 4 components, but 3 are added on to the main AI. So their idea is maybe you have a black box AI. Maybe you train this thing to be a 100% end to end. We wanna be able to audit this. You know, we wanna be able to have a a much better sense of what's going on than we can get by just brute force poking at it with situational testing. So how do we do that? The 3 components that they add on are a world model. So I it seems pretty clear that these cars have some version of that. Yep. Then they advocate for a an explicit safety specification. I don't know if the makers have anything like this today, but they're envisioning things there like acceleration should never exceed whatever, or we should never get closer to another object than whatever. And to make that clear, 1 of the benefits that they see on this is that it enables a societal conversation about what the safety standard should be. And then the third piece is a verifier, which is the thing that takes the output of the core black box AI and simulates what will happen if we do that in the according to the world model, and does that still, you know, fall within the safety standards. It seems like that's what's going on here. Does that sound like the architecture that these companies already have?

\>

*

Timothy B. Lee: (31:24) Yeah. At a high level, that's what they're trying to do. I would I assume that Waymo has internally, either explicitly or implicitly, rules like that. Don't exceed this acceleration. Don't get too any closer than x to a car. 1 of the things I think is difficult so Mobileye has a a standard called the responsible sensitive safety standard that tries to do this to say, here's our, like, formal specification of how we think a self driving car should behave. And I think that's perfectly nice idea. I think that 1 of the things that makes it really difficult is that you're not just trying to optimize your safety. You're also trying to optimize for getting where you're going and not irritating people. So 1 way you can make a vehicle safe is have it be very conservative. And anytime it's not sure what's happening, it slows down and stops. And at least if you're not on a freeway, that's always safe in the sense that nobody's gonna die from that. But if you have a vehicle that's super overcautious, it's gonna cause traffic jams and piss people off. And also if you have a fire engine coming, like, then you could indirectly cause safety problems. And so I think part of what makes this problem challenging is you want to never get in a crash, but also behave like a normal driver. And when it is safe to go, you want to go. And I think that is much that trade off, I think it would be much harder to capture in a simple some kind of standard. It's just there's lots and lots of different iterations. And ultimately, think you just need a ton of examples, ton of training data, and that's something you need to learn. And, yeah, I'm all for more transparency, but I think it's like a a hard problem to formalize it.

Nathan Labenz: (32:45) Yeah. I think the way that they would respond to that and I can't, speak authoritatively on this framework, but I think the idea there would be that the core AI that you've trained that this sort of guaranteed safe framework sits around can still be responsible for the sort of human driver experience and the not annoying people and all the fine points. And it's okay if those remain opaque in terms of exactly how they're working. It could be a little bit more permissive than and in fact, would almost by definition, you need it to be a little bit more permissive than what the system actually does. But you could hopefully get it down to a finite number of and hopefully reasonably understandable number of rules of the thou shalt nots of this problem, which obeying all the thou shalt nots is not sufficient to have a great user experience,

Timothy B. Lee: (33:40) but it

Nathan Labenz: (33:41) would hopefully at least be sufficient to say we can make some quantitative guarantees around, like, very bad things not happening.

Timothy B. Lee: (33:48) Mhmm. Yeah. My guess is that, like, Waymo would pass that fairly easy. And the other issue is that sometimes you have planning mistakes, sometimes you have perception mistakes where you think that the child is a garbage can or something. And I don't think there's any way you can formally prove that a car won't miss misunderstand its models. Yeah. I think you could probably apply a model like that to the prediction and planning parts or at least the planning part and say, given a particular world state, we can prove that it won't make a mistake relative to the specification, but that's only gonna capture a slice of the possible errors it could make. But, yeah, I I would be happy to see companies companies do something like that as part of the safety case they make to the public.

Nathan Labenz: (34:25) So going back to the frequency thing and the sort of planning, this maybe is just not disclosed, but I guess my experience in, I would say both of these cars, but the Waymo 1 is is most fresh in my mind, was that it did feel like the response time was human like. It felt like it was responding as fast as I would respond. I actually don't know what my own response time is. Maybe you have a stat on that would be, like, the human point of comparison. But do we know how far these things are planning and predicting out? Like, how far does their world model extrapolation go? And and do do we have any sense for how often they are updating their outputs to gas break steering?

Timothy B. Lee: (35:14) I don't think they've disclosed anything like that, and I bet it's complicated. I remember maybe a year ago you had a drone maker on where they talked about there being different kind of levels of abstraction, where there's a, kind of high level, like, navigational thing that maybe updates every second or something. And then there's like a real time make sure that drone stays stable thing that's operating at, like, a millisecond level. My guess is it would be a similar thing where there's some systems, the system that kinda figures out what's our goal direction that's probably pretty slow, but then somebody runs into the car, that loop that's went slam on the brakes, maybe it's very fast. But, yeah, I don't think they publish any specific data on which things happen at which frequencies.

Nathan Labenz: (35:50) Yeah. That was a I deep in the catalog there. I appreciate that. That was July, Haik Martiras from Skydio. And I learned a lot from that episode just in terms of how these sort of different frequency and different level of abstraction control loops interact with each other. I think they had at least 4, maybe 5. The very lowest level 1 was, I think, like, tens of thousands of hertz, and that was just at the level of the voltage being applied to the the motor could change that fast. And then, yeah, at the outer layer, it was like language model instructions of, like, planning how am I gonna go survey this bridge or this this building or what have you. And connecting all those things where each level, you know, interfaces with the 1 above and below it was a huge part of what they had invested. And they're really just adding on the last layer now. Like, they built interestingly for them, I think the and this may be quite different from what I'm gathering from the self drivers, but pretty much all of the lower level stuff was either explicit code. Like, they've done a lot in explicit code or, like, small very small neural networks.

Timothy B. Lee: (37:01) And Mhmm.

Nathan Labenz: (37:02) That was all there. And now with this new generation of AI, they're able to layer on a 1 more layer at the top where the user could give, like, high level instructions. But a lot of that stuff below was, like, older technology and had been built out, largely in explicit fashion over a long time. Yeah. So interestingly, a great callback. Okay. So we don't know ultimately how fast they can respond. I guess I can only say it felt like I wasn't noticing anything, you know, faster or I wasn't feeling any lag relative to what I would have done. But it sounds like a pretty qualitative statement. But maybe let's do a deeper unpacking of some of these technology stack decisions that folks have made. We've covered the 2 form factors that you can get the car and you can buy it or you can go on an app with Waymo. 1 other 1 that is is maybe worth considering or maybe you think it's a red herring is the aftermarket kit, like the retrofit style, whether that's Kama perhaps or some other thing. Do you see that being a a meaningful form factor that we should also be thinking about?

Timothy B. Lee: (38:10) I think Kama is a very interesting company. They have a very active open source community and they've managed to build add support for a ton of cars. So lots and lots of, if you have a reasonably new car, there's a good chance you can apply Comma to it. And Comma is they sell a device that's just like a little smartphone, I think, basically with a camera on it. And you plug it into there's a every car has a port that gives you access to the car's onboard network and you plug it in and it gives you kind of Tesla autopilot style level 2 self driving. I don't expect that to be like a big market because for 1 thing it's been a regulatory gray zone, comma, I think the fact that the user downloads the software separately and installs it themselves, I think may give them a little bit of they've had some friction with NetSet, the federal regulator about this in the past. So partly for a good regulatory reason, but also just because I don't think most people wanna screw around with open source software. So I could imagine if I think the bull case for Kama is maybe at some point it would license this technology to car makers to build Kama on. But I think there's lots of good reasons to have that be vertically integrated, ultimately have your car manufacturer pick a good, system and put it on there for you.

Nathan Labenz: (39:12) Yeah. It definitely takes the notion of AI hobbyists to the next level when you're strapping it onto your Chrysler minivan that's up from a year's old.

Timothy B. Lee: (39:20) The other thing is I I think the the carmaker can integrate the sensors much better. So it's impressive that Comma can do what it does with basically 1 smartphone camera, like, mounted, like, on your windshield. But in the long run, you're gonna have I think Tesla has 8 or 12 or something cameras in there. I think in the long run, that's what you're gonna want. And you might as well have software that's integrated with that specific sensor suite as opposed to having some aftermarket thing that's not gonna work as well with the specific characteristics of your vehicle.

Nathan Labenz: (39:45) Yeah. It's amazing that they can do much of anything with just 1 point of view. Does that does Comma also suppose or require that the car itself has additional No. I think It's really just using

Timothy B. Lee: (39:57) the camera. There might be a rare sensor, but, yeah, it's, like, much fewer sensors than, a Tesla has. And, yeah, it's I think it's just it's just based on the the camera that's in the little device you put on your windshield. So it's it's pretty impressive.

Nathan Labenz: (40:07) Yeah. Cool. Okay. Then let's talk hardware maybe for the other leading strategies. Tesla is famous for an all camera approach. I don't know if they they're still strictly on that. 1 look at the Waymo and you can tell that this is doing something a little bit differently. So break down, for us how people are sensing the world.

Timothy B. Lee: (40:28) Yeah. So the 3 major centers sensors people talk about are camera, lidar, and radar. Tesla's used to have radar, and I believe about a year or 2 ago, Tesla took away the radar. So they're, like, really committed to this camera only approach, whereas Waymo has all 3. Waymo has a lidar sensor on the top that's a big long range, and then it has, I forget the exact number, maybe 4 or 6 LiDARs around the edges of the vehicle that are short, shorter range, wider angle. And I think that's partly for if a pet or a kid were right by the vehicle, like, wants to know that. And then it has a bunch of cameras. And I think I I don't know how many several radars. So they have 20 or 30 sensors on the vehicle, whereas Tesla, I think, again, think it's 8 or 12, something like that cameras and older vehicles had 1 radar. And these sensor modalities, I think, have different strengths and weaknesses. Like 1 thing radar does that's good is it can tell velocity is in addition to like, directly measure velocity in addition to to kind of position, which can be useful on a freeway. There are some LiDARs that do that, but most of the LiDARs people are using do not have that kind of velocity sensing ability. LiDAR has the advantage that it is high enough resolution that it can get a pretty good 3 d structure for things, and it can directly sense. The LiDAR bounces a lidar laser off the object and then measures the time to return, which means it can do an exact distance measurement. Whereas camera, of course, just gives you an image, and then you have to do post processing to figure out what's the 3 d structure of this 2 d structure of thing you're doing. And so Ladder gives you a 3 d point cloud that is different. And anytime you're doing anytime you're trying to understand the world, it's useful to have diverse types of datasets because they tell you different things and they're gonna have different phase failure modes. So, yeah, that's why why Waymo does it that way.

Nathan Labenz: (42:03) So I think I've seen online analysis that I I think you may disagree with that basically says the camera approach is the winner. Like, cameras are cheap. They're commodities. The LiDAR is, like, more expensive. It takes more maintenance. But my sense is you don't think that's actually as big of a deal as commonly argued.

Timothy B. Lee: (42:26) It's definitely true that LiDARs cost money and take work to install and and maintain. If it were the case that you could get very high quality with cameras, that would definitely be preferable. I guess I think it's just important to think about it for in an engineering terms, the goal here is to make something very safe, and nobody has managed to make something. We'll see. At least we don't have clear evidence yet that anybody has managed to make something that's like very clearly safe. And so if you're a company deciding there's a kind of safety versus cost trade off and you can pick different points along that, and Waymo has chosen to go with the higher cost, higher safety realm. There are some people who claim it's somehow like a disadvantage to have LiDAR because the camera only for some reason, like cameras let you use deep learning more or something like that. And I think that's just a misunderstanding of the technology. There's no reason you can't have a neural network that uses LiDAR as well as cameras. The big thing that I think the the big advantage LiDAR has, as I was saying before, is it can do direct distance measurements. So you can create a neural network that has takes multiple images and do parallax effect, compare kind of angles of things between different images and do a pretty good estimate of how far things away are. And so you can generate a point cloud that kinda looks like a LiDAR pug cloud. The the difficulty with that is that if that algorithm misunderstands what it's seeing, it thinks something is an object that isn't or misses the existing object, then it will get a wrong value for that. And the problem with that is that is, I think, can be correlated with other failure modes of vision. And so in the average case, it's gonna work almost as well. But what you really care about is these edge cases where something weird happens, there's an object your stack hasn't seen before. And LiDAR will always tell you there is something so many meters ahead that's a size. And if I were the CEO of a self driving company, I'd be really nervous about doing the camera thing because it might work 99% of the time. And then the 1% when it fails, then my company gets a $10,000,000 lawsuit. I think when this approach works, it makes more sense, but it is an empirical question. If Tesla continues improving rapidly, maybe in 2 or 3 years, it'll be showed that like you can do it with cameras. Like I'm not like super dogmatic about this, but they aren't there yet. So I'm not I don't think that's that they've kind of made their case yet.

Nathan Labenz: (44:30) So, does this it seems like this may tie to a couple other aspects of the overall strategy that these companies are taking. Would it be fair to say that the LiDAR strategy is a balancing force just that might serve to even out the data scale advantage that Tesla has. I guess what I'm thinking there is, like, Tesla's got a lot more cars on the road. It's collecting presumably a lot more episodes or whatever. But if those inputs are fundamentally noisier, then maybe you just need a lot more to ultimately learn what you need to learn. Whereas if you have much more precise measurements of exactly what the structure of something is or exactly how far away it is or exactly how, you know, fast it's moving, which sounds like the the way most getting all of those all the time, then maybe if that's just a lot more accurate, then you have a less obviously, everybody still needs a lot of data. I could see that if it's, like, that much more precise. Could see that, hey. Maybe you could get away with an order of magnitude or maybe even 2 orders of magnitude less data. Does that seem like a reasonable way to think about those trade offs?

Timothy B. Lee: (45:39) I think I think that's probably true that Waymo has less data, but it's probably higher quality both because they have letter. And they also because they have professional safety divers who are like annotating, like why they disengaged. They get a clearer signal for the reinforcement learning kind of thing where you have a clearer signal about this was a mistake out of the car versus this was the time when the diver just needed a bathroom break, that kind of thing. So, yeah, I would say in general, Waymo has less data, but I would expect it to be higher quality. But I think the main thing that's going on is just that Tesla's business model does not allow them to put a 10000 hour lighter in every car. And so they have to do the thing they're doing and then they're doing their best to make that work from a technology perspective and maybe it will work or maybe it won't. But Waymo's strategy is they're vertically integrated and and they have a fairly high margin business like tax. Like, if they can get the I guess we'll talk about the economics later, but if they can get it working, it's as potentially pretty profitable. And so they can just afford to buy and they've only got a few 100 cars on the road so far. So it's not that expensive to put pretty expensive LiDAR on every vehicle, and the bet is that over time, it'll get cheaper. And so then it won't be a big deal once they're at scale.

Nathan Labenz: (46:40) Yeah. So why going back to the Tesla can't do it, if I understand that well, you just said correctly, the case there is, like, at scale, a $10,000 LiDAR maybe becomes a $1,000 LiDAR. Maybe that could work for Tesla. They're already not the cheapest cars on

Timothy B. Lee: (46:56) the road. No. Absolutely. But like when in 2016, when Tesla started selling what they characterized as full self driving ready thing, back then the lighter costs, I don't know, $50,000 or something. Then also they don't they put the cameras on every vehicle, but only 10 or 20 or 30 or something percent of people sign up for it. And so that business model, can't it just would be way would have been way too expensive to do it back then. At some point, yeah.

Nathan Labenz: (47:20) Can't sell that over the year.

Timothy B. Lee: (47:22) Yeah. Once LiDAR scales up and gets cheap and if it's 1000 dollar sensor, then yeah, maybe they will start putting Elon's pretty dug in, so I don't think he would wanna admit that he was wrong. But yeah, if if at at some point, LiDAR will get pretty cheap and I I expect actually, we're starting to see I think it's happened yet, but we're starting to see OEMs planning to put LiDAR other car OEMs planning to put LiDAR on their vehicles. I think you'll start seeing LiDAR be a standard kind of component like radars in a lot of cars and eventually I would expect Tesla to add it. But yes, so far I don't think it's gotten cheap enough that it would make sense. There's also different level of kind of quality of LiDAR. So the the big spinning LiDAR that's on the top of it, well, that's still thousands of dollars. There are, I think, some other ones that are hundreds of dollars that have less field of view, less range, etcetera. And so those, I think, probably could put in a vehicle, but probably are not powerful enough to add the kind of value that the big LiDAR and the Waymo adds.

Nathan Labenz: (48:12) Are they hard to maintain? I've also heard that this is an issue where, like, the Waymos are going into the shop, you know, regularly for LiDAR or maintenance, even if it's all good otherwise? How how big of an issue do think that is?

Timothy B. Lee: (48:25) That I don't know. Wemo makes their own LiDAR, and so it's pretty secretive. There's different in the broader market, there's different models. There's some that are more solid state than others. So, yeah, I don't know the details. But I I think it's like anything else, like, it's probably somewhat hard to maintain now. But as I improve the technology, it shouldn't be a an issue with kinda scale.

Nathan Labenz: (48:43) It's interesting how secretive this stuff is. It feels like this would be the sort of business where it's hard to copy even if you give some details of how you're doing it. So Mhmm. Given that and given the critical role of trust in a system like this, am I off base for thinking that maybe they should be a little bit more transparent about what is actually happening?

Timothy B. Lee: (49:08) As a reporter, I would love that. I think they've been reasonably transparent by the standards of like technology companies. Google is not telling people like what's happening in their data centers. They have published a number of technical papers, machine learning papers about the various algorithms that users won a couple of years ago where they're using transformers to predict the behavior of vehicles on the road, stuff like that. And then they do have a safety report they put out or they put several safety reports with crash data and stuff like that. So I think they've published a lot of the information that you want to evaluate safety. I think they've been an okay citizen on that. I would certainly like them to be more transparent about it in a number of areas, but I don't think they've been, like, notably secretive. To Tesla, it's a similar story. They have they've had a couple of AI days or, self driving days where they've provided a lot of detail. They've given talks about how various things work. And so I think we have a kind of high level understanding, but the the technical details are both some proprietive, but also I think they're changing pretty fast. And so if you look at something that somebody said 2 or 3 years ago, it might just not be the the same as what's happening now.

Nathan Labenz: (50:05) Yeah. That's for sure. You you can't go back and read the, GPT 3 paper and expect to be making good inferences about what's going on, with the latest models. So I would assume there's been quite a lot of change. It seems like the big thing has been the overall story in almost all of AI is toward more and more stuff being trained into the neural nets. Like, this is classic Elon. Best part is no part. Relative to, like, early versions, I would guess there's probably been an order of magnitude reduction in just, like, how many sort of components there are in the overall software. Is that your sense too? Do we know what are the sort of explicit things that remain? I guess you could think of that as sort of the safety spec or some sort of governance perhaps or but it could be more than that too.

Timothy B. Lee: (50:57) That's the kind

Nathan Labenz: (50:57) of thing they haven't really shared. For example, like, how explicit do we think that planning still is? I mean, I recall the 1 language model for lane changes or sort of intersection where lanes can get weird. Tesla showed a module where they had essentially used a language model like structure to figure out how how does 3 lanes become 2 as you pass through an intersection and whatnot. Previously, presumably, that would have been done with a bunch of rules. And so I guess I'm just I'm wondering,

Timothy B. Lee: (51:24) do we know what pockets

Nathan Labenz: (51:25) of these hard coded rules remain

Timothy B. Lee: (51:27) Yeah. Today? I don't think so I I definitely think the trend has been towards more learned systems, less hard coded rules. Weibo's approach has been to publish academic papers sporadically and not be super specific about the overall architecture. I don't remember off the top of my head how recently they published something about their planning stack, but they'll say, like, here's a network we're using that does some aspect of it. But I don't know if they've ever published kind of an overall paper that says here's her old overall architecture and all the networks are using and how they fit together. It's more, I when they have something that's interesting and I guess not so sensitive that they don't mind really viewing it, that's and they will publish a paper. Tesla's little different in that they are more presentation oriented. And so there's been, like I said, 2 or 3 or 4 presentations where Elon and a few of his kind of top AI people will give an hour long talk saying showing demos of various things they're doing and explain a little bit how the technology works. But again, I don't think they've ever given like a whole kind of architectural review here to all the networks or all the software modules that make up the system.

Nathan Labenz: (52:23) Tesla once famously open sourced all their patents. Do you know what their status is on that today? Are they just not patenting this stuff, or are they no longer open sourcing all patents?

Timothy B. Lee: (52:36) I've not looked into that. I think that it's normal for companies to patent a bunch of stuff and then mostly not sue each other over it. So that would be my guess, but no. Haven't looked into detail if they're if they're still taking that open approach.

Nathan Labenz: (52:47) Yeah. Interesting. I mean, yeah. These things can sometimes have very weird feedback effects where, you know, I remember my dad used to say that the thing about a patent is you have to tell everybody how you did it. So even if you're protecting that, you're also giving a lot of hints and people can learn a lot from what you've done even if they can't.

Timothy B. Lee: (53:05) You're not doing. You file a of patents. You can patent a

Nathan Labenz: (53:07) lot of Distraction.

Timothy B. Lee: (53:08) Work out and yeah.

Nathan Labenz: (53:09) Yeah. Interesting. What are the roles of maps in these systems? Does that differ across companies? Like how exactly the maps have to be? How often do they have to be?

Timothy B. Lee: (53:21) So this is something that I think Tesla people talk about a lot and I think is the difference is probably a little smaller than people say. So classically, Waymo had a very detailed map of what's called a HD map that shows at a very fine grained level 3 d map of like all the fire hydrants and trees and whatever in an area, whereas Tesla did not. And there's been a lot of kind of back and forth about this where the claim is that Tesla is better because it can do this without it, but it's not clear it actually is doing it better. But I think that so there's a couple of things to say about this. 1 of the, I think criticisms of Waymo's approach is it's very expensive to collect the map data. And I think that is gonna prove overblown because you can automate it. So the first time you enter a city, obviously, you have a car have a car driver on and and collect the data. But then once you have a fleet driving around, you can just use the data that you're collecting. In fact, I know they did that. Actually, I visited Weimos to the way back in March, and they showed me, like, a hard drive in the trunk that's collecting this data and they upload to the mothership every night. And so then the data collection becomes trivial because you're just the cars are driving on the roads anyway and you can just upload the data as it goes by. And then I think you'll be able to the other step is the cleaning and labeling of the data and, like, integrating into the map. I expect that to get more automated over time. And I don't think this is gonna be a big kind of impediment or a big, like, cost factor for Waymo in the long run. And Tesla actually has they they had a tweet a couple of years ago where they forget how they described it, but they, like, talked about how they could use their map data to build a map. Now they weren't saying they were using that for driving, but they are also doing the same thing where they the obvious thing where as their cars drive around, they collect data and then they build that into a a 3 d model of the world that they could use for simulation and for various kinds of testing and stuff. And so I think you'll see some convergence here where and the other thing is like Tesla just obviously uses a map for if you give it an address, it needs to know which roads to take. And so it's following a map too. And it's I think you can see if you look at the visualization, it has some idea of like where the lanes are, even in places that are in the view of its cameras. It's less detailed, but it still has a map that it's relying on. I think it's something where the the approaches will converge over time.

Nathan Labenz: (55:20) This kind of goes back to the how far off road can you go or whatever. Does that mean the cars have to be online to navigate? Like, they can't download the whole map of the whole United States. So do you have a sense for how they are loading maps as they they move around?

Timothy B. Lee: (55:39) I don't know exactly how that works, but at least right now, like, Waymo doesn't operate in the whole United States. I think they'll

Nathan Labenz: (55:44) They probably could download the whole map. Yeah.

Timothy B. Lee: (55:45) Yeah. They could get them. I'm sure they have a big hard drive on there. I think they could probably download all of San Francisco. My guess would be in the long run, there'll be a different card. You'll have a 100 mile radius around your current location or something, and every night you'll download for for that. So, yeah, I don't think I don't think it requires them to be online. I don't know all the details of of how they do those updates and stuff, but, no, I think it they have a local copy of the map.

Nathan Labenz: (56:09) So everything is happening locally. This is another thing I wanted to make sure I had a good read on. There's been some talk. I we did an episode with the now, defunct Ghost, which was experimenting a little bit with online uses of vision language models. They had a an investment from OpenAI, and they were looking at, can we use these vision language models for, like, longer time horizon planning? The sort of reading of the road signs and figuring out, like, which lane I should be in or how to like, where the drive through is at this restaurant or whatever these sort of scenes that are, like, just higher level analysis required. Are any of the live players doing anything like that to your knowledge where they actually do make a sort of remote call for any part of their operation?

Timothy B. Lee: (57:00) Yeah. I'm not sure exactly. They definitely do have a network connection and you can do things like call customer service. I'm pretty sure all the safety critical stuff is done locally. But 1 of the things we haven't talked about yet is the kind of remote monitoring approach because the Waymo cars are quite good, they're not perfect. And so sometimes they do get confused and Waymo has said they never have people remotely actually driving the vehicles. There's nobody with a steering wheel and pedals. But what the vehicle will do is they will ask questions or ask for confirmation. They'll say, I'm think I'm planning to do x. Is that safe? And the person will respond. So I think if a vehicle lost connectivity, it's possible after a certain amount of time, it would slow down and come to a stop because it, like, wants to be able to ask for help. I'm I'm not totally sure how that works. I think there were some incidents with, Cruise where that happened, where they had connectivity problems. And so you ended up with the cars with their flashers on getting in the way. And it wasn't that there was anything locally. It was just that the network was down and there was some stopped out of an abundance of caution. So there's definitely some interaction with the outside world, but I think they try pretty hard to have it be in a safe state where if you lose the network connectivity, it's not dangerous. It's just at worst, they've been they have to slow down and inconvenience the driver.

Nathan Labenz: (58:03) You mentioned cruise there. Are they ever coming back? It seems as far as I could tell, not not at the time that they had their very unfortunate incident, which I think is also directly related to this sort of remote help. Right?

Timothy B. Lee: (58:18) Mhmm.

Nathan Labenz: (58:18) That my sense at that time was that they were on the same level as Waymo where they were both operating and, like, driverless and had some ability to get help. I wonder if you think that is was a misperception, and they were actually way behind. And, you know, it's crazy to me that they had 1 safety incident and shut everything down. Like, it seems like it couldn't have been that much of an entirely, like, a house of cards. They were they didn't have a lot of safe trips too. So So up with Cruise?

Timothy B. Lee: (58:46) A a few things to say about that. So they were pursuing a similar business model to Waymo and were growing at a similar pace. I would say industry insiders all through 2023 were like a little nervous about Cruise relative to Waymo. Like they seem to have more incidents. They weren't like wildly worse than Waymo, I think it seemed pretty clear to me their technology was behind, which I think makes sense because they're a few years younger and just I think they were under a lot of financial pressure because GM has deep pockets but not as deep as Google's. A lot of Ford had recently shut down Argo, which was their version of Cruise a year or 2 earlier than this. And so I think Kyle Lotr, CEO felt a lot of pressure to show that this could be a commercially viable thing. And so he was pushing pretty hard to commercialize and turned out to be turned out he pushed a little too hard. I would not say it was just 1 safety incident. The incident that led to their shutdown was a case where a human driving car hit a woman, her body then bounced in front of the cruise car and the cruise car did not stop in time. So her body ended up under the cruise car. And then the cruise car after coming to a stop, pulled over and dragged her several more feet under the vehicle. And then when this became a new story and regulators asked about it, the claim of the regulators is that they showed the portion of the vehicle where the cruise hits the car, but it's not did not show the part with the dragging. Now they they deny that. They say you guys weren't paying attention, blah blah blah. But I'm inclined to believe that. So there was a previous incident where cruise was accused of blocking an ambulance where a car got stuck and the ambulance wasn't able to move. And I talked to them. They showed me a video of the incident from their kind of internal God's eye view model. And they said, okay. We'll show you this video, but you cannot take a bit. You cannot take like a screen capture of it. And I was like, can you like send me a diagram so I can show you? Absolutely not. So they were like I think they were like in defense attorney node as opposed to like transparency, make sure the public understands what's happening mode. And I think that attitude really irritated I think understandably irritated the regulators. And so I think it was less that it was less that they had 1 mistake and more that there was a pattern of not being as transparent and clear with the public or with officials or with journalists as they could be. And so they shut down their service. They are trying to come back. I believe they have vehicles with safety drivers operating in Houston, Dallas, and Phoenix right now. Okay. So I

S0 (1:01:03) didn't hear that.

S1 (1:01:04) Yeah. They are planning to come back, but that puts them 3 or 4 years behind. Wemo started doing driverless in 2020, and so they're now several years behind Wemo. And so I think they're probably plausibly still the number 2 company, but they're pretty far behind at this point.

S0 (1:01:18) Is there any way for us to get trajectory moving? When you say something is like a couple years behind, you've said that about both Tesla relative to Waymo and also Cruise relative to Waymo. It seems like Mhmm. I could make a case. I wonder if you would find it compelling that, like, maybe they're not really years behind Cruise. Like, they have a PR issue. They need to put some safety drivers in. But does that really mean they're, like, years behind? And and for Tesla, maybe I would just make it you go go back to that data case. Mhmm. If they can collect a 100 x more data or maybe even 1000 x more data, they got millions of cars on the road compared to don't I know how many Waymo's there are, but it's millions. Right?

S1 (1:01:51) That's definitely possible. I'm yeah. I'm not saying they're gonna I'm I'm just saying 2020 was when Waymo had the honest to goodness driverless service on the road. And it's taken them 3 years, 4 years since then to scale that up to still a pretty small scale. I'm sure it's possible to do it faster, but I guess I see that as 1 of the clear milestones is the point where you take the safety driver out of the car is a point where you show a level of confidence in your technology. And right now, Waymo is the only company in the Western world that had that level of confidence in its technology. And, yeah, maybe Tesla will be able to scale it much more quickly once they reach that point, but they haven't reached that point, and Waymo reached that 0.4 years ago.

S0 (1:02:30) Let's run through a couple other live players. I always like to do the the live player rundown. I'm very interested in what's going on in China always. And if we could inspire anything in this conversation, I would love to see us race China for self driving cars as opposed to for weaponized AI or whatever. So what's going on there? Do they have a a company that we should be inspired to not be left behind by?

S1 (1:02:54) So China has several. I think Baidu is 1 of them. I honestly have not read much about those because it's just so hard both with the language barrier and the fact that I can't go to China and the fact that they don't have a free press. And so there's been some rumors that they like suppress stories of crashes. Probably China, I think is the number 2 country on this stuff after The US and they have pretty substantial self driving car deployments. But it's not clear to me how much of those are driverless or what the safety case safety records. Yeah. I can't tell you, more than that. And I don't know of anybody who's doing really good in the West who's doing, like, really good reporting on this because it's I think it's hard to to tell what the apples to apples comparisons are.

S0 (1:03:30) At just a notional level, though, do I understand correctly that it is possible to get, like, a Waymo like experience in China where you summon a car and get into it and there's nobody else in

S1 (1:03:42) it? I think there is. Yes. And I think they may be at a larger scale in some places than The US. So, yes, I think they are they're in the same ballpark as Western companies.

S0 (1:03:51) You wanna do a little rundown just of other notable players in the space? Amazon does a hand in the game, so on and so forth?

S1 (1:03:57) Yeah. So there's 2 other companies that are in the robotaxi game at the kind of with the kind of business model and scale that women cruise are. 1 is Zoox, which is was a startup that Amazon acquired a couple of years ago. The other is Motional, which was created there's a long history, but it was like an Aptive subsidiary and then they brought in Hyundai as a investor. And then I think Aptiv got itself bought out anyway. So now I think it's a Hyundai subsidiary. And they are both at the stage of kinda testing the technology, have not started doing driverless or commercial. And I'm, like, interested in what they're doing, but I have not written about them much because at this point, I I guess, like, I start taking companies seriously when they start having driverless because I think that's now the the cutting edge. So I think they're a little behind, but certainly, Zoox just with Amazon behind them, like, they're gonna have resources as long as Amazon wants to have them. And so I could see them being the fast follower of what most of a success. Like, are going to want alternatives. And so that's who they might look to. There's a startup called Wave, W A Y V E, in The UK that raised, I think, billion dollars a few months ago. And they have an interesting approach and they're doing the kind of end to end approach we've talked about. I believe they actually have a foundation model type thing that has, in addition to driving data, also has some, like, text and images and stuff. And you can talk to the to your self driving car. Apparently, demos people have been impressed by demos, but they're, again, early enough that it's hard to evaluate. I haven't been to The UK and and to try it out, but that's they're certainly somebody worth watching. And their business model is they, I think, are hoping to license to OEMs as an alternative to Tesla if if you've got if you're Ford of Toyota and stuff or somebody and you say, we we don't have our own homegrown version of autopilot, maybe that Wave will be able to supply them with, a competitive version. And then the 2 other companies that I would mention, there's a title called Aurora that actually used to have the same business model that Waymo's talk that Wave is talking about now where they licensed OEMs. But a couple of years ago, pivoted to trucking. And there's a couple other companies that are doing trucking. Think Aurora is probably the most important. And so Aurora says they're preparing to do long haul, like, 18 wheeler trucking routes, I think, between Houston and Dallas in Texas. And they say they're gonna do it by the end of the year. That that's another case where I think I would be nervous if I were the CEO of a trucking company because the failure mode, they're

S0 (1:06:03) so Yeah.

S1 (1:06:05) And so that's not where I would like to stick, but I hope they they managed to to make it work. And then the final 1 is a company called Neuro, which was started by a couple of guys who left the Google self driving car product like 7 or 8 years ago. And, they are making street legal robots. So there's like sidewalk robot companies that are like very small and slow, but these are like smaller than a compact car, like washing machine sized vehicles. They go 20 or 30 miles an hour and I think could go faster that don't have a driver or anything. It's just a delivery robot and it's used for delivering pizza or groceries, things like that. And they've been like on the verge of commercialization for several years. Seems like they've done a bunch of pilot projects. And again, it's not totally so they're 1 of the ones I'm most confused about because it seems like that should be an easier problem. So you don't have to worry about safety as much, at least for people inside the vehicle since there's nobody in the vehicle. I've heard that 1 of the issues they have is finding for for regulatory reasons, there's limits to how fast they can go because they're in a category that's not designed for, like, freeway speeds. And so finding a neighborhood that has a right mix of a grocery store and then some residential streets, and you'll have to go through any, like, high speeds. I've been told that's 1 of the issues they're having, but that's another company that's working on self driving, but not building a taxi service.

S0 (1:07:16) Yeah. I think that's a great little glimpse of the future. It's just that the form factor of the car could get out or the of the vehicle could no longer a car at that point, could start to look really different if you don't have to have a driver sitting there and made the normal posture. You can imagine people sleeping in cars or whatever, but then also you can just imagine small things that just don't take people at all. I think that's possible Cambrian explosion of very different looking things on the road is like a a cool future to imagine.

S1 (1:07:46) Yeah. 100%.

S0 (1:07:47) So let's talk about these safety statistics. Like, where are we on this? It sounds like we probably only are far enough along on a couple of companies to even have a sense. But my kind of qualitative sense has been that the companies seem to be reporting that their products in here, mean, Waymo, seem to be reporting that their products are safer than human drivers seemingly substantially. And then my mental model of this has always been like, I'm not sure I should take that fully at face value, so I'll discount it. But even if I discount it, I seem to I feel like I get back to basically, like, similarly safe to human drivers if I just sort of, you know, discount whatever percent. And then if I ask myself, okay. If they're already similarly safe to human drivers, like, why don't we adopt them more? I assume that it's just a quirk of society that we are gonna insist that the AI powered technologies are not just on par, but are actually, like, a lot safer, maybe, like, order of magnitude safer than human before we would actually all agree that this is something we should go forward with. Mhmm. How would you complicate that story? Is that a good summary? Maybe you can get more specific in terms of the metrics or how how we should be thinking about these. Obviously, metrics are always complicated when you dig into them.

S1 (1:09:05) Yeah. So a lot This is super important. Yeah. A lot there. Some things I disagree with. So let's so to start with it, let's just set the baseline for what human beings do. So the most important thing as statistic, I think, is that there's a human driven car is getting a fatal accident about once every 100,000,000 miles driven, and Waymo has driven about 20,000,000 miles. And so they are not yet at the point they've had 0 fatal crashes, but that is not yet at the point where you can say anything about are they safer because you would expect a fraction of 1 death for the number of miles they've driven driverlessly. The other 2, I think statistics that are helpful. 1 is crashes with injuries, which are in the ballpark of a couple million miles per crash like that. And police reportable injuries or crashes, which are serious crashes, but not necessarily crashes that have once involved property damage, but not necessarily injuries. And so, like I said, Waymo has 0 deaths, so they have a perfect record, but there's not enough data to really say much. On injuries, Waymo had a report, last December where over 7,000,000 miles, they had, I think, 2 or 3 injuries. So that's every an injury every 2,400,000 miles. Think that works out too. And, they did some statistical analysis of human drivers on comparable roads in the same metro areas, same kind of roads. And they worked out to run once every 350,000 miles. And so they're like maybe 5 times safer on a per injury for injury crashes. And then police reported crashes, it's 2 and a half times safer. It's half 1000000 miles per crash for the Waymo's and 200,000 for the for human drivers again on in very broad terms. And I think that's good. I think that's pretty impressive. And but I think it's I think that's decent evidence that they're safer, but because we haven't gotten to the fatal ones and that's where the ones you really care about, like you'd hope that they're all correlated. You'd hope that the fact that they're safer on the lower severity crashes means they're also safer on the higher severity ones. But the higher severity ones are so rare that you can't say that for sure because there might be some kind of random bug that happens every 50,000,000 miles that gets somebody killed that, just isn't correlated with the law. So I think that to some extent, the jury's still out. My guess is they're safer, probably by a significant margin, but we're gonna need 2 or 3, 400 mile million miles of on road driving before I think you can really say those are statistically significant. 1 of the things that, so I've been pretty impressed with the transparency of Waymo. So I've I mentioned the 7,000,000 mile study. They also did a a study when they got into 4,000,000 miles where they partnered with Swiss Re, which is a, insurance reinsurance company. They provide insurance to other insurance companies. And so because of that, they have a database of every insurance report ever filed in the auto industry in The United States going back several number of years. And so they're able to do they're able to look in their database and say, for the specific geographical areas where WIM was operating, they will list of, all the crashes and they're able to do apples to apples comparisons. And they had a similar thing, 2 or 3 times safer than a human driver based on injury crashes and kind of police reportable crashes.

S0 (1:11:55) Yeah. That's that's compelling coming out of a Swiss Re who presumably has to ultimately price the insurance for this. Right? Strong incentives to be right in their business.

S1 (1:12:05) Yeah. And strong incentive, I think, not to put their name on it unless they think that the evidence is credible. I think that was a smart thing for Weibo to do is obviously a company puts out data saying we're very safe. You're gonna wonder if they cook the data. And so they can find third parties who are willing to look at the data themselves, maybe supply some of the data themselves and then put the name on it. That's compelling. Tesla is a I do not think Tesla has been as transparent about this. They have a safety website where they say that drivers with autopilot crash once every 7,000,000 7,500,000 miles, which is significantly better than human drivers. The problem with that is we don't know the distribution of those miles both plus so for 1 thing, 1 of the big things that actually we should talk about a little bit more in general is there's videos between freeways and like city streets. Freeways are easier in most ways, like crashes are much less common on freeways, but freeways are also where most of the worst crashes happen, right? The fatal crashes. If you get in a crash on a freeway, much more likely to be a fatal crash. And so the fact that Tesla is much safer, has a lower rate of crashes might just be that people are turning autopilot on in the kind of the safest environments and turning it off in more difficult environments. And Tesla has not done the kind of thing we might have done where they've given the data to their pet experts or brought in third parties to allow people to try to control for that kind of thing. The other source of data we have about Tesla is disengagements. And so there's a crowdsource site that where people like upload videos or records of their drives. And right now that's showing that the latest version of FSD has a disengage model once every 300 miles, which it's hard to know how to think about that because on the 1 hand that's pretty good. That's a lot of miles. If you drove a car for a 100 miles and it didn't make a mistake, that would you'd be pretty impressed. But as I said before, Waymo, 8 years ago was saying that they were going 5,000 miles between disengagements. So the human DARs are actually quite good. And so making a mistake every 300 miles is like not, I think still not close to human level performance. And And I think this is actually, this is a pretty common thing is I frequently see people say, wow, I took a test drive and it was so amazing. And it's how many miles do do 20 miles? That just 20 miles a day. If it does a perfect drive for 20 miles, that tells you nothing about how good it is. You need to drive it for thousands of miles to know if it's human level because the average human driver, 99% of the time, if you do a 20 mile drive with that, it's not gonna be have any mistakes. I think Tesla is definitely making progress. And the other thing is that's crowdsourced. So again, there could be big self selection issues where maybe people are more likely to turn on autopilot or FSD on roads they know work well and or on any easier roads. I think it's hard to put like too much stock into that number.

S0 (1:14:32) It's wild to me that there's not more there. It's just such a strange it's also like maybe an argument for a lot of the complaints that we hear about everything is overregulated and you can't do anything and it's all we're all just handcuffed by the government. Maybe that's a lie and this sort of puts the lie to that. Like, there are others with this would there already been a good safety report and Mhmm. We're all just tolerating it.

S1 (1:15:01) Yeah. So I was yeah. I should answer the second half of your previous questions. Yeah. So 1 thing that has happened recently is NHTSA, the federal regulator on this stuff, has started requiring companies to report more crashes. And so we now have better data, I think, across the whole country. We used to have only in California. California had good data because the state regulator required it, but now the whole country does. What we don't have actually is like the denominator. We don't know how many miles Teslas or anybody else is driving. And so you can probably look and see how many Tesla crashes there have been, but you don't know how many of those how many total miles Teslas are driven or how many FSD was turned on or where they were driven. And so I would yeah. I would absolutely like to have more regulation. But I do think you were saying like it seems like society is like resisting this. I really do not think there's much in the way of policy barriers here. California regulators recently gave way more approval to operate all the way down the San Francisco Peninsula down into Silicon Valley. That was several months ago and they just haven't started doing it. And I think the reason is that they, and this is something we should talk about is they don't do freeways yet. And I don't think there's any regulatory reason. Think it's just that Waymo is not confident they can do freeways safely. And I think it's a combination of kind of caution on safety. Like they're doing the things they think can do safely, but they don't quite think they have it a 100%. But the other thing is that just because they have a vertically integrated business model where they're owning the cars, they're building depots to charge and clean and repair them, etcetera. It just takes a lot of time to just do the work. And so they were in only in Phoenix 2 years ago than they did San Francisco last year, 18 months ago, and now they're doing Los Angeles and Austin. They've grown like 3 to 5 X in the last year. And if they continue growing at that rate, then in 5 to 10 years, they'll be nationwide. But I think they just you look back at Uber and Lyft, it took them a few years to go from just in San Francisco to nationwide. And so I think Waymo is growing close to as fast as you would expect for a company that kind of thought they had a successful product. And it's just going to take because of the business model they have, it's going to take them a while. Now, obviously, if Tesla could just push out a software update and say, okay, everybody's driverless, they can do that very quickly. But I don't think they're ready for that technically. I'm not actually sure if there'd be a legal restriction on doing that. I don't think there would be. I think it's least I don't think that's the main issue.

S0 (1:17:04) Yeah. That's fascinating because we do hear that so much from Silicon Valley in general. Right? This sort of narrative is so often repeated at this point. Right? Where everything's we can't do anything with atoms. Right? It's we all move to bits because we can't do anything with atoms. Mhmm. Now here we have a very regulatable, very suitable by times upwards of 1000000000000 dollar company that is just doing it, and nobody's really telling them no. And, again, even without what I would consider to be just like the sort of safety transparency that I would expect from pure market discipline, honestly, it was like on a consumer reports basis, I would expect to see a little bit better numbers than

S1 (1:17:49) that sounds like a lot of It's like, if you'd asked me 10 years ago, my expectation would be, okay, they'll build the technology and then there'll be some kind of regulatory process. I didn't know as much about the regulatory state at that point. I think there's a couple of things going on. 1 is that like just the auto industry has traditionally been worked on self certification. So NHTSA publishes a big thing called the Federal Vehicle Motor Safety Standards that just says, here's all the things your cars have to do. And when you when a carmaker makes a new car, they're supposed to go through the list, test to make sure it meets all that, and then file a report with NHTSA saying we meet all these requirements. And so if you so what Waymo does is they buy a vehicle that's already certified and then add some sensors and and compute in the trunk. But because it's already certified, like, they just meet the requirements by default. And then the second half of regulation is states. So the federal government regulates vehicle design. States regulate drivers, which until recently was human drivers and driver's licenses and traffic laws and stuff like that. And so there is some variation in the strictness of different states. And so California is 1 of the strictest. They do have reporting requirements. They do require a license to do various kinds of testing, but it hasn't been that hard to get the licenses. And like I said, they're the strictest. So like Texas is like much more wide open. It's like pretty much anybody can do anything. And actually, do see like cruises not going to Texas because I think they found California to be a little too restrictive, but they're not like launching a huge network. They're like I think they're mostly still worried about we'll get sued if we get in a crash rather than, like, regulators come down on us.

S0 (1:19:11) And in the cruise case, what did the regulator even tell my understanding was that they voluntarily stopped their operations without being forced to do so. Is that right?

S1 (1:19:20) I I think the California regulator did ask them to stop. There there was a a crash in August involving a fire truck. I think that the cruise was not at fault, but there was some question about whether they could have handled it better. And they were asked to reduce their fleet by half in August as they and then there was a October crash. California, I think, asked them to stop, but then they voluntarily suspended their other operations where they had, I think, Austin and Phoenix, and they voluntarily suspended that. So it's a mix. There is some pressure from regulators, but it's definitely not like Cruise is, like, anxious to get going and the regulators won't let them.

S0 (1:19:50) Yeah. So going back to the original like, the gold standard of safety data right now comes from Waymo. And is it fair to say that you apply a discount to that? Do you take it at face value? Would you say their reported number is, like, your point estimate for how safe they actually are?

S1 (1:20:10) So it's funny. Like, the tricky thing actually is that, like, a crash is a pretty clear cut thing, and they have a lot of sensors and a lot of, like, personnel. And so I think it'd be pretty hard for them to fudge their, like, crash data. The hard thing is actually knowing how often humans crash because if you look at the data, they'll have things like we sidesliped a shopping cart or we got in a pothole and got a flat tire or things like that. Or I think there was an incident where like a skateboard ran into a cruise car that was stopped. Or actually, there's a bunch of ones where they're, like, parked and another car back. So all those things are in the Waymo and Cruise Safety Report. So we know every time that a Waymo or Cruise Car collided with another vehicle or another person. What we don't know is how often that happens to people because if a person sideswipes a shopping cart, they're not going to stop and file a police report. They're just going to move on. And so part of the trick actually has been like collecting the data. So 1 thing Cruise did a while back is they had this service called Maven that was like a rideshare service or a Zipcar style car rental service. And they had some of those vehicles outfitted with sensors so they could collect they they would rent vehicles to Uber drivers and then use sensors to collect data. And so they had a a database of all the crashes that happened for Uber drivers. Anyway, so, yeah, I'm I'm pretty high confidence in the quality of their data, of their crash data. And I think their safe director is not that much better than the national record, but because a lot of their stuff in San Francisco, they try to get like San Francisco specific data, which is I think it's much more likely to have crashes in San Francisco because it's like high, low speed dense urban situation. And I'm pretty I think that that it's very likely true that they're 2 or 3 times less likely to have police reportable injury crashes than human drivers. And I think based on that, it seems pretty likely that they are less likely to have fatal crashes, but we can't say much about that because we don't have enough data yet.

S0 (1:21:48) So the highways thing remains like 1 big barrier to, like, super broad adoption and, I guess, maybe just even manufacturing bottlenecks and whatever. But Yeah. If that is all true and seems very reasonable to at least have a not super wide confidence interval around that given everything you just said, then why isn't there more, like, enthusiasm for this? If Kennedy were president, he'd be, like, giving a big speech about how we've achieved this modern technical marvel and, like, life is gonna change. And I don't know if you have a stat on this, but, like, people spend a lot of time driving. Right? Just the pure hours that you could get back in some sense to read a book or whatever, if we could really bring this thing online. We just talked about it. It doesn't feel like it's the regulators. It feels like it's the culture. Like what where what's going on with the culture?

S1 (1:22:41) As a person who's written about this beat and wanted people to be excited about this, this is something I wondered about for a long time. I think the expansion, I don't think the sole expansion is not about lack of demand. I think it's about the stuff we talked about. Basically technology doesn't quite work and it just it takes a lot of work to expand I think that's the main thing. I think the fact that it's geographically limited does make it hard for people to think about clearly. Everybody's assumption, including my assumption, when they first hear about self driving cars, oh, it's a car you'll buy and you'll push a button and it'll drive itself wherever you want to go. And for various reasons, that's not how it works. I don't think it's how it's going to work. But then people put it in the bucket of, that's like a research prototype or a thing that's not ready for prime time. So that's 1 of them. But I also think that the freeway thing is really important. So I think we should talk about this a little bit. So Waymo's safety strategy is basically if you're not sure what to do, stop. And on city streets like that works very well. And then they try to get that rate of stopping low enough that it's not annoying other drivers. And that's worked very well for them. That doesn't really work on the freeways, right? Because if you're going 70 miles an hour and there's a lot of other cars on the road, if you like slam on the number 1, it takes longer to stop. So you're gonna be able to stop before you get to whatever the problem is. And then somebody might rear end you and it causes a traffic jam and stuff. And so that's why it's taken them longer to get to freeways because they need to They still need to get to what's called the minimum risk condition, but that means getting to a shoulder or getting to an exit. And so that might mean you have to drive for a half mile or something. And so that's just a harder problem. And so they're working on that. They started testing on freeways in the Phoenix area earlier this year. And I hope that means that in a few months they'll be able to start, but not commercially. I hope that means in a few months they'll be able to start doing commercial freeways, but they haven't yet. And this is actually another reason they haven't expanded is that it's not a very compelling product if you don't do freeways. When I was in San Francisco, something I would do is anytime I would book a Waymo ride, would also pull up my Lyft app and pull up the same ride and look at the time and the cost. It was also a little more expensive and it was always 5 or 10 minutes slower. Waymo is more expensive? Waymo is a little more expensive. Yep. And Waymo is also slower. If it was a longer route where the Lyft would have taken a freeway, it would be a lot slower. Even if there wasn't a freeway, would still be a little slower. I think that's just because it's a cautious driver. It'll like follow the speed limit and not run under lights and stuff like that. But it can be a huge difference. So there's a video a few months ago where somebody did a test drive of Waymo versus Tesla in the Phoenix area. They took the longest route you can do in Waymo service area and it was like a 2X difference. It was like 20 minutes and 40 minutes because the Tesla got on the freeway and went all the way in the freeway and the Waymo was going on service streets the whole way. And so it's just not a very compelling product. I think they have plenty of demand because it's a novelty and people, and I think there's some safety benefits and some people like not having the driver, but there's this offsetting factor of it's just not as fast as a human driven car. I think that'll go away as once they figure out freeways. But right now it's like they want to being ready for expansion and they want to be collecting data. And so they are expanding. But I think the really fast expansion will happen after they figure out freeways and some of the other things. So that it actually has the chance to be a profitable service. I don't think they're gonna it's not gonna be profitable until they figure Freeways out.

S0 (1:25:41) Yeah. It seems like that it's funny. These sort of mirror image approaches of each other. It seems like that can't be too far off. I coming back to the culture or the the politics of it, what would a warp speed look like for this if I was to make a late push for the democratic nomination? Would I show up at the open convention and say, hey. Here's my platform. Here's what we're gonna do to actually bring self driving cars online up. If we can suspend disbelief and assume people actually want that, is there something that we could do as a more society wide level to say, let's enable this technology? Let's not make it, like learn or deal with every quirk that we have. Maybe we could clean up our system a little bit and make it a little bit more friendly. What would that look like, if anything?

S1 (1:26:28) I don't think there's a ton that you'd do. Certainly, could preempt some state and local regulations. I don't think those are a huge factor. But for example, in California, there's been a big fight over individual localities want the ability to, like, license and approve these things, and the state has preempted them. But I don't know if that fight will continue. So having you can certainly have federal regulation that says you have to allow it in every location and maybe move regulation to the federal level so it's a little easier. I don't think it's a big factor though. Yeah. I've heard you mention on previous podcasts that maybe we should change the built environment. I don't actually think that is very important because the hard thing is not figuring out where the lanes are, where the stop signs are. These vehicles are all very good at that. The hard thing is the edge cases, the crash sites, there was a funny case where crews drove into wet cement. I really like another callback. You have Martin Casado on the talk about fatale distribution. It's really that. It's like there's lots of just weird situations in the world and you have to be ready for all of them. And having self driving only lanes or putting sensors on the roads or anything like that, like it wouldn't hurt. But I think it, I don't have any confidence in the government's ability to do that quickly. And I don't think it would add very much value because the kind of common cases are not the problem. Like the vehicles already can if the road's empty or traffic's flowing smoothly, it's pretty much perfect. It's when weird stuff happens. And I don't think there's anything you do to the infrastructure to make weird stuff not happen or make it easier to deal with. Actually, 1 1 1 other thing that I do think so another set of things that have been happening in San Francisco that would be good to replicate is the inter interaction with law enforcement and first responders is very important. So 1 thing that Waymo recently added is the ability to for the, like, the San Francisco 9 1 1 or the fire department to, like, geofence an area. So this is a fire zone, so keep vehicles away from it. And then Waymo cars will automatically route route route around that. And they've also been struggling with the interface. So if a car gets stuck, 1 thing is nice is the firefighter can just jump in the driver's side and drive the vehicle. So having those kind of standards of here's how government can signal cars where to go, and here's how they can take control if it gets stuck, and here's supporting that kind of thing. The most useful thing at the federal level is both do something at the federal level and also develop standards for state and local governments. So I would say that's the big thing, but I don't think this is the main issue. I think the main issue is just like technology and then figuring out the business aspects of it.

S0 (1:28:34) It's a bummer. I was I was hoping that there would be some the platform for me to run on. But the

S1 (1:28:38) thing is coming, this is the thing is it's not like nothing's happening, right? Like Waymo, as I said, is I think they were doing about 10,000 trips a year, a week and a year ago, and now it's like 50,000. So another 5 x growth a year for 2, 3 years, and it's gonna be a significant thing. Like, I would like to get a little bit faster, but it's not yeah. It's not the thing like it's stuck. It's just that The US is a big country, so it takes a while to get to everybody.

S0 (1:28:59) I do still think of that stop sign, and there's also a true a stop sign that's behind a branch of a tree on my way to the highway, which my friend who lent me the Tesla specifically called out and was like, because you live here, that there's a stop sign at every corner in our neighborhood. But it misses that 1 every time, and so you're gonna have to watch out for that. I do feel like there is something there, especially if we're getting into like really long tail things. Just make sure the signs are all visible. There are

S1 (1:29:27) But but like, has a map. Difference? This is why you need a map. This is why it's helpful to have a map is like when has the map and I assume that stop sign is correct there. The thing to do there would not be to put to do anything to stop signs. It would be like publish an accurate database that says here's all the maps with their GPS coordinates and whatever. So that could be a good thing is if the government provided data to companies that said here's all the traffic rules and places you're supposed to stop and stuff, that could work. So, I'm not opposed to that. I think it would definitely be like in general, it'd just be nice if stop signs were not covered up, but that's like a 2% faster kind of thing if that's not the main thing holding things back.

S0 (1:29:59) Yeah. Gotcha. Okay. In terms of the market and how things develop, going back to really where we started, there's these competing visions of a car that you buy and own like a standard car today that drives itself. Obviously, that's what Tesla's pushing toward. And then we've got the robo taxi service. To what degree do you think those remain, like, separate things? Obviously, we've heard from Tesla that they have plans for robo taxi. I don't know what would limit the car companies wanna sell cars. Like, I think they might be scared by the idea that, jeez, if if utilization of cars goes up 10 x, like, what does that do? Demand for cars. So I would imagine if a whatever. If a Toyota or a or a Ford wanted to do a deal with Waymo, there would at least be some strong reason for them to think, can we make this something that people can have parked in their driveways most of the time instead of actually on the road all the time? Yeah. Yeah. Do you see, like, a convert is there a possibility of crossover, or do you see convergence, or are these kind of?

S1 (1:30:58) So I used to be 1 of these people who thought that self driving is going to mean that nobody's going to own a car. And I've chilled out on that a little bit because I think people really like to own cars. You've car seats or you like to have your golf clubs in the trunk or you just like the security of knowing your car is in your driveway and it'll be ready whenever you need it. So I do not think think car ownership is going away, although I think it will probably get as taxis get cheaper, which I think self driving will make it like the equilibrium will be such that there's fewer own cars and more taxi rides. But the way I envision this market working is I think that you're going to need support infrastructure maps, but also other kinds of things anywhere that we're a level 4 self driving car operates. And so I think what you're going to see happen is Waymo will expand nationwide. You'll have a taxi in every area. And then once Waymo service area is 67% of the country, then Waymo can go to an OEM and say, how about if you add a Waymo service where it's a $100 a month and then your vehicle has built in like the Waymo service where you can subscribe to that and then you push a button and it'll drive itself. And I think that'll probably take 10 years to get to the point where Waymo's not able to offer that. And maybe somebody like Wave will get there independently. Going the other way, I'm pretty skeptical of. Tesla claims they're doing robotaxi. I think you need that infrastructure. Teslas are gonna get stuck sometimes. And if there's nobody in the car because what Elon has this vision that you'll buy your Waymo and then go on vacation. And while you're on vacation, your car will be driving around earning money for you. But what if your car gets a flat tire? Like, you're not gonna come home for your vacation and fix a flat tire. Like, presumably, Tesla is gonna have to send somebody. And it's not like it's impossible for Tesla to build that infrastructure, but that's like not the business they're in. They certainly, I don't think have started like building that infrastructure. And so I think it'll take a few years at least for Tesla to be ready to offer a Waymo taxi service. And I'm not sure they have the corporate culture or the infrastructure or anything to do that. And we'll see. But I guess I don't I'm not that optimistic about the robotaxi part of that. I think that they're more likely to stick with the service they have now, and maybe at some point, they'll be ready for driverless. But, I think it's probably a few years away still.

S0 (1:32:56) Who do you think does have that culture? Because I wouldn't think of Google as being, like, the obvious company to really wanna handle all the flat tires. Uber? Yeah. Think this is a think it's a big question. Drivers lapse, right?

S1 (1:33:08) Yeah. No. I think it's probably going to be I think that's a big open question. Definitely. I do think 1 of the other reasons that Waymo may be growing relatively slowly is that Uber has or that Google has always operated in high margin businesses where massive scale software and so they just pay top dollar for the best talent and don't worry too much about like individual productivity. They're not like scrappy the way Amazon is. And so I could see totally see a scenario where their technology works perfectly. They're not making any mistakes, but it's just everything costs like 50% more than it would if Uber were doing it. You actually see this to some extent with Uber and Lyft as well, where Lyft takes like half of every fare. And it's because they have this building of 5,000 software engineers, that's actually like a large fraction of Lyft's revenue. So I do think that companies are going have to figure out how to be a little leaner to run-in this kind of more cost conscious market. And I can imagine a number of ways, maybe when we'll have some kind of franchising thing. So a while ago they had to deal with Avis to help them manage some of their vehicles in Phoenix. I think this was the 2018 time period when they thought they were going to scale rapidly. So I could see them either licensing their software to other people or having some kind of franchise model or a sub country. I don't know. There's lots of different ways you can imagine, but I'm sure that there won't be like the nationwide Waymo network. I doubt it'll be like all like Waymo employees doing everything. I'm sure they will have some kind of partnerships with other country companies. Uber is a good example. I do also think getting the 2 sided market of rideshare to work and build it up is very expensive. Uber and Lyft spent a decade spending billions of dollars to do that. And so 1 approach would be just you make Uber the partner, right? And I don't know if Uber owns the vehicles or maybe the third company that owns them, but as the ride hail app, like maybe in the future, like actually most of the way more vehicles are dispatched through Uber or Lyft. I think that could be fine. So yeah, in terms of market structure, think it's very much an open question. And I don't think Waymo, because it's not actually a profitable business yet, I don't think Waymo is like thinking super hard about this. The easiest thing for them to do is just to run their own ride hailing services and maybe that'll work. But once they get to, okay, like this is actually like a viable service, which I think is probably a year or 2 away, then they'll start thinking about how do we scale up really fast, and that probably will mean finding partners or or thinking about other business models.

S0 (1:35:14) Yeah. Amazon definitely comes to mind. There is somebody that already manages, like, 1000000 vehicles and has a pretty good reliability rate for making sure the package ends up on my porch and dealing with it when it doesn't. So Yeah. Some ways, that's really in their wheelhouse.

S1 (1:35:30) Yeah. I can see a world where a year from now, Zoox starts doing driverless, and then they scale it much faster than Google. And so they end up being as big as Google, as big as Waymo 5 or 10 years from now. Yeah. That's 1 reason I wouldn't count them out is they have a lot of those, that expertise that that Google doesn't have.

S0 (1:35:45) Are there any other surprising scenarios that you think are underestimated? Like, 1 that I just came up within preparing for this was Tesla currently charges $15,000 for the feature. An argument could be made that they should just give it away to scale even faster and try to dominate and then figure it out later. They can always charge again later. I don't know if you think that is at all realistic. And then I also wonder about just like weird partnerships, alliances. This is something I've been really struck by in the context of AI in general is just how we've got OpenAI partnered with Microsoft and Apple at the same time. Like, what? I never thought I'd see a single core technology provider could play both sides of that divide. So, Is there any and also they're all the at the same time, their core product is free. Right? They're good. You can go to chat GPT and use GPT 4 for free. And yet they've managed to do these strategic partnerships. Is there is there a move there that Tesla could make something free and partner with Afford or whatever?

S1 (1:36:46) I don't think that like the cost of FSD is their main bottleneck. The $15,000 is just a way like when Tesla was a kind of desperate situation, 3 to 5 years came from a cash perspective, this was a way to raise cash. And they are shifting to a monthly model. Like you can get a $100 a month F and C. And I think that probably makes more sense. It's long run. Think that's right. Yeah. Per month. I think that probably makes more sense. It's going to be a service, not just like a piece of software you buy 1 time. So I think that does make sense to have a recurring kind of revenue model. In terms of like other ways the market could unfold, to some extent, I think that the kind of crazy wild west bunch of money flowing in period that in self driving was that like 2016 to 2018 period. And I think there's been like, I think it's like probably 5 years behind the kind of generative AI. And I think that there was some of that kind of stuff happening back then, but then a lot of those projects just went out of business. And I don't expect to see a lot of that the way we do now. Obviously once if Waymo becomes very successful or Tesla or somebody, a new generation of startups might come in. But right now it's just, I think in a consolidation mode where the companies that are left are just paying on trying to survive other than so the non Waymo ones. It's just can they get to market at all? And so nobody's going want to put new money into them until they show some success. And the question is, yeah, can they survive or do they go out of business?

S0 (1:38:03) There's sort of an analogy there to the large language model market, I would say, which is this has happened in compressed time. But just over the last couple of years, you've seen this. Obviously, emergence of, like, very notable leaders with compelling technology, a bunch of people who have raised, like, very significant money to try to get into the game. And as soon as we're already hitting a period where that second tier is being thinned out, it's almost like the history of self driving has like rerun for large language models in 10% of the time or maybe 20% of the time.

S1 (1:38:36) Yeah. The current large language model market feels a lot like that early self driving market to me. There's a lot of I have trouble working out the kind of math of the second, third, fourth tier LM companies, like how they're gonna make a profit. Like, I'm sure 1 or 2 of them, but there were a lot of self driving companies that were just like, we're doing self driving and they were like sixth the or seventh best self driving company and they didn't make it. And so I would not be surprised. And I also see a similarity in this kind of long tail problem where people like, all of can do 90% of X and how hard can it do the last 10%? And what you learned is self driving is that last 10% or last 1% really hard. And I expect to see some of that. I don't think we'll see the same degree of crash because clearly there are some things, customer service automation and various kind of documents. So I think there clearly is our use as prelims. I don't think they're going to go away. But yeah, the crazy valuations I think are going to be hard to justify. I definitely expect some consolidation or some of the kind of second tier companies to go out of business in the next year or 2.

S0 (1:39:30) Yeah. I think that there's a musical chairs game that's already underway, and the music is it doesn't seem like it can play that much longer for a lot of these second tier language model companies. Yep. Which is not something I take any pleasure in coming to that conclusion, but it does seem hard to argue otherwise at this point. Yeah. The I'd be interested to hear a little more. Know you have a map from that goes a little, I think, a a little deeper even than what you just said on what your expectation is for, like, future of AGI and whatever. I would put a little pin in that I think it is a lot easier to control the environment for language models, And that is a skill set that if we do in fact see this stall out that some are predicting, which I don't predict to be clear as I think, yeah, we're just stuck at GPT 4 plus a bit. Then what I would expect to happen over a long time is this kind of FST sort of slog, except instead of grinding out for all of the additional nines that you need for a a functional FSD product, I think what we would see in the language model side is like a lot of systematically sanding down rough edges of different different business processes so that you can, like, reliably feed the language model something that it can handle. And in that way, you could probably automate still a ton of work across the economy. It's just that everybody's in this moment now where we're not used to the technology, and there's the expectation that it's gonna get a lot better and a lot easier. So do I really wanna put in all this elbow grease now to scaffold this thing up and make sure that I have it dialed in when I can maybe just wait for g p d 5, and maybe it'll solve all my problems. But it sounds like you you don't think that g p d 5 is gonna solve or even 6 is gonna solve all our problems based on your decade following self driving.

S1 (1:41:14) Yeah. I don't have a super strong prior about this. And I do think the the the 1 obvious difference between LMs and cars is LMs don't kill anybody. And so really hard thing about self driving is it has to be close to perfect or you don't have a product at all. Whereas with LMs, there are you say, there are some either some like narrow domains where you can get it perfect even now or domains where the human being checking the output. And so it's fine if it makes mistakes sometimes. And so I definitely expect there's enough, like you said, enough business processes and kind of small scale stuff that there'll be a profitable business for somebody. But the hopes of the really big stuff where we're going to have AI doctors or AI lawyers or programmers, things like that. Like I guess 1 of the similarities I see is that the, I think people are maybe underestimating how much the hard stuff is different from the easy stuff. People see, okay, Tesla can drive on the freeway. And so we're like almost how much harder can it be to do cities or to do crash sites or things like that. And similarly, like the routine doctor's visit, I think you can do pretty much with kind of a checklist, but then part of what's valuable as a doctor is they'll recognize, Oh, this is a really weird situation. And there was a information from a checkup you did 2 years ago that is relevant to this. And that long context, like abstract reasoning kind of thing. It seems to me like LMs are pretty bad at that still. And I'm not going to say that GPT-five definitely won't do that, but just scaling up the transformer does not seem to me like it's necessarily going to do that. Think that it's going to be some need to be some like some kind of deeper improvement to how these things work. And, I think that might take a few years. I'm very interested in your Mamba piece or your Mamba episode back in December got me interested in that. I'm working on a piece about Mamba. I can see that being 1 piece of that, and that seems to be several years from being a large scale. But anyway, yeah, I have a lot of I have a lot of uncertainty about this, but I guess I've just learned to be skeptical about confident claims that, oh, this if we just scale this up or keep doing this, it's gonna everything's gonna work. Sometimes that's true, but but often it's not.

S0 (1:43:08) Yeah. I think if there's 1 thing I've learned, it's and I recall professor Mike Levin saying this about a very different subject, but he was like, if there's 1 thing I could say is that we should not be confident because we've been repeatedly surprised, and it seems like we're we'd be I don't I there's no rational basis to expect that the surprises will stop on that could go in either direction.

S1 (1:43:27) And I think we've been at we've had a period where the surprises are most of the upside is all these other things, autonomous work better than we expected. In a similar way, like in 2014, 2015, 2016, self driving cars kept pretty quickly from like nothing to like having things that were like pretty close. But then you can enter a period where the surprises are the other direction is now we've updated our expectations and we were optimistic. And then actually that was right. You're on the kind of other side of the S curve. So I don't know when I hit that, but I'm just like based on my experience, I'm expecting that for periods like that to happen.

S0 (1:43:57) Yeah. Honestly, I think it would probably be in some ways for the best if there was a little bit of a just longer window of time for people to absorb what technology already exists before just getting washed over by yet another wave of it. Maybe last thought, and then I'll I'll give you a chance to if there's anything we didn't cover that you can that you wanna touch on, we can add. But I'm in Detroit, obviously, a famous auto town. You're in DC, famous rulemaking town. Which 1 of us do you think gets robo taxi service first?

S1 (1:44:29) Oh, me. No no question.

S0 (1:44:31) You think they'd want to go there to impress regulators?

S1 (1:44:35) Waymo has started testing here. I think that so several things. 1 is that cities are easier. We have very few freeways here. I I think probably Detroit has more freeways.

S0 (1:44:44) Got a lot of freeways.

S1 (1:44:45) Yeah. 2, they are testing here. The third 1 is snow, which I don't think is going be a big obstacle, but I think if you look at the place Wimla has gone so far, it's like California, Arizona, Texas. So I think they'll go from the South to the North. And so, yeah, I expect all the southern cities to pad it before any of the northern cities. And I think DC is probably southern for that purpose because we only have 3 or 4 days of snow.

S0 (1:45:08) Breaking my heart. I don't know if this is something I'm quite ready to move for, but an advocate for being willing to change the environment, this is maybe something I should at least consider on some level.

S1 (1:45:17) If you can come up with a policy agenda to get rid of all the snow in Detroit, that would I think that'd be pretty popular. But

S0 (1:45:23) Yeah. No doubt. Difficult.

S1 (1:45:24) Other than, like, massive global warming.

S0 (1:45:25) It may imply other it may imply a new downside. Cool. Well, this has been a great run through of an industry that I am super interested in, and I think we're both hoping is really gonna start to work sooner rather than later. Anything that we can touch on that you wanna make sure we mention?

S1 (1:45:39) No. I I think we covered it. I really appreciate the thoroughness of these episodes. I really I rely on you for situational awareness of a lot of things that happened in the industry. So it's been fun to be at the other end of that process.

S0 (1:45:50) Thank you. I appreciate that. And it's a lot to look up to, but I will do my best. For now, Timothy b Lee, author of the Understanding AI newsletter, which you can find at understandingai.org. Thank you for being part of the cognitive revolution.

S1 (1:46:04) Thanks, Nathan.

S0 (1:46:06) It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.