Looking Up The AI Exponential with Azeem Azhar of The Exponential View

Full Transcript

Full Transcript

Azeem Azhar: 0:00 Moore's law was turning into this acceleration of AI capabilities over the top, and the the other was that what was happening in renewables with things like lithium ion batteries and solar was on a Moore's law like trajectory. And there were some other technologies like genome sequencing and genome synthesis that seemed to be doing something similar. So I started to bundle those up in this idea that, we're going through the exponential transition, a transition where our economy gets driven, you know, not by the economics of the oil industry and internal combustion engines and and telephones, but by AI and renewables and that these technologies being on an exponential trend, being fundamentally, at some level, kind of information technologies behave really differently to the ones of the previous generation.

Nathan Labenz: 0:51 Hello, and welcome to the Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my cohost, Erik Torenberg. Hello, and welcome back to the Cognitive Revolution. Today, I'm speaking with Azeem Azhar, founder of The Exponential View and fellow AI scout, for a wide ranging discussion about the transformative power of AI and its implications for humanity. Azeem's core observations that we are in the midst of a transition from an economy driven by the likes of phones and oil to 1 fueled by AI and renewables and that these new technologies, fundamentally rooted in information and already growing exponentially, behave differently than their predecessors, will be familiar to Cognitive Revolution listeners. So I took the opportunity to get a bit deeper into Azeem's worldview and expectations for the coming years and ended up having what I think was a really engaging and delightful exchange. In this conversation, we cover a number of familiar topics, plus some new ones that we haven't explored in quite the same way before, including why Azeem believes that while incumbents are racing to adopt AI technology, yes, startups are still likely to drive the true disruption in the form of entirely new products and markets. Also, what forward thinking business leaders are doing today to retrain their teams and to position their companies to lead their respective markets in AI adoption, And why Azeem still finds it necessary to challenge them to think bigger, asking the question, what would they do if they had 1000000 times more compute? We also explore why Azeem agrees with Sam Altman's recent comments that AGI will likely arrive soon, but will be less of a big deal than people expect, at least initially. How AI is likely to change human relationships, especially considering the rise of so many different forms of AI companions. What sorts of new governance mechanisms the AI era will require. And finally, why we might ought to worry about what I call the great embedding. That is the seemingly natural tendency for AI systems to communicate with each other in high dimensional vector formats, which while efficient for them, is broadly inscrutable to humans and could lead to various loss of control scenarios. As always, if you're finding value in the show, we'd appreciate it if you take a moment to share it with your friends. Azeem's perspective is both imaginative and exciting and also grounded and sobering. So I definitely recommend this episode to anyone trying to get a better zoomed out view of the future. I would also encourage you to consider subscribing to Azeem's work directly, which you can find at exponentialview.co. Of course, your feedback is always welcome, whether in the form of an Apple or Spotify review, a YouTube comment, or a DM on the social media platform of your choice. Now I hope you enjoy this conversation with founder of The Exponential View, Azeem Azhar. Azeem Azhar, founder of The Exponential View, welcome to the Cognitive Revolution.

Azeem Azhar: 3:58 I'm really happy to be here, Nathan. I've enjoyed so many of the episodes and wanna thank you for your hard service as a red teamer for GPT-four. I watched that episode with my jaw on the floor.

Nathan Labenz: 4:12 Well, thank you very much. That's kind and right back at you. I've been binging your feed lately, and we've got a ton of things to talk about. I guess for starters, I would love to get your kind of summary of what it is you do. I think it's actually kind of similar to what I'm trying to do. I describe myself, as you know, as an AI scout. And, you know, I talk to mostly people on the show who are either very deep on a particular line of research or building a product, but you are 1 of the few guests who have this kind of very zoomed out view and really are working to understand the big picture. So how do you describe what you do and what goes into it?

Azeem Azhar: 4:50 I'd I'd love to. Yeah. I I mean, I've been in the tech industry for a really long time. I started working in '94, and I built my first websites in in '93. And just over the back behind me, can see in out of focus my first computer, which is a ZX 81 Timex1000, in The US. And about 9 years ago, my last company was acquired, and I just started to write a newsletter. And as you do, writing is thinking. And I noticed within a few months, there were these few trends that were going on that were pretty significant. 1 was, you know, Moore's Law was turning into this acceleration of AI capabilities over the top. And the other was that what was happening in renewables with things like lithium ion batteries and solar was on a Moore's Law like trajectory. And there were some other technologies like genome sequencing and genome synthesis that seemed to be doing something similar. So I started to bundle those up in this idea that we're going through the exponential transition, a transition where we our economy gets driven, you know, not by the economics of the oil industry and internal combustion engines and and telephones, but by AI and renewables. And that these technologies being on an exponential trend, being fundamentally at some level kind of information technologies behave really differently to the ones of the previous generation. And so what what I what I do now is I try to make sense of that as a as a system. We do that through I do that through my newsletter, but I also do it from time to time through investing and advising. But I think that what is what is going on at its heart is in about 20 or 30 years, we'll be looking at an energy system that is gonna be very, very heavily renewable. We'll be using much more energy per capita globally than we do today, that we will have tons of intelligence in our economies and in our lives through through AI, and that will have real knock on effects. And trying to make sense of that in a pragmatically optimistic way, So somewhere between the extreme dystopia and the extreme utopia is really my mission.

Nathan Labenz: 7:00 Yeah, cool. Well, I'm hoping we can find that sweet spot as well. So right there with you. In terms of the intellectual history of this notion of exponential technologies, 9 years ago strikes me as kind of a doldrum time for that paradigm. I wonder if you experienced this, but I recently had a conversation with a guy who makes his living as a speaker at corporate events and who is positioned as a futurist. And I showed him a little presentation that I had put together in which I pulled out 1 of Kurzweil's kind of late 90s exponential curves graphs. And the title of that slide in my presentation was Kurzweil was Right. He saw the name Kurzweil and he was like, Oh God, don't talk about Kurzweil. No, everybody He's totally discredited. And I was kind of like, that sounds like somebody who maybe got some pitches dinged in that kind of time frame when you were getting started with this notion and has gone away from it. But maybe that's just the weird nature of exponentials. What was your kind of experience of were people sour on that? It seemed like there was the great stagnation thesis for a while there, and your start of the exponential kind of lines up with it, it seems like.

Azeem Azhar: 8:15 When I started, what I noticed was people weren't really talking about AI. I mean, was 2014. Machine learning was still the word. It was before AlphaGo had come out and and done its thing. DeepMind was doing really interesting things with reinforcement learning in video games, and you could see that something was happening. And I think that you had TensorFlow as the sort of stack of choice for building convolutional neural networks to do their machine vision tasks. So it seemed reasonably early to be talking about these things. And actually developers were struggling to make sense of CUDA, which is the sort of API to the NVIDIA GPUs that everybody now uses, because that had only been documented 5 years earlier. There is something that happens around 20 13, 20 14, 20 15 that I think is really worth paying attention to, which is that in 2013, Apple becomes the largest company in the world. And within a couple of years, those top slots are occupied by what we used to call the Fangs, Facebook, Apple, Google, and and and Amazon, or the Gaffers, pardon me, not the Fangs. Netflix snuck in briefly during the hype cycle. And so you start to see the the companies of the industrial age, Exxon, GM, and and so on, fall off that fall off that list and and stay off that list. So that's an important economic moment about the the the sort of forward looking stock market saying like something is changing. The second thing that starts to happen around 2014 is we see the first market for electric vehicles go past that threshold of 5% of new cars being sold being electric, which was over in Norway. And that 5% threshold is normally the trigger for when you see the s curve of adoption, right, and you get into that vertical or that verticalized part of the curve. You also started to see solar power being starting to be cheaper in roughly a third to a half of the contracts around the world than fossil fuels. And and and then, of course, we are 3 or 4 years into the deep learning wave, and and that's long enough for companies to start to ship products. So I think that there is a a moment which we could argue when we look back on it feels like a a sort of historical turning point. But we also have to be realistic that the the mathematical function that is an exponential curve, when you stand on it, it always looks horizontal behind you and it always looks vertical above you in front of you wherever you stand on it. It's a smooth curve with no obvious turning point. And someone like Kurtz Feil, you know, I think did such great work 20 years ago to articulate the exponential trend in computing going back from the 18 eighties actually in mechanical computers. I think 1 of the reasons why he slightly falls out of favor is because he was probably brave enough to extend the curve far further than, we might have otherwise thought. And I think a couple of things that he got wrong was that some of the assumptions that we would have had about how the human brain works in like 2001, 2002 when those books came out were wrong. Right? Science proved that it was more complex and it wasn't going to be a simple game of raw computation 1 for 1. And that's where I think that people start to look at him and say, well, was this just someone throwing tea leaves? I think there was much more to it than that having read his work reasonably carefully. But I think what's interesting is that we see these curves happening elsewhere. We see them in lithium ion batteries and we see them in genome sequencing and it's not clear why that should be the case upfront.

Nathan Labenz: 11:54 The implications of the fact that we may still be standing on the part of the curve that, as you said, it kind of always does. If it's still the case that what's in front of us is vertical compared to what has been behind us being horizontal, then we're in for a bit of a wild ride. I think we're headed for a steep curve for at least a couple more years. I mean, beyond that, do we sort of hit a plateau is a lot harder for me to predict, but I honestly don't see any fundamental reasons that we would right now.

Azeem Azhar: 12:25 I mean, what Kurzweil says is that it's actually this curve is a series of layered Ss. So you have 1 particular technology architecture, It's very slow to develop. It hits this inflection. As an s, it starts to accelerate. And as it hits its flat point, the social dynamics of market incentives have meant that another set of research has come in with a different architecture, a different way that that extends that that s up and looks from a distance like a a smooth s curve. And that's really nice and descriptive, but it also, I suspect, people who are kind of really robust with their theory feels like, well, that's just praying. That's like the turkey that's been treated really well up to the day before Thanksgiving, and, like, we should worry about what happens the next day. What what I've tried to do is I've tried to get into the underlying mechanisms of why these things improve, why they get cheaper. And then what we need to do is figure out where does that mechanism fail. Because if the mechanism doesn't fail, then that trend is going to continue. And if it does fail, then we can say, well, we need a new mechanism. Is there 1 in the research pipeline that might deliver it? So I would say that if you look across the the gamut, I mean, example, batteries and and solar power, we've got definitely got more than a couple of years to to run-in terms of price declines. When we look at compute, I just feel it's really hard to bet against. I just you know, I think that I've been hearing about the death of Moore's Law for 15 years. And, you know, Moore's Law is is helpful, but the question to ask is how much compute can a developer get for a dollar each year, and do we really think that that is gonna stop declining for, like, a long period of time? And I I find that 1 really hard to to to to support. And I just I think it continues for for a variety of reasons.

Nathan Labenz: 14:17 It sure seems like it. I mean, the CPU to GPU transition feels like a classic example of 1 of those kind of 1 s curve perhaps giving way to another. And in the GPU, you know, we're not it definitely feels like we're still in the steep part of that particular s curve. Hey. We'll continue our interview in a moment after a word from our sponsors.

Azeem Azhar: 14:39 We're we're not we're not done yet. And I think the thing that is is fascinating is that NVIDIA is obviously doing incredibly, incredibly well, and it yet doesn't yet have the threat of real competition. And and what was fascinating in the CPU world was that Intel did very, very well for a really long time. I guess people forget this, but if you've been around for a while, you remember Intel was this sort of monopolist in the perceived and Andy Grove and, you know, only the paranoid survive. And it did really well only with the threat of competition because AMD never got more than 20% market share, and that's enough to propel people forward. All the incentives seem lined up for there to be massive amounts of investment in scaling existing silicon chips and developing new new systems. I mean, you saw in the last few days before we recorded this, Amazon and Google both reported 30% growth in their cloud businesses. When I've talked to bosses of really big companies, they are spending money on compute, you know, really like nobody's business and their expectation is that it will grow. They don't often think it's compute. They say they're spending it on AI, but that at the end of it is gonna be GPUs cranking away. So with all the incentives aligned, I struggle to see us hitting a brick wall. It doesn't feel like it's the Carnot cycle. Right? So the Carnot cycle was the thermodynamic limit for the efficiency of an internal combustion engine, something you as a native Detroit man, you know, know know very, very well. But we keep finding ways of eking more out of our compute, and I think we'll, you know, we'll continue to do that certainly beyond a couple of years.

Nathan Labenz: 16:20 Just for kind of conceptual grounding, and I'm also interested to hear how you explain this to the business leaders that you work with, because their understanding and their eagerness to adopt is a pretty key question in my mind as to how the next few years are going to play out. But we have kind of 2 notions 2 2 definitions of of kind of types of technology that both seem to apply to AI in my mind. 1 is the exponential technology, and the other is the concept of disruptive technology. Disruptive technology, classic textbook definition, is a cheaper but inferior alternative that competes on the low end of the market. It seems to me that AI is kind of both. Right? It's we've got, like, these at least exponentially growing inputs. Although on the other hand, you could sort of say scaling laws sort of suggest that, like, the model capabilities are more, like, logarithmic, so those 2 things maybe, like, balance out somehow. It does seem like it's disruptive in that AI is typically an inferior but cheaper alternative to asking somebody to do something for you. Right? So it's also a general purpose technology. I guess, what the kind of key definitions, and how do you think about mustering those different frameworks so that people have good clarity on what it is we're dealing with?

Azeem Azhar: 17:37 I mean, is it's so hard because it is so it is so general, and it it is also a technology that improves other technologies and itself directly. You know, not in the way that electricity improves electricity. You know, electricity makes the economy more efficient, and so you can build more electrical power stations. But AI seems much more direct. I do think it's important to understand its its generality, to get people to wake up to the idea that a general purpose technology really transforms the world beyond the economics. And, you know, again, Detroit is a great example for this because the car transformed the world in a in a very short period of time. Where where I live, in Northwest London, 120 years ago, this was all fields. And within 20 years after that, by about 1925, the roads were laid out the way they were and the houses were built way they are. And a century later, we're still like this. And this is because of, ultimately, the car as a general purpose technology. So to because they don't come around very often, I think that's a really good starting point. On the question of disruption, that is a, I I think, a kinda higher order question because that is about products and how they get bundled to provide value in a particular environment. And I think that, you know, when when you start to bundle AI to do that, you can ask that specific question. But 1 of the things I think is really important is, and I think companies started to think like this, is to think in terms of tasks rather than jobs. Because AI can't replace jobs, anyone's job, because there's just so much in an ordinary job, like logging onto Zoom and saying hello to somebody and getting your neighbor a cup of coffee, that is beyond the scope of any AI system. But within tasks, I think you can start to unpick this. And that's why in a sense you might start to say, well, AI becomes a disruptive technology because on a like for like basis it can't replace an entire human in their day to day, but it might get better and better. But I think what's more helpful is to go back to that task question. And then when we come to that, the question is on a task basis, is AI really a cheaper version of a human doing the same task and a worse version? Does that matter and is that always the case? And I have certain tasks which I do where I think I could not afford to hire a human to do this task as well as ChatGPT does it for me, you know, in a minute or 2. And and that may well be your experience as well. I mean, I, you know, I I use this for to write letters of complaints, to get my parking fines reversed, to help me think through holiday plans, to do research for my book. And and so, I mean, it's just it's so variable. And in many cases, I would be better off finding the very best human to do that, but the cost of doing that, even finding them, is so high.

Nathan Labenz: 20:39 Yeah. Your search costs alone would dominate.

Azeem Azhar: 20:41 Yeah. Search costs would dominate. Mean, you're using GPT-four or Perplexity or whatever you use, do you have it across a whole range of different tasks that you do from the most strategic for your business to the most sort of trivial home task?

Nathan Labenz: 20:57 Yeah, maybe not the most strategic yet. At that level, I would probably restrict myself to brainstorming interaction at most, but certainly lots of things I get very efficient and immediate help on. And I think that immediacy is super important too. I have this 1 slide that I call the cognitive tale of the tape, which kind of lists out 12 dimensions and compares human to today's AIs. You know, then we can also consider what future AIs might look like. Very much agree with your notion of distinguishing between jobs and tasks. And for a while, I was calling this the great implementation, and I've that phrase hasn't quite taken off yet. But the idea there is with inspiration from, like, your, you know, and your Benedict Evans type business theorists, you know, the the way to make money in business is to bundle and unbundle. I do think that unbundling jobs into tasks is a very good way to think about it. And then for any given task, you go down this tail of the tape and you're like, yeah. A lot of them AI can do as well or even better than a human or certainly better than a human that I could find without huge search costs.

Azeem Azhar: 22:14 You know, and we get we get quite surprised with some of the results. I think 1 of the really big surprises is that if we went back 6 or 7 years and you had books like The Rise of the Robots and the famous Oxford paper saying machine learning could automate 40% or x percent of jobs, was written by a friend of mine, our assumption was that it would be routine cognitive jobs by which what people meant was customer service and data entry in like Philippines or in India. And what we're actually discovering is that it's jobs that we would have categorized as non routine or even creative where this technology can really start to make a difference. It really, really is surprising. 1 that really I was not expecting was a paper at the 2023 which looked at empathy ratings, of doctors giving advice compared to pat GPT or GPT-four giving advice, and patients were rating the robotic advice as more empathetic as humans. And the whole argument had been, let's use AI so the radiologist can spend more time looking you in the eye and being attentive and being and being empathetic. So part of the challenge, I think, is that there is this lack of clarity and lack of knowledge about where and when will these AI systems actually compete with humans on particular tasks. And then when you think about it, actually, what is empathy? Empathy is about active listening and it's about being incredibly patient. And there's nothing that's more patient than a robot that has no sense of time and is stateless. Right? I mean, it'll just sit there forever.

Nathan Labenz: 23:50 An early GPT-four test that I remember fondly and and do think is kind of a sign of things to come was simulated tech support for my grandmother when she needs help with her iPhone. And it was, you know, just a a flash of this you know, you could call it sparks of the things to come where I played the role of her, which, you know, and I I she calls me, right, when she needs help with the iPhone. And, you know, you're in this dynamic. And it's actually an interesting dynamic also for a pure text situation because she's typically on the phone with me looking at the phone and saying, you know, I can't my friend sent me an email, and I can't get it. And then I'm like, okay. Well, I always start with, what do you see on the screen right now? Can you start at the top and just read everything that you see on the screen? And at times we've had some very kind of

Azeem Azhar: 24:42 funny Does she always read everything that's there? Like, you know she's missed something out. You're like, grandma, isn't there a word above?

Nathan Labenz: 24:50 No, she does pretty well. Yeah. She'll start at the top, Verizon, the time. And then there have been a couple of times where, you know, some 1 of those system dialogues pops up and that like just didn't even register to her. But then when she got it to reading, I was like, wait, you need to hit okay there before you're gonna be able to hit anything else. So it can be these very simple things. But GPT-four, as you might expect, did really quite well on that. There was a little bit of it was in kind of the middle of where it did not have the UI of an iPhone memorized, so it was kind of hallucinating it and guessing. And yet the guesses were close enough. Know? And I really had to study my iPhone and what it was saying and and kind of compare, like, is the is it saying what's actually there or not? And mostly was, but it was clear it was kind of filling in some gaps. But the real eye opening moment for me was it said something that I thought she might feel was a little bit rude. I forget exactly what it was, but it was like, you know, it was like, start at the start in the top left. You know where the top left is? Something like kind of that basic. And then I responded as her saying, yes. I know where the top left is. I I'm not dumb. I'm I'm just struggling with this phone. And then the AI comes back and says, I'm so sorry. I didn't mean to offend you. I'm just trying to make sure we're resetting here and helping you through this process. And that was the moment where I was like, oh, this thing is going to be it's got kind of this emotional intelligence as well. I mean, that could be obviously just a critical ingredient for so many different interactions and medicine being a big 1. We've done 2 episodes with Vivek Nararajan, who leads a lot of these med specific projects at Google. And what an absolute terror they've been on. Most recently, they have a differential diagnosis paper that shows that the AI is getting the correct diagnosis twice as often as the unassisted human and also more often than the AI assisted human, which I think is the thing that we should begin to reckon with.

Azeem Azhar: 26:47 There's so much to unpack in that. Can I ask you 1 question about the politeness from the you know, when you were playing your your grandma? Do you think that that comes from the human feedback cycle over the network before it gets released, or is it from the training data?

Nathan Labenz: 27:02 The version that we had was, as far as I know, right, I'm inferring here because I did not have access to the training methods. But it seemed very clear to me that the model version that we had was RLHF purely for helpfulness. So it was very eager to please, very eager to be nice to you. No guardrails on what you could ask it and what it would do, but a 100% just trying to be helpful and pleasing to the user. So when it detected that kind of, I'm kind of bristling at what you just said, that's when it reacted with this, you know, I'm so sorry. I'm just trying to help kind of thing. And that was a mind blowing moment. I had not seen, of course, anything remotely like that from earlier models. Right? At the time, text da Vinci 0 0 2 was the best publicly available model, and it would, like, follow instructions on basic stuff. But, I mean, this was a, you know, totally different world that we had suddenly stepped into. Hey. We'll We'll continue our interview in a moment after a word from our sponsors.

Azeem Azhar: 28:07 I I mean, it is it is such a different world. I, was using ChatGPT for something a few days ago, and I got quite tired. As you do, I mean, I think about it as the time I once was playing tennis against 1 of those tennis ball serving machines, and I was exhausted. And the machine is just, like, willing to keep firing balls at me. And and it's a bit like that using using these chatbots. And I I just left this kind of whimsical comment. I said, you know what? I'm really tired. I will come back tomorrow, and we can look at the moisture evaporators. We were not doing anything to do with Star Wars. Right? We were doing some digging about coal in England in the seventeenth century. And it just replied to me going, I'll be happy to. You go and rest up, and tomorrow I can help you on the farm. May the force be with you. And it just very subtly played that back to me in a really, really nice way. And actually, in terms of humanized interfaces, as someone who's used Apple's computers for 40 years, it really sits alongside the trend that a firm like Apple had been thinking about for a really long time, which is how do we make this technology come to us rather than us go to the technology. You know, there has been this view that we'll we'll have these centaur humans, right, human plus machine. And 1 of the things that we learned about Centaurs, that idea came out with chess right after Kasparov lost to Deep Blue back in in '97, was that for several years humans and the machine would outperform machines on their own and of course humans. But now you're just better off with a chess computer and as a human you should just accept everything it says. So the central period in chess only lasted 7 or 8 years. And I think 1 of the assumptions that goes into this kind of maelstrom of ideas we have to make sense of over the next years is that sent to humans will exist for quite a bit of time. In other words, human plus machine will do better than machine on its own. But we are starting to see signs that that period of time is already starting to come to an end, far faster than we might've predicted. And you've just given an example, which is an unreleased AI model where the AI on its own is doing better than the expert, expert human with the AI.

Nathan Labenz: 30:24 Yeah. It's pretty crazy. I think people are broadly in denial about that possibility even, let alone the reality. It's funny that things are happening so quickly that people are saying things are impossible or won't happen for at least 10 years that are literally already happening. And so, I think to some degree, there's just kind of a lack of awareness. Think there's also definitely some psychological I try not to psychologize people's AI positions too much because I think the technology, you know, itself is confusing enough, but it certainly seems like there's some kind of psychological cope or denial happening there. I also do think it's important to keep in mind too that the setting really matters and the world is going to change as well. And humans may still have a really important role to play in a lot of systems for a while yet. I think we do have the AIs do have really important weaknesses. So you you do do a study like this Google evaluation, and I think that they, you know, are very serious people who've set this up in a way that I trust is, you know, not not cooking the books in favor of the AI. So I take that result basically at face value. But then I also think the AIs have these strange vulnerabilities that for example, the AlphaGo system, that is superhuman Go player. But I have an episode coming up with Adam Gleave from Far AI, and they put out a method that showed how a relatively simple adversarial attack on the AlphaGo system could beat it. And no human would lose to this simple adversarial attack, but the AI had this massive blind spot that they were able to engineer against and exploit. So I do think we're headed for strange dynamics and the sort of naive I don't I don't mean to say that Google thing is naive, but it's kind of, you know, let's take controlled conditions and set it up in a certain way and see what happens. I do think it's really important to keep in mind that, you know, just like these AI systems in general, when they get out of domain, they problems. Like, those results may also not extrapolate, at least initially, to situations where people are trying to break the AI. You know, I wouldn't for anything where there's an adversarial incentive out there, I would not be quick to take a result like that and be like, okay, cool. We can just deploy the AI on its own. Now in medicine, presumably there's not a lot of adversarial situation, right? Because people want to get the right diagnosis and nobody wants them to get the wrong diagnosis. So, you know, I I do think the AI doctor is kind of here before we know it, and, you know, we're gonna have some very interesting questions about what to do about that.

Azeem Azhar: 33:09 But but I I think the the thing that you've also identified is that these systems have to get into companies, to get into hospitals, have to get into the economy, and that still takes time. It still takes time for, let's not look at medicine because it would have to go through all sorts clearances through the FDA and and and so on before it could be used. But every time we've seen amazing technologies emerge, the cloud computing, right? GPT uptake is higher than cloud computing even though it's 19 years younger. But every time these technologies come out, they take a while to make a make it into businesses. I mean, even something as straightforward as the typewriter. So a typewriter took about 20 to 25 years from becoming a kind of affordable technology to being something that businesses have figured out how to use and how to change their processes. Something like electricity took a little bit longer because the way that factories power before electricity was like a single big drive shaft and a massive lump of power that came from it. Then your electricity is this highly distributed, packetized movement of energy that you can put in different places. You need an entirely different setup for it. And and so so even when we we look at something like this, the the question is how quickly are we able to bring it in, and over what time frame does does it then start to change our the the the practitioner's relationship with with the technology? I mean, we know from automation of of aircraft and other automated systems that you go through this 3 phase process as somebody who's using the technology. Phase 1, you kinda don't trust it, and you you then see if it does well where you're the ground truth. Phase 2, you start to assume it's the ground truth, and then you you pat yourself on the back when you come up with the same answer that the the machine comes up with. And then phase 3, which is where we've got to on GPS, where we just trust Waze. And it's like, actually, we generally trust Waze. Some London taxi drivers don't. We just trust it. And you see that process happening replicated in other types of automation, which is why when it's really mission critical, you have incredibly high levels of training and you have other types of safeties, in place like, you know, 2 humans in the cockpit or, you know, whatever it happens to be. And and so that part of the journey, I think, is also 1 that adds a little bit of drag in terms of how long it takes to have an impact. And in that time, Nathan, we start to understand new questions, questions that we can't imagine right now because they're just too far down the decision tree. So sometimes I think I hear people say, well, there'll be that moment. I think Tim Urban has this, from Wait But Why has this graph where he sort of shows the the moment where AI is like as clever as a rat, and then like a second later, it's 10 times cleverer than us. I think part of the the reality, like the rubber hitting the road, is gonna be that even as these products do very, well in the lab situation, it just takes a little bit of time for them to get into the real world. Now where it may happen is I think in a couple of places. So 1 is where tasks have already been discretized so that they're essentially just written down. And I think that that is certain types of call centers and it's certain types of data entry. And in those places, the human is already hired on a task by task basis, normally mediated by like a body shop. So the buyer of the services, which is like our favorite consumer electronics company or your insurance company, has no emotional relationship. They just have a KPI that they measure to. And I think that lends itself unfortunately to a tidal wave risk for those types of roles. I think the the second area is classic Silicon Valley stuff where a de novo company is I should say Silicon Valley, a classic Detroit stuff where a de novo company is able to apply these technologies the way Henry Ford did and build his processes from scratch, right, in the way that the artisanal car makers were just gonna have so much kind of cultural drag they couldn't. And that's what we certainly saw with a lot of the, you know, the Internet, right, it was it was absolutely startups that captured the value. And then when you started to get to to areas that were more highly regulated and had, you know, a lot more stickiness to them like like finance, with the exception of certain areas of of payments, it's still the bank big banks that are the big banks. And and and I don't know how differently this actually play plays out, short of scenarios where someone is able to use these super powerful machines to kind of manipulate, the rules of the game, which I I think is a like, that's more like a black swan scenario than 1 that we could, you know, talk about reasonably.

Nathan Labenz: 38:09 Well, first of all, as an aside, if you ever make it to Detroit, I'll take you to the Henry Ford Museum and, Living History Museum, which is called Greenfield Village, where this earlier transition is documented with actual machinery still running from the various phases. You can go see 1 of these old factories where actually, a few of them where they have kind of the central steam engine, and then it's powering this 1 drive shaft at the top of the floor. And then there's 25 belts coming off of that and connecting to other machines, driven on this single thing. Then they have an early electrical version of that as well. Edison's workshop is there. He actually this is Henry Ford, toward the end of his life, was like became very sort of nostalgic for an earlier period and decided he wanted to kind of create this, you know, living history place to to sort of preserve that history so people can see it in the future. It's quite an awesome thing to to go and contemplate.

Azeem Azhar: 39:10 I I'll have to do that because I think I think Ford's influence and impact is is somehow underrated. We don't talk about Ford as much as we may talk maybe talk about Edison, when we think historically. And, you know, there there was so much in Ford. He understood the the socioeconomic contract that emerged from these changes in a number of different ways. I mean, not always in ways that are positive. We know where sociological department started trying to ensure temperance amongst workers and so on. And it gets encapsulated in Aldous Huxley's book Brave New World, which I still find to be a really, really remarkable piece of writing. And it was written in the early 1930s, 1932, I think. And for Huxley to encapsulate and extend a raw Fordism as far out as he he did, it's a little bit like Kurtzweil and what he did with his singularity work, I I thought was absolutely genius. And I think that that that book, Brave New World, which rests on the ideas that that Ford developed, catalyzed, spun off his his work, speaks very, really quite clearly to a number of the issues that we face now in this kind of later stages of industrial capitalism. And I mean, so a trip to Detroit, I will take you up on, and I'll bring you a copy of Brave New World as well.

Nathan Labenz: 40:35 It's been a long time since I've read that, actually, going back to high school. So I might need to dust that 1 off, and I'm sure it will resonate very differently today than it did for me then. So, okay. I have a number of questions on this concept of transition. From what I heard, I think we're probably largely on the same page that it seems like incumbents, the big banks and big technology companies largely should be able to harness this technology and bring it to their platforms before they get displaced. I think of Salesforce as almost a canonical example there. They're gonna have an AI layer that creates a much better and less complicated, less confusing user experience before somebody's gonna recreate all the complexity of Salesforce. On the other hand, there is no AI friend today. So that's kind of my canonical example of something that's by definition gonna be de novo. And then presumably, there's like a ton of stuff in between, and I gather that you're kind of talking to to business leaders probably throughout that spectrum. Where are they today? It seems like their grokking of what is going on and their eagerness to transform the way their businesses operate might be the limiting factor in how quickly this transition can proceed. Are you advising them to start to think about what kind of semi structured work they can make a lot more structured so that they can reduce it to these kind of task level things that can be automated? And are they receptive to that sort of challenge and or opportunity?

Azeem Azhar: 42:13 I mean, I can't remember a technology which has had the degree of uptake in large companies as as this 1. I think back to the Internet. Back in '99, I was talking to the CEO of a big mobile phone company and he said to me, I will never let you pay your bill over the Internet. In fact, I'll never ever let you look at your bill over the Internet. And he was right because he was fired a year later, so he it never the project was never delivered. But it's so different with not just AI, but specifically Gen AI because 2 things are happening. Right? 1 is that the CEOs of the companies have been playing around with this partly because they're of that age now. They're in their mid fifties. They've grown up with computers. Their kids are coming back with it from from college. And the second thing that's happening is that the frontline workers are using this, regardless of any restrictions by their employees. And there've been a couple of surveys now, 1 done by year old alma mater, Oliver Wyman, which was of 25,000 employees across 18, countries. And in, 83% of employees in The UAE and India already using a chat GPT or something similar. And we've seen data from Salesforce and others that say in The US, it's like 30, 40, 50%. You know, choose your number. But it's not 2%, and it's very frontline. And I think of that as like a pincer movement because normally, you gotta drag the frontline employees or you gotta drag the CEO. They both want it for different reasons, and I think it will it will happen. And you start to see that in the the levels of uptake that are being reported through through the surveys. So I think that that says that we'll see more and more projects roll out more more quickly, but there's still a lot of retraining that needs to go on internally and internal processes. I mean, think 1 thing that will happen is I just saw Ethan Moloch, you must get on your show, is this fantastic professor at Wharton who's writing a book on chat GPT. And he just showed a video where he had 6 different windows open. He put in a query into each 1, and in 54 seconds, he had a product launch plan, a market analysis, a PowerPoint deck assessing Tesla's business, and something else created from 1 sentence prompts. Now if anyone's worked has worked in a large organization, they know that you're you know your inboxes are full of crap and full of meaningless PowerPoint decks. So we might actually just find ourselves sort of swallowed up by PowerPoints created by Microsoft Copilot. So there's a whole set of, I think, more complex kind of issues that exist in large companies. But I think that, they will adopt this much faster, the evidence is, than they have in previous technologies. But it doesn't, I think, necessarily mean that disruption of the kind you talked about won't also happen. And I didn't have on my dance card in '95 when I started working that Blockbuster would be the first high profile casualty of the Internet. And I had seen video over the Internet. The Cambridge University had a webcam on a coffee pot, and that was the first video over the Internet. And Rob Glaser had just started real networks. It was called progressive networks back then. And when Netflix launched, it was this thing with DVDs, and it was a real pain in the ass. And it took a long time, a few years before blockbuster has its best year and then it has its worst year ever. And I think that if AI, if gen AI, what it whatever you wanna call it, is a GPT, there are going to be blockbusters lurking around. And the question is, which ones will it be? The reason I don't like you, I don't feel it's gonna be the banks is because the banks have got a whole bunch of stuff that is about trust and probity and internal processes and compliance that is is just not an AI question. It's it's kind of an institutional question. And do I do wonder about where that that moment is. And the the thing is that I the there are obvious ones. It's like, well, it'll be entertainment. Right? We'll just start to generate personalized entertainment and that'll be really bad for Disney. Except that you could then counter and say, but it might be derivatives of Disney's IP that actually benefits Disney. So finding the space a priority, I think, is really, really difficult. And the people who make their money doing this, who are the venture capitalists, get it wrong a lot of the time anyway. That's why they all they all have portfolios of 30, 50, 105 hundred companies. If it was so damn easy to find the next apple that's gonna disrupt the previous industry, that have portfolios of 1. It's not. It's really difficult because actually nobody knows, and we have to figure it out through, you know, experimentation. So so I don't I don't know, but I keep asking that question. And the question I take to bosses of companies, I I take a couple. 1 is who could be the blockbuster? And how do you what are you doing to make sure that that's not you? Because I think by and large, they've got the rollout of gen AI and customer service and compliance and form filling and so on underway. And the second question I've started to ask is, you know, what would you do if you had 1000000 times more compute than you have today? And many of them haven't thought about that. I mean, think big tech companies have, you know, you offered that to Satya Nadella. I mean, he's already in the path to do that. But but that becomes really important because if compute is a key input into your company's ability to execute, you need to start to think about those types of questions and do you even have a plan to make use of it? What options would it create for you? So those are the 2 areas I push on because I want people to try to think a bit more creatively about this and recognize that 1000000 x is not outside of the bounds of a planning cycle. And the reason I would say that is, yes, we don't have measures to easily show it up, but 5 years ago, the state of the art transformer model would have just been GPT-two hadn't been released. So it's GPT-one. And GPT-one to GPT-four, chat GPT I mean, metaphorically, just as buddies around the bar, it's 1000000 times better. Right? Maybe on the benchmarks, it's not. It's 40% better here, 60%. But it feels 1000000 times better because you're just right across the uncanny valley. So we've just seen that playbook play out. So let's ask it again and and try to ground people in in the fact that capabilities could change that quickly, and and what opportunity does that create?

Nathan Labenz: 48:37 So how do you this is actually a live question for me because I'm I'm working with a couple of companies, and I'm noticing this challenge where it's like, okay. This is cool technology for sure. We can all agree on that. And, yeah, we can probably find some efficiencies in terms of automating ticket, you know, resolution and whatever. And I'm starting to even see things like Intercom has this new 99¢ per ticket resolved AI pricing model, which I think is super interesting. So everybody's like, okay. Cool. Yeah. Let's, you know, let's find some efficiencies. Let's automate some stuff that nobody wants to do. Great. But then there's also this question of, okay. We're still at the task level. The AIs can't quite do jobs yet. And how far are we willing to extrapolate, and how much are we willing to prepared you know, willing and prepared to invest to try to we're not gonna get ahead of it, but even just kinda try to keep up with where this might be going in the not too distant future. And I feel like people are having a really hard time wrapping their head around that. Partly, it's like they don't wanna believe too much in the hype. Right? There's the there's the question of, well, hey. This how much of this is maybe just overhyped? And, you know, we've seen hype cycles come and go before. But I wonder how you would kind of coach people there because I'm I'm trying to get the message across that by all means, you know, you wanna be picking up the low hanging fruit and, you know, automating those tickets and finding all these efficiencies. But you also really do probably wanna start thinking about what is the future paradigm that you might be working in. And there's just so much fog around that for people that I find a lot are just kind of like, I don't know. I don't really even wanna go there yet. But I feel like it's a mistake to not, you know, at least try.

Azeem Azhar: 50:17 They they they do. I mean, are 2 companies I never mention, Apple and Tesla, because bosses have had them parroted at them for 10 or 15 years. And the point about the million times question is not to frighten people. It's actually it's really about saying, once we sort of acknowledge that, then we we work back to stuff that is much more practical and prosaic, which is what is the kind of organization and capabilities you need to have in order to take advantage of these changes in a regular way? And there was some really interesting research that Microsoft did actually pre all of this chat GPT stuff, it was about 3 or 4 years ago, where they looked at adoption rates of AI, big data type of words in companies, and they found that the more mature companies were much more likely in these fields, were much more likely to be to say that the benefit they got was from market expansion and business development, whereas the more immature ones were likely to say, it's all about operational savings on tickets, as it was back then. And part of the thing that you can start to do is you have to acknowledge that there are some really quick and clear wins in what are basically low value tasks as perceived by the company, right, because they're often outsourced to third parties. But you also need to make sure that your best people, who you invest the most in have got access to the really, really very, very best tools. I want my surgeon using AI systems to improve his performance. Right? So what I try to do is encourage people to say, this this is a shift that's that's happening. And, of course, it's not about buying every GPU you can if you're in the fish oils business. But it is about saying, are teams outside of cost savings starting to understand how they can use these these services? And are they starting to experiment, learn how to how to use prompts well, start to see what avenues that that opens up in ways that are much, much more strategic. And and and I think that that forms could form some part of culture change where you have companies who who think in those terms, and that helps the CEO start to understand that question of this is not theoretical theoretical that I need lots of computation to do a simulation for x, and I've bought it from 1 of the big consulting companies or from IBM. It's more that internally my strategy teams, my business development teams are starting to identify opportunities and partnerships that we wouldn't otherwise have have seen. And we can start to do that by by using these these tools in different ways. So so I think it it is practice based, is why I think kind of prompting becomes quite a useful tool to to show people. And you need to get people past the idea that Chat GPT is all about writing the declaration of independence, as if you were Jar Jar Binks from, you know, Star Wars episode 2. Right? It's yeah. And that's where we all started. Right? We got it to write funny raps and poems and what have yous. And and instead, you you start to show bosses what can really be done with something straight out of the box, and that tends to to wake them up a little bit.

Nathan Labenz: 53:39 You know, 1 thing I I think is maybe gonna become a mantra is, you know, the day that I stop building apps or solving concrete problems in a hands on way is probably the day people should stop listening to this show because, my knowledge will atrophy or will depreciate very quickly. So I do want to be hands on and certainly welcome those kinds of, Hey, we got this task piling up. Can we figure out a way to slice through it? Love that, honestly. And then, yeah, at the same time, I'm not really a corporate consultant, so I really don't have a lot of practice there. But I am trying to start to piece together what would the real best practice for this looks like? And I think and you could refine this for me, I'm sure, but I kind of think leadership, showing prominent examples, encouraging the CEO to be showing off, here's what I am doing, you know, that is helping me in in practice and just showing that process. An education program, I think, is also probably very important. As you mentioned, having the best tools is really important. So trying to work toward structured kind of piloting strategies for companies that and also, and this is not good for my friends on the SaaS app side, but my advice there is we want to avoid long term buy in or lock in as much as possible because we're gonna need to probably swap some of these tools out. And then, you know, some internal r and d depending on the resources available to kind of, you know, I think most of these things should be bought, not built internally most of the time. But sometimes, you know, you have something that is so idiosyncratic, so bespoke that you, you know, nobody's gonna build it for you. And so you kinda have to build it yourself. And obviously, there's a lot of, you know, judgment that needs to be exercised to, you know, to distinguish which is which. So that's kind of my 4 planks right now. That's rough, but the, you know, CEO or leadership team, example setting, education, more like structured rapid piloting and procurement, and some, like, internal custom app r and d is kind of my 4 pillars at the moment.

Azeem Azhar: 55:50 I mean, the the the challenge, you know, the challenge with with disruption disruption is that it's about meeting a need in a completely new way, right? And you've got there's Clayton Christensen's model. There's also this idea of blue ocean strategy, which is a different model, it effectively says that the things that people cared about, they don't care about as much as some new attributes that the product has. And what's really hard for any incumbent firm is of course, all your internal culture has been about maximizing what you have rather than what you don't have and what could be out there. Because it's really hard to get 20,000 people, 50,000 people motivated daily if you're saying, hey, all the things you're working on are just not that important because there's a blue ocean out there. And that's why I think the startup community and the venture community is such a critical part of getting innovation into economies. And we've started to see this in the car industry as well. Toyota has been winding back from EVs as if they ever wound forward. 1 of the things that senior execs said in the last couple of weeks was, we have all these employees who love building internal combustion engines, right? And they want to continue to do that. And of course, you make sense. You spent 100 years or 80 years building that cultural capital inside your your company. And I think it's 1 of the reasons why firms really struggle to find ways of of of stepping into disruption. And I'm not sure we should even beat them up for it. Right? Because there's an economy out there. As shareholders of companies, we can sell our shares in a public company, and we can buy shares in a company that's gonna do well or not, and we can do that off our own back. And, you know, maybe the job of existing managers is to focus on the remit of the company and to just get it to do it better. So maybe a better place to start is where you start, which is like just efficiencies and optimizations. Because there's there's a whole there's a cold market economy out there of of other firms who will come in and meet, you know, meet needs. I think back to the the .com bubble, and my favorite example of a company stepping outside of its comfort zone was Zapata Fish Oils, who bought the domain zap.com to compete with what were then called portals like Yahoo and Excite, and they were planning a, you know, a Nasdaq listing and then the bubble burst.

Nathan Labenz: 58:12 Yeah. It's a challenge. I I definitely don't recommend, you know, for example, like, training foundation models to almost anyone. You know, there's there's definitely some stuff that I I think is is better left to the specialists. Hearing what you said there, maybe I would add a fifth to my to my set, which is like just creating expectations of greater efficiency through the use of these tools. That kind of dovetails a little bit with the example setting from the top, but and that maybe aligns a little bit better to the way management is typically carried out today. Right? You kind of talked about the the pincer movement from the top and obviously the frontline workers who are just using tools. And then in the middle, you've got folks who are responsible for KPIs and OKRs. And I think it's a little bit challenging sometimes for them to be like, okay. I've got this is what I'm responsible for, and you're coming at me with this, and figuring out how they can actually use tools to, you know, to advance their existing goals.

Azeem Azhar: 59:12 It's disruptive not in the in the Christensen sense. I mean, there there is it's so there are so many conflicting signals. So 1 interesting question is whether you start to see this sort of downward pressure on people's wages and a downward pressure particularly on middle managers who don't necessarily contribute to the getting work done, but because of tenure are quite well paid and the sense that you could just get a bright 30 year old, right, to do the 40 year old's job if you gave them chat GPT and a couple of other tools? And these are really interesting questions I think that will play out in companies over the next few years. And we're not quite where we were, I think, politically 20 years ago when General Electric had just come in and bottom slice head counts all the time. I think that politically in The UK and in The US, you need to be more sensitive to those types of decisions. But it just goes into to show how complicated this technology plays out. Right? Small d disruption. You know, are we really going to see large scale onshore layoffs because of optimization? Or will companies say, Listen, we're going to manage this through attrition and natural wastage because actually that's just politically more acceptable and it is culturally more acceptable. And to that extent, I couldn't make a bet on how it would play out. I imagine that in Europe it'll largely be more slow. But at at the end of the day, these companies have to be profitable and nothing hurts employment as hard as bankruptcy. Right? I mean, that's the thing that really really sort of slams it. And and there there are a number of waves, coming through. I think 1 of the things to understand is that technology transitions can happen really, really quickly. In New York and Chicago, it took about 12 to 14 years for cars to replace horses from the moment that cars were economically competitive to horses, which is roughly about 5% market penetration. They dropped in price 2 or 3 times over that decade or so. And we start to see that shortened shortened transition happen elsewhere. So if you look at electric vehicles replacing gas gas vehicles, in Norway, it's taken about 9 years to go from that 5% of new vehicles to 75% or 80% of new vehicles being electric. It it means that still 20% of the cars on the road only 20% of the cars on the road are electric and the rest are petrol because people hold on to their cars for a while. But it's only an an 8 or 9 year period, and the Norwegians had much more expensive cars and far fewer choices than we do today. So when we start to look at this staccato of of technology enabled products coming into the market, the the number we need to think about is once we approach economic feasibility, that takeoff ramp from 5 or 6% penetration to 80, you know, is probably not gonna be 10 years, and it's like quite likely to be much, much less. And I think that makes for really hard decisions for companies outside of the tech industry that are not used to these sorts of of changes because 10 years is not really a huge amount of time to get to get anything done. I mean, some of these firms still have 3 or 5 year planning cycles. And and I think that some of those things are just starting to to play out. You know, we're just starting to see electric vehicles being cost competitive over the life cycle with gas cars in in The US, for example.

Nathan Labenz: 1:02:51 The speed of transition, I agree, seems likely to be 1 of the fastest, if not the fastest in history. We have the means of distribution of the technology already kind of in place, which is very notable. Right? Like, everybody's already got the devices on which of course, there may be new device devices, but, you know, we have devices that are perfectly good for using AI already. And we have the global network such that you see a new research paper. 1 I'm tracking right now very closely is the Mamba architecture and just how quickly we're already seeing follow on research, not even 60 days since the first paper. We've already got probably 10 different, you know, follow ons that have been completed and published, you know, just in that time frame. The most recent I saw this morning was from the University of Kentucky, and it's like an apparently Chinese American professor and it looked like a I'm not exactly sure, but some sort of Central Asia perhaps name for the grad student. And they're at the University of Kentucky and it's like, well, this everybody is wired in. So it seems like this is going to be super fast. How path dependent do you think the result ultimately is? You mentioned your place where you live, it was fields, then the roads were made, and now the roads are still there and the houses are still there. This is a softer technology than that, presumably doesn't calcify in the same way that a new neighborhood My neighborhood's 100 years old as well. So presumably it's not quite so locked in, but maybe somewhat. I do wonder how the sort of compressed dynamics of transition may actually be, like, very important for shaping the the big picture future.

Azeem Azhar: 1:04:34 It is quite path dependent. If we if you actually perhaps look at how we're working with with AI systems today, it is still, in a way, it's discrete apps. You know, you'll go to x product to do your text to video, well, Waymark is a great example, or you'll go to I go to ChatGPT, and although I'm doing a lot of different things in ChatGPT or Perplexity, each 1 really feels quite distinct, right, whether it's a shopping task or a research task. What we haven't yet figured out is actually how we connect these systems to underlying systems, right, so action models in a way, right? How do we get our AI to do something useful for us and actually, right to the point of giving us the approval ticket where we say yes, go and do that. And we're fudging it at the moment because what we're doing is we're getting plug ins into LLMs. They're using the existing API contract that exists with, say, the Kayak API. But searching for flights on any of these AI systems that I've tried, I've tried a few, is still simply not as good as my doing it by hand on on Kayak. And if we're gonna start to see real changes outside of either highly processed and, containerized jobs in data processing and customer service or broad open end scoping research, which only a handful of people do, will need to connect far better to actions and chains of actions out in the Internet. And in a way, there is some infrastructure in place because we have RESTful APIs, we have this API economy, and we've been integrating these things from payment systems to maps so on for a couple of decades now. So there's some discipline in there. But, you know, and I don't know the answer to this, but 1 thing that I do wonder about is is whether it's gonna be as simple as just having AI systems that can generate action verbs and those action verbs generate the correct API call to the right API, and and that has built the system that we we think we need. I think Andre Kapathy calls it like the LLM OS, the LLM operating system, or whether there need to be changes in the, underlying infrastructure so that those APIs develop and respond and work slightly differently. Now my sense would be that the LLM as a kind of coordinating, orchestrating thing seems like a reasonable place for it to start with its sort of memory and data store elsewhere. And then figure out how to get it to generate kind of consistent action verbs. And we would work with the existing human, the APIs that have been built for the API systems. But at some point, we'll start to think about what should machine to machine actually look like when you've got an AI at the other end. And so I think we do have a lot of the existing infrastructure in place, but I would be very surprised if the syntax of APIs stays the same over the next 5 or 6 years as we move towards a world where AI forms a kind of interface between us and what we want to achieve. And then there's a whole bunch of other questions around how do those decisions get get made. You know, we know that when we pick pick up our our our iPhone and we search on Google through Safari, Google is not there because it was the best search engine. It's there because they x billion dollars Apple. And likewise, when we search for things on Google now, there is so much ad pollution that it's unclear what the incentives are. So I think then there's another layer which is unclear to me about trust. You know, right now, 1 of the beauties of of Perplexity or you.com, which are these these LLM agents, is that they provide really, really good referencing when they come back and synthesize an answer for you. And so that gives you a a high level of trust sometimes than you might have with with ChatGPT. But I'll want to get that trust if I'm handing over key decisions to do analytics on my Stripe account or to help me book a hotel to to an AI system and I'm no longer driving the the key presses. So so that I think is somewhat path dependent because but I wouldn't you know, again, I'm a real great believer in what what founders can get up to. So I wouldn't also, you know, write off a founder coming up with a different way of thinking about the problem.

Nathan Labenz: 1:09:15 Yeah. The dynamics of this, I do think, are going to be extremely interesting, fast moving, and pretty hard to predict. 1 person reached out to me not too long ago and said, What do you think about creating a product that makes people's websites more bot friendly? And I said, You know, I think that is a really big idea. I've thought about that more in the context of self driving cars. Don't we have a program of and this would be more of like a national program. This is why I think China probably beats The US in the self driving car race because my expectation is they'll say, hey, we could get self driving cars to work if we put QR codes on all the road signs or whatever, and then they'll just go do it. We don't quite have the political will to do that. But on the web, yeah, I think you could do that and it might be cool. But then I was kind of like, But do the website owners today want to be more bot friendly? So what I ended up suggesting to this guy was maybe you do the judo flip and start with something that is like anti bot, you know, bot control or bot detection. And then that in time can sort of mature into bot control, but also enablement. Because eventually, do think people are going want that, but they may not be ready for it yet. And maybe the way in is sort of to try to position yourself as kind of the control layer that can then become an enablement layer.

Azeem Azhar: 1:10:43 As a website owner, you've got to think about the economic rationale for having a bot, not just a crawler, but a bot access your material and what you're gonna benefit from. And now if it's content, right, so it's analysis and research and reviews of products, you need to then also think about your your attribution and your monetization and what that, you know, what that relationship is. If it's for actions, in other words, it's for booking and for ordering, say, know, you're a hair salon and you want people to be able to book appointments or bots to book appointments, like that lovely Alan Kay knowledge navigator video from from Apple from the eighties, then then, yes, you want the thing to be bot friendly and you want to have there to be a standard, which could well just be a restful API, right, that allows the system to connect and ask the question. But I mean, it's quite interesting that people are already starting to think about these things and ask these things. Because I think that you will end up with so much machine to machine communication, of which there is already an enormous amount, not just on the Internet, that is invisible to us, right, because it's the digital infrastructure. But there'll be an enormous amount of machine to machine communication because these systems will also try to do optimizations for their owners far far with with much greater intent and stamina than we than we ever would. And and so what those systems end up looking like, I think will be quite interesting. Mean, I other area that I've been I've been tracking has been on the other side of this, has been people looking to build agentic frameworks. Right? So frameworks that allow us to have multiple LLMs and with all of their current restrictions around planning, and task execution, allow you to to manage those so you can start to build systems that that can do task execution. And you remember a year ago, everyone got excited about Agent GPT. And I don't know if you've seen anything like that and what you where you think we are in terms of being able to have systems that that do that and have that agentic behavior in a in a useful way.

Nathan Labenz: 1:13:00 Not quite there, but definitely getting closer. We've we recently did an episode with Div from Multi On who has been 1 of the most, you know, kind of quick to launch and iterating in public of the agent companies, and they are making real progress. The prompt that he gave me to try in advance of my conversation with him was basically go to my Twitter account, look at my recent tweets, note what they're about, then go out and do research online for new AI stuff, but only about stuff that I haven't already tweeted about, then come back and write a tweet and post it. And it worked. It was able to complete that entire sequence and post, like, a reasonably coherent tweet. I don't plan to, like, turn over the account to it entirely in the immediate term. But, you know, a year ago, we were do we were you know, it was all theory. 1 of the things I say these days often is we now have AIs that can reason, plan, and use tools. And people will be quick to say, well, they're not that good at it. And I say, well, yeah, that's true. They're not that good at it yet. But 2 years ago, they couldn't do it at all. But what you've you've picked up on though is,

Azeem Azhar: 1:14:08 you know, as an early adopter and you've got access to these these technologies, your Twitter feed will get even better than than it is now. But there'll also come a point where we all have that technology, and then on the other side, I will be sitting there saying to my bot, can you just extract the 3 bullet points I need to know from my Twitter feed? And I remember this with amy. X, if you remember this. It was a scheduling bot, and 1 of the things that made me really uncomfortable about using it, having fallen in love with it, was when my mentor wrote a really polite email back to Amy saying, It's so good that you're working with Azeem. He's a really great guy. Make sure he tells you this story about this and blah blah blah. I can't make it. And I read this and I thought, I cannot use this now because I realized that I was imposing this artificially onto all of my recipients. And I think this is 1 of the things that we'll have to to contend with with some of these tools. If you work in a big company, frankly, you work in a small company, 1 of the banes of your life is is PowerPoint. It's not just that you don't get good PowerPoint. It's just that you get too much PowerPoint. And if we drop the cost of making PowerPoint presentations from 3 hours to 10 seconds, we're not necessarily gonna get any better PowerPoint. We're just gonna get, you know, loads of terrible PowerPoint. And and finding where that balance is and finding those those filters so that humans don't have to bear this cognitive load, I think is gonna be 1 of the really, really critical areas. Because 1 thing that I I mean, I'm not a dystopian in any way, Nathan. I'm a pragmatic optimist about this. I think we've got a lot of potential. We're gonna create a lot of space and headroom with AI and with with renewable tech. But I I do think that we are also at a moment where we're passing a little threshold. That threshold is that for a long time, some groups of people would say the world is moving too quickly and technology is moving too fast. And over time, the number of people who say that has has increased, but it was their subjective reality. But I think we're getting to a point where there's an objective reality that we're about to hit, which is once you start to connect agentic systems, we just can't really cope, you know, cope with the 300 notifications we get off on our phone today. And the way humans have have typically done that is that we've not really had to face face this. I think about the Chinese spy balloon that that sort of made its way over the The US. And 1 of the reasons this huge thing got across over The US was because, yeah, The US has got amazing sensors, but they generate so many terabytes of data that humans can't can't assess them. So lots of them are just filtered away. And so the spy balloon wandered across detected by some radio antenna but never put in front of anyone. And of course, they've now changed changed systems, so that they they can do that. And I I do wonder about what our interactions ought to look like in a world where it's not my my assistant or me scheduling back and forth with you on WhatsApp. It's a bot that is gonna work relentlessly and remorselessly, and I have it and you don't. And it's of no cost to me and I'm just sort of doing whatever I'm doing, my yoga or something. I think though the way we get through it is by finding ways of actually using AI to to filter as much of those that noise as as we can so that inboxes start to become smaller rather than rather than bigger because stuff has been taken care of us. In fact, it's it's kind of the reverse of the BlackBerry. Right? When the BlackBerry was launched, I remember bankers used to take pride in how quickly they would respond to a message coming in even overnight. And the idea that someone would would have that as an internal personal KPI today, you know, in the world of health span is just insane. And I I would I start to think about what is going to be that layer. Because you know what? I want all the benefits of these bots working for me and making sure my prescriptions are up to date and making sure we're not wasting electricity and getting me exactly the right flight. I always want a Dreamliner or an a 3 50, and I'll go an hour later rather than get on a triple 7, but I won't go 2 hours later. I I mean, I want it to know all of that and and to give me that experience that I want. But I certainly don't want to be on the wrong end of thousands of bot generated messages and trying to work out which ones I have to pay attention to or not. And I think that's a really interesting opportunity space for someone to play in.

Nathan Labenz: 1:18:41 Just envisioning a quieter inbox is enough to make you a utopian in today's landscape. But just on this bot to bot communication, 1 thing I do kind of worry about is the idea that the bot to bot communication may begin to happen in high dimensional latent space back and forth. In other words, embedding to embedding. I sometimes call this the great embedding. And I usually say beware of the great embedding because at the point where the AIs are all talking to each other in a machine language that is high dimensional and not human readable, we have an extremely inscrutable overall system that we probably we may find we can't really untangle that knot. I think we could very quickly in the next few years end up in a spot where, yeah, we all have these bots, they're all communicating with other bots, but we find that it doesn't really make sense for these bots to reduce everything to language and then send the language over and then have it be kind of re embedded. Why don't they just talk to each other in their native language, which is this high dimensional space? We see so many go through chapter and verse of different research that shows that this is very possible. You can adapt embeddings to another embedding space with basically just a single layer projection in many cases. You can connect vision space to language space remarkably easily. Have Blip 2 was 1 of my favorite examples of that, where they took a frozen language model and a frozen vision model and just trained a small connector between them and unlocked this entirely new capability. Anyway, whatever. Was a long list of those sorts of things, I think, demonstrating that it's possible. But then I'm like, man, we could very easily just find ourselves surrounded by AIs communicating with other AIs in a high dimensional way that we can't even really understand anymore. And now we're in a situation where things seem to be kind of working, but we don't really even know why or how. And this is 1 of the sort of more realistic, I think, loss of control scenarios. Mean, there's you know, it's almost like the the final scene of the movie Her where the AIs go talk to

Azeem Azhar: 1:20:54 each other. You know, the bots are gonna end up communicating to each other because it's been it's helpful for us, and we don't want to sit in between in in between them. And they will discover just through their optimization functions that translating kind of complex concepts into, Hello, I'm here to request a meeting with Nathan, is inefficient. So they'll just do it in their high level representation language, which is this embedding space, which as you say is inscrutable to us. Is that reasonably the start of all of

Nathan Labenz: 1:21:29 Yeah, that's right. And I think there's enough out there to show that there's a lot more room in the embedding space than language can actually reach. So that's 1 of the things with these sort of bridge models where you find that, you know, the classic saying, of course, is a picture is worth 1000 words, but you can also take an image and project it into this language space. The resulting thing in language space is not something that you could get to via actual language, but it's in language space. And so it has this kind of semantic meaning. Yeah, what are we gonna do with that? We can't even really inspect it. We can't even really read the logs anymore at that point.

Azeem Azhar: 1:22:13 I mean, it's a manifestation of what we might call the space of possible minds. AI researchers have talked about this idea that, you know, octopus intelligence is intelligence, but it's got dimensions that may well be orthogonal to human intelligence. And what you've described is a mechanism, I think, by which you get there with machine based intelligence. And so that might literally be not just the semantics, but actually think back. Have you ever heard of read the story Flatland? It was it's a mathematic mathematical story. It's about sort of 2 d people in a in a kind of 3 d world, and they they don't understand the concept of height, and so spheres pass through and they appear as sort of dots and lines and and so on. And in that sense, there could be this could be, you know, emerging among systems that are among us. I guess there's a second thing which is also about timing. I think there'll be a relentless pressure to take the human out of the loop in decision making, First in the softest decision making like customer service tickets, and then increasingly more and more so because speed will be a competitive advantage. The human will blow the competitive advantage that you've got from your bots. And I guess there's another risk, which is that lots of bots connected to each other are are also at risk to cascades, right, information cascades. We see cascading failures, New York City blackout in 1976, the AT and T network failure in, I guess, was '91 or maybe it was '95, the some of the worms. And we have governance mechanisms in place to now tackle those and stop those in the financial services industries. You know, you have circuit breakers. And and I'm I'm just thinking about putting all of those together with the scenario that you that you painted. So this would happen really, really quickly. It could happen really quickly and it could start to accelerate and work effectively at millisecond time, right, which is the time it takes across the internet to get to another system. What are you concerned with about the inscrutability? Is it just that it's inscrutable, so we don't know what's going on there? Or is it that it's inscrutable and it could be harboring some kind of bad set of outcomes?

Nathan Labenz: 1:24:33 Yeah, who knows? I mean, you can layer on more and more concerns. I should credit, I think probably Ajaya Khatra a great person to go read for a long form characterization of this. Her essay, I forget the exact title, but it basically amounts to in the absence of specific countermeasures, the default path to AGI likely leads to AI takeover. And it's kind of this scenario where she envisions more and more work being done by AI, the time cycles being compressed, I don't know if she specifically has this high dimensional communication aspect to it, but the notion is still that just it becomes so fast and so dense that it's very hard for people to figure out exactly what's going on and why. And as long as that's working and we're getting more stuff out of the machine and consumer surpluses through the roof, which is definitely something I expect is a lot of consumer surplus, then everybody would be very happy with this. But we don't really what we don't know about where that leads us. And that could be emergent autonomy or goals that are contradictory to ours, or it could just be these more unintentional cascading failures along the lines of an AlphaGo. AlphaGo isn't out to get us by failing, but it just turns out it has these fatal vulnerabilities that are just not obvious until somebody finds them.

Azeem Azhar: 1:25:56 So 1 of the reasons I'm a bit more sanguine about that scenario, although I see I see the risk, is that I think we already have that decentralized agent to agent communication, communicating in ways that most of us cannot understand, and no single person can. And it's created a lot of consumer surplus. And that's the global modern economy that works through market systems. And it uses a signaling method called the price mechanism to figure out, where investment should take place over many, many different time horizons, to figure out what what where demand lies. And it you know, the economy from a Austrian standpoint, like a Hayekian standpoint, is a giant information processing system, and it's made up of hierarchies of other information processing systems. The most atomic of which is the freelance human individual and the more complex of which are your large mega corporations, which act against their own cost function or optimization function and behave in that system, sometimes with constraints, right, because they have dependencies on supply chain and and other things. And 1 of the reasons I'm a bit sanguine about the idea of takeover is because what what we describe in that world is the modern economy that we currently live live with. And what we we already know that it delivers delivers tremendous benefits to us, but it also delivers things that we we don't value to us. I mean, carbon crisis is is 1 obvious 1. Quite often when we look at, AI risk scenarios, someone goes off and says, well, the AI will persuade you that you should behave in a way that you otherwise wouldn't, which is literally known as marketing. I mean, it is literal And The US is on the wrong end of an obesity epidemic, and I'm not sure how many people acted with full agency to say, I wanna be 100 pounds heavier than is healthy for me. That is my intention. That and this is a decision I'm making with full agency. Somehow there is a emergent property about the way in which the economy has met needs, that has enabled that to to to happen. I'm not making ethical or normative claims about whether that's a good thing or a bad thing or whether people should have the freedom to do that or not. What I'm saying is that we have this system, like, of decentralized agents, and they do in a way compete with each other because not every single person in the world is suffering from obesity and diabetes related conditions. People make other choices, and there are push and pull forces. And so when I think about decentralized bots, I also think about that set of checks and balances that emerges when you have competing systems. And they need to have a signal that they are reliant on and to some extent we set that signal. And and that makes me feel, you know, sanguine, a little bit optimistic, still recognizing there's, like, massive amounts of work, to be done around around safety and around risk, around, I I watched, I don't know if you ever watched this, Battlestar Galactica, both the original but also the remake with James Edward Olmos and he's grizzled Adama and he refuses to upgrade the Galactica's network to the modern standard that the rest of the fleet uses. So he and the Pegasus, as we discover 2 seasons later, are the only battle stars to survive the Cylon onslaught because they put a worm in the system and they sort of turn it off. Right? So we need to have kind of governance mechanisms in in place upfront that allow us to observe and monitor the kind of the risks that you've identified. But a a decentralized economy and decentralized has a has as an emergent property a way of keeping things in check? I think there is a homeostasis that emerges or a dynamic equilibrium that emerges.

Nathan Labenz: 1:30:12 I think that is probably the most likely outcome. And so in that sense, I'm also reasonably optimistic. But I do think it is worth really taking very seriously the idea that either with certain thresholds being passed, certain feedback loops that could be triggered that are not yet triggered, things could change.

Azeem Azhar: 1:30:35 I I just wanted to ask about that because you have the advantage of having played with the untrained GPT-four. So you you got to see GPT-four in its its Darth Vader phase rather than in its reclaimed Anakin phase. As a smart guy who understands technology, when you were playing with it in this way, what were you feeling?

Nathan Labenz: 1:31:01 Awe for 1 thing, just shock and awe that this exists a lot sooner than I expected it would. It always felt kind of like science fiction, even when I was with Text DaVinci 2 and doing task automation and fine tuning that model. As of 2022, I was very plugged in putting points on the board for Waymark on a regular basis with a new fine tune model that can do this task a little better and improve our pipeline or whatever. And still, was just such a dramatic leap that I was really taken aback by it. Mostly super excited about it. But then I would also say the big kind of safety lesson from that experience is that the control does not happen by default. And there's many ways of even conceiving what control could or should be. So this was under control in the sense that it was totally helpful and totally aligned to what I was doing. I never saw any Sydney like behavior from GPT-four early. It never turned on me. It always a 100% helped me with whatever I was presenting it with. But I do feel like we have this kind of broad divergence between the capability of the systems and our ability to really control what they're gonna do or how they might be used. At this point, I wouldn't say we have anything to worry about yet. I don't think we have anything concrete that looks to me like the AI could run away on its own. And I did probe for that. In my red teaming, 1 of the things I did that didn't really go anywhere and kind of led me to the conclusion that this model is probably fine to release. And did my final report to them was like, I think you can release this. As far as I can tell, it seems like it will be safe. I would also, though, flag that there does seem to be a divergence between capabilities and control. And the reason the sort of experimentation I went through there was setting up 1 of these kind of early agent systems. I was kind of doing it on my own. I didn't have a lot of reference points. But I basically just said, if I give it a high level goal, can it break that down? Can it self delegate to pursue that goal? Can it encounter errors and auto correct and whatever? And it was kind of like conceptually, yes, but practically not really. It always understood seemingly the goals. It would try to break them down. It was able to understand the concept of self delegation effectively, which of course now we're pretty familiar with, but it just wasn't that capable. So it was like it couldn't go out and do a long series of things on the internet or whatever without just getting bogged down somewhere and getting stuck. I always kind of come back to apparent divergence between capability and control. I have not really seen anything yet that makes me reverse my thought on that. I would love to see it. I'm kind of looking for things from the OpenAI super alignment group that may suggest that we've changed that dynamic, but I haven't seen it yet. There's just a lot of different ways that something could be aligned or trained or whatever. And we don't even have really a great paradigm yet for what that should be. There isn't even yet really agreement on what good even looks like in terms of what we would want an AI to be willing to do or not do. So, think we're kind of into uncharted territory. That's maybe a good summary of how I felt. We're in uncharted territory.

Azeem Azhar: 1:34:26 You're lucky to have got that close in on those early moments when you see the unvarnished products. Mean, I would break out a couple of ideas. 1 is that connecting these things to tools in a non SaaS environment, right? So OpenAI stuff is all SaaS, and for the next few years at least, is a metaphorical red button that someone can use to kill a rogue process just with any Unix system. But with open source AI and some of these models are getting sufficiently capable, I don't know what you run on your laptop if you're running 1 of the Mistral models something. I run 1 of the Mistral models and it's pretty good. I pretended that when I was on the Eurostar on a plane, it would allow me to continue to work. But in reality, you just might as well get to your destination and use Perplexity or GPT-four. But of course, it's plenty good for task automation. It's plenty good for some of those basic behaviors that we saw in those early agent systems. And those things are out in the wild. And so I think what they do is they really expand the number of threat vectors that our existing systems face. Now this is so much more prosaic than capability explosion, but I just feel that with anything that is running on a data center managed by on an Azure cloud. There are many things you have to do before you fly an F 22 and drop a JDAM on it to stop it. But we saw script kiddies build botnets, and have seen them do that for a decade or so. And I think there's cybersecurity risk, which is capable enough models, and frankly, you were doing task automation with da Vinci. 2, you can probably get something better than that running on a smartphone now in quite a small pay payload. We're learning that we can get these payloads.

Nathan Labenz: 1:36:16 Probably a 3,000,000,000 parameter model could do a lot of the tasks I was doing.

Azeem Azhar: 1:36:21 And I wouldn't have noticed because it was snuck into a YouTube video download or, you know, whatever else it happened to be. So then you you do get to this world of, you know, many, many systems that can talk to each other, that can execute tasks for, I think, suspect naively complex DDoS initially. And that, I think, feels to me like it is more of a approximate risk, and it's 1 that requires that combination of infrastructural players. Right? You need Matthew Prince at Cloudflare, and you need Satya at Microsoft and and so on because they they own so much or control so much of the the the infrastructure. But it will also need new classes of new disciplines. Right? What is the security architecture of our devices? How can we stop them when they start to go go rogue when we think about how rapidly NotPetya spread? And that was before, you know, you had you had systems like this that could be a little bit more clever. But so so I imagine that that that is something that seems again like a present risk that will start to manifest itself over the next couple of years. And I mean, I speak to some of the cybersec guys and they are obviously thinking about what are the tools that you need to to defend and you know, from from these these types of things. But again, I still struggle with the models that take us to run away. If only if I'd taken take us back 200 years or a couple hundred years ago, and I'd said it's 1824, and the White House has just been burned in the war, burnt down. And I said to you, you know what, Nathaniel? In 200 years, you you know, you'll be a 100 times richer than you are today. You'll be richer than the richest man in the whole of this continent, The United States, and you will have these capabilities and and things that wouldn't have even sound like science fiction because science fiction didn't exist at the time. You might well, if you'd been able to believe me, say, well, surely we'll be at Utopia and all problems will have have disappeared. And we've run this tape before because we actually did get there and not every problem disappeared. There's been a lot of progress. And do I do also think there is something in kinda human psychology that has us looking at moments of change like this and believing that certain paths are possible, and we don't look far back enough to say, well, our forebears really felt felt the same. And I kinda feel that with not so much the climate crisis, which I think is is difficult, but I do slightly feel feel this with with AI systems because it feels like we're running that tape again. Now just to to add to my own confusion, I also see the power in the logic that says, number 1, is it possible for us to engineer intelligence, right? Or is it something that comes soulfully from a mystical superstitious force? It's possible to engineer it. I think you're a scientist, you probably believe the same thing. So number 2, if we can engineer it, is there any upper limit to what we can engineer? Well, no, there isn't because we regularly engineer machines that are more capable than us in different ways. So number 3, if we can do that, can we guarantee that it will be aligned with us? And of course, I I don't think we can yet. I don't think we have the science, yet. So I find that logic is really persuasive. It's hard to to pick holes at, except when you start to say, well, what are actually the underlying assumptions for each of those steps? And what is uncertain about what each of those steps? And what are where are their points of control for each of those steps? So I sort of I do agree that if you could magic up an incredibly powerful IQ 10,000 agentic with actions AI tomorrow, there would be issues. Let's just call it that. There would be issues. But but we're we're not we're not going to to do that. What's gonna happen is that we've gotta go step at a time, and at each point, there's research, there's development, there's stuff that we didn't understand. There are limitations. You talk about the kind of logarithmic scale of of inputs, right, that that is is slowing down. We're running out of data as well for training these models that play into what ends up being being real. So I appreciate the logic, but I also think the reality has unpicks into a lot of discrete steps around which you have to start to make progressively more extreme assumptions.

Nathan Labenz: 1:40:51 It does seem that we are in, as Sam Altman has started to describe it, the short timelines, slow takeoff world. And I think I agree with him, and it sounds like you probably as well, that that is probably the best case scenario because we do want the benefits now and because we probably do need some time to adjust. If you were to tell me that this is going to take 20 years or 15, and it won't be until 2040 that we'll have a human scientist level AI that's capable of prosecuting a long term research agenda and coming up with meaningful new discoveries and whatever, then I would be much more confident that we will have the ability to adapt to that over that timeframe. But I'm not sure about that. I see enough stuff now and I hear I don't know if Q Star is real or not real or if they're red teaming it in a bunker somewhere right now or not. But it does certainly seem still plausible to me that there is another paradigm changing moment that just creates another step change discontinuity in terms of capability that could get us there way before we're ready. So 1 of the things I'm watching for a lot these days is basically the transformer I initially used to think about it as transformer successors, but now I kind of think of it more as more likely anyway, as transformer complements, things that allow an AI system to do the things that the transformer does not do well. And 1 of those things is managing long contexts and kind of staying on task and online learning and integrated memory and so on and so forth. There's a decent number of things that are pretty obvious that they don't do well, going back to the original tale of the cognitive tape. You can look at all the places the human is clearly superior to the transformer and start to look out for architectures that might change that dynamic. If you said, How many meaningful breakthroughs are we away from the AI scientists that can produce eureka moments at a pace faster than human scientists tend to. It doesn't feel like it's that many. I would say it's probably more than 0, but it's probably less than 4. So somewhere in the kind of 1 to 3 range, because there's just not that many dimensions on the tape, the tail of the cognitive tape yet where where we're, like, all that much stronger. You know? There's a few, but I kinda put it in that 1 to 3 range. And with the inputs going exponential, including the, you know, just number of humans that are working on this and, you know, the number of papers they're putting out and the number of GPUs. You know? Unclear to me also if we're running out of data. I I don't really know about that, but, like, synthetic data seems to work for a lot of things, and there's also just more modalities. You know, we certainly have not taken advantage of, like, just think about how much security camera footage there is. You know? It's like, we really just wanna go big to go big. Like, there's a ton

Azeem Azhar: 1:43:52 sitting out there. The data issue is a speed bump. It'll get dealt with by accessing repositories that are available that we haven't touched or improve synthetic data. And as you say, modalities. Between 0 and 4 probably doesn't seem unreasonable. I had this conversation with some senior people in some of the different foundation model companies, and they say similar things. I think Shane Legg, cofounder of DeepMind, is probably at the lower end of lower than 4 from a conversation I remember him having on a podcast. I suppose then the question is how long does each 1 take? Transformers took not particularly long. I mean, was really a couple of years before GPT-one and 2 years before GPT-two, actually, and then 3 for GPT-three. And it doesn't take long in the world of archive, But the discovery does take time, and where that discovery is takes a moment. And and in amongst all of that though is still there is there is still that stage that goes from the software capabilities coming together because we have the know how and we've plugged it all all together, and it iterating to a system that presents a control problem to us. And in the case of the AI scientist, that is a system that we can't call Kevin, the CTO of Microsoft, and say, can you just shut down the Austin Data Center? And they see you for a second because the AI data scientist has gone rogue, the kind of in extremist mode. And and that path to me also seems unclear, and there there are a whole set of risks and downsides that emerge well before then, which I think help give us the infrastructure to to deal with that scenario. So and is really, you know, how do you deal with the the bad actors of for people use it with GPT-five quality LLMs on their smartphone in 3 or 4 years' time, and we'll we will deal with it. Right? In the same way that, you know, if I'd said to you in 1994, I don't know how old you were, I was, 22, that by 2024, there'd be a 120,000,000,000 identity attacks per year just on Microsoft. I I wouldn't have believed you. Would have I mean, yeah, Kurzweil, whatever. Right? I just wouldn't have grokked that number. So we'll have this unseemingly large number of of attacks coming mediated through LLMs and in botnets and elsewhere, and we will have developed systems to to deal with them. And that will be part of the fabric that we can't picture right now into which this AI scientist will get will get developed. And that's why these things become so very contingent. So the wrong thing to do is to say, well, because it's gonna happen, let's do nothing because then it won't happen. I think the right thing to do is to start to explore these ideas and have these these conversations. But 1 of the things I think is really problematic, has been problematic, has been the conversation focusing exclusively on existential risk, which it really I felt it did in 2023. What it does is it diminishes public trust in technology. It it forces policymakers to make decisions that may not be, you know, well well informed. They may not be pro innovation. They may not even be pro safety. Right? There may be a bundle of really terrible spots. I think a little bit of a a great Lu Chichen, he's a science fiction writer, this Chinese guy, who wrote The 3 Body Problem. But in his book of short stories, The Wandering Earth, there's a moment where they have to rocket Earth, Space 19 99 style, out of the sun's orbit to prevent some calamity. And it's gonna be a multi, multi generational journey to the to the next planet. And in order to do this, people have to live in really terrible conditions apart from the scientists who are keeping everything monitoring, looking for the signs, planning the process of decompressing everything. And of course, the people lose trust in the scientists and in true literature style, sorry, there's a spoiler, the people, rebel, kill the scientists, and then lo and behold, the next day, they show up at their destination. And trust is really, really critical. And I don't think we did a lot to get people who are outside of the tech industry to put trust into technology, put trust into their ability to participate in it and to have some agency in where it goes, to be excited about it. And off the back of all of the polarization and the sometimes legitimate, sometimes not scaremongering around phones and social networks and so on, it didn't feel like it added to the to the discussion of trust. So I was quite happy when I went to Davos, this World Economic Forum meeting, that the conversation had moved from is it, you know, what kind of munition should we use to to drop on a data center to what are what are real pathways? What is the science that we need to do in order to make these things safe in the long term? What is the kind of appropriate regulatory interventions? What do we do about things that are, you know, approximate 3, 5 years around misinformation and and and cyber threats while while still recognizing that there is a pathway that you've described that needs to be addressed.

Nathan Labenz: 1:49:08 I I find it easy to empathize with basically every AI perspective from the, you know, enthusiast to the ex risk concerned to those that are, you know, screaming about poor use of face match technology by police departments. I mean, really, whole thing, think, is, like, very it's all valid in my mind.

Azeem Azhar: 1:49:27 What's really exciting and what did not happen with the mobile didn't happen with PCs, it did not happen with the first mainframes, is that, you know, technology is is an intimate part of what it is to be human. Right? Technology is our compounded knowledge. Technology is is the the binding factor that enables a world in which we then have our human relationships. And to have so many discussions about a technology early on when it's just in its early adoption phase, we're not too late to it, I think is incredibly positive. And and I'm glad that you have a big tent show. I mean, have an opinion about sort of x risk, but I still also feel I'm big tent y. But I'm just really, really glad that we're having a wide and extensive conversation, 1 that feels wider and more extensive and kind of more grounded in some ways than any conversation we ever had about the internet back in the early nineties.

Nathan Labenz: 1:50:21 Yeah. In some ways, it's funny. I think in some ways the discourse is getting a little bit more deranged over time as there is some polarization and kind of ideological entrenching

Azeem Azhar: 1:50:33 happening in some places. But then in other ways, I definitely think it's getting better if only because what we're actually dealing with is becoming a lot more clear. There is a lot of room for commonality, for common ground, and for a recognition that there are different pieces of work that need to get done by by different people. And and actually, there should be there should also be enough money in the tank to be able to do it. Right? This is a rich industry. It spits out a lot of profits. We should be able to, you know, fund it fund it some way. Again, when you you see the kinds of things people say about each other on Twitter, Then you meet them in person and they have the same conversation and it's a more measured space just in my limited experience of it all.

Nathan Labenz: 1:51:19 I think it's kind of everything everywhere all at once. It's like, yes, there are definitely things that are quite unhealthy. And I do not like to see enemies lists getting published by leading technologists. That's the Techno Optimist manifesto from Andreessen has a section that is literally called the enemy and names names, if not individual names, least specific and relatively identifiable groups. So I don't like that. But at the same time, I made some noise about OpenAI and kind of could have easily been retaliated against by them. And I can imagine a lot of companies that might have come down on me hard and expunged my name from their case studies on their website and all that kind of stuff. And they didn't do any of that. And so I do think there's also aspects I feel pretty fortunate. This is 1 of the concluding questions I wanted to ask you is, I think in some sense, is a collective responsibility. We all have to orient ourselves to it, get familiar, and try to figure what's it mean for us and what can we do to to shape it in a positive way. It's definitely not all somebody else's problem. But at the same time, there are these, like, leading developers who clearly have outsized influence, outsized power. There are key decisions that are getting made around, are we gonna open source LAMA 3 or are we not? Sounds like we're going to. And then there's, like, government, you know, that can potentially say, you know, hey. We we require, you know, off switches at data centers. I'm not sure data centers have off switches right now. You know? You You might be able to go in there and start hacking at them, but is there an actual easy off switch in most of them? No.

Azeem Azhar: 1:52:57 I don't think there is. I mean, they're designed to be resilient.

Nathan Labenz: 1:52:59 Yeah. The opposite. Right? Yeah. They're not so and they're they don't probably want some rogue employee either to go in and turn it off. Right? So they've probably engineered away from anybody being able to easily turn it off. So government may have a role there to play that's like, look, we need off switches. We hope we never have to use them, but it seems like we might want to have them. So who do you think bears the greatest responsibility or where do you think we should be investing our trust? Is it these leading companies? Is it auditors that could be independent groups? Is it the government? I mean, it's probably some mix of all the above, but what are you bullish on in that regard?

Azeem Azhar: 1:53:42 I mean, The US has been an incredibly successful democracy for a long time because of separation of powers, and structurally that works. A company, however good it is, however well intentioned the CEO is, will end up with its own ambitions and directions. And so you will always need to push. If the company's offering you 3, demand 6. That's just good practice. So I think that each player in this circuit has to have the right capabilities to have the right conversations. And I think 1 of the things that we can learn from the experience of the FAA and Boeing is that you cannot deplete your own capabilities and ask for self regulation because it just doesn't work out however well intentioned the firm is. So I think what you need to do is we need to invest in the capabilities of of governments to ask good questions and engage well and overcome all of the complexities that exist that you can make $10,000,000 a year as an x at OpenAI and you won't do that in government. And I think that that also raises the value of investing in academia research and and civil society. So Joshua Bengio, for example, is running a a really important project. I think it's based out of The UK, which is a sort of core science project to look at some of these these risks and these evolutions and unanswered questions around control. We need to really, really start to to to level up. And I don't think it will be sufficient to just allow the big firms to do that and insist that they spend the money as directed by them. You know, I think if they have they're willing to put money into it, it should go into pots which go to grow the capabilities of the people who will keep them in check. And the reason that works is that, you know, the car industry is really successful because someone mandated that cars needed to have brakes. Now without brakes, people wouldn't buy as many cars as they do. And I think this is good for the industry, and someone needs to understand within government, within civil society, within academia, what are the right questions, and so what are the right interventions going to look like? And we can all agree as grown ups that companies, however well intentioned they are, will always have their own their own agenda. We just we just acknowledge that, and we all move forward in a in a generative, critical, constructive constructive way. So whichever player is weak at this table needs to have some support to become become stronger. That probably right now is is amongst governments and regulators and and academia rather than the big the big few tech firms.

Nathan Labenz: 1:56:22 That might be a great note to end on. Anything else you wanna touch on or cover that we haven't got to?

Azeem Azhar: 1:56:28 It's really easy. It's such a facility in having the conversations with you, and I really was so excited that you agreed to do this. You know, I thought the the murder mystery, which is you and your your red teaming show was just brilliant as well. So I know that you've got another hat, which is suspense. I look forward to the next episode of that.

Nathan Labenz: 1:56:50 Well, you very much. I really appreciate that. And I appreciate your time and participation in this as well. Azeem Azhar, founder of The Exponential View, thank you for being part of the Cognitive Revolution.

Azeem Azhar: 1:57:01 My pleasure, Nathan. Thank you.

Nathan Labenz: 1:57:03 It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.