Full Transcript

Nathan Labenz: 0:00 Hello, and welcome back to the Cognitive Revolution. Today, we're sharing a crossover episode that I recently did with Liron Shapira, creator of Doom Debates, who you may remember from his interview with OpenAI's Rune, which we cross posted late last year, and which I still think, by the way, is very much worth listening to. To be clear from the jump, today's episode is not a debate. Liron did challenge me early in the conversation to be a bit more, shall we say, consistently candid about my genuine sense that almost anything could happen from the unimaginably positive to the devastatingly negative as we develop and deploy more and more powerful AI systems across society. I appreciated that challenge. As And you might have noticed in recent episode introductions, I have adjusted my approach to make sure that I at least mention the incredibly high stakes of advanced AI development at some point in every episode. Beyond that, though, this is a news and analysis episode in which the 2 of us compare notes on some of the most recent AI releases and discourse that we each found most interesting. Among other topics, we discuss GPT-4o image generation and what it means for graphic designers and related businesses like Fiverr and my own Waymark. Replit CEO, Amjad Masad's declaration that he no longer thinks you should learn how to code. Peter Level's provocative claim that being an entrepreneur now has more job security than a job. Gary Marcus, who ridiculed OpenAI's latest $300,000,000,000 valuation and why, though his AI denialism is now totally indefensible, his question as to whether or not OpenAI will ever actually pay off for investors remains a good 1. Emmett Shear's new AI safety organization, Softmax, their organic alignment strategy and how that fits into the overall alignment and AI safety big picture. A viral AI not kill everyone ism tweet about the human caused sixth mass extinction event on the planet Earth and what we should infer from that as we consider our own AI future. Anthropic's new mechanistic interpretability paper, which I found characteristically outstanding, but also honestly kinda worry has been overstated in the understanding it actually achieves such that it could perhaps be used perniciously in political debates going forward. And finally, how much hope we each have that effective international cooperation can be achieved either via some sort of emergent stable equilibrium or via AI governance treaties, which would be backed by monitoring and verification regimes. I had a lot of fun recording this, and I'm very interested to learn how much value you find in this format. Would you like to see more of these rapid response news and analysis discussions, or do you get enough AI news elsewhere such that you'd prefer that I stay focused on deep dive interviews and analyses? Your feedback will help shape the future of the feed, so please let me know what you think, either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. For now, I hope you enjoyed this conversation with Liron Shapira, creator of Doom Debates, as we break down some of the most interesting recent developments in AI.

Liron Shapira: 2:58 Alright. Let's do some, news and social media. Connor Lee finally got a haircut. So this is Connor Lee with a haircut. This is what he normally looks like. He looks really good. I think this new hairstyle is working. The stock went from a peak of around $1.50 down to $1.045. So I'm embarrassed to say I went long when it was up 150. Like, I bought some options. Like, I'm I'm kind of, destroyed here. How about you?

Liron Shapira: 3:34 Welcome to Doom Debates. Today, I've got a very special episode. It's a collab with Nathan Labenz from the Cognitive Revolution. Nate and I actually go back all the way to 2010 when we were both living in Silicon Valley. We were both doing startups, and he was recently an undergraduate in chemistry at Harvard. That's his education background even though he later went into Silicon Valley startups. As you know, Nate hosts the popular Cognitive Revolution podcast, which is a highly educational podcast. It's 1 of the deepest dives that you can regularly see, interviews with people who are hands on deep in the field, all kinds of AI engineers and safety researchers, diverse topics and perspectives, and it's driven by Nate's own interest and curiosity. So it's a very authentic podcast. Highly recommend it. Nate is also a software engineer. He's the founder of Waymark, which is a turnkey solution for small businesses to generate video ads. It's known as 1 of the first companies to integrate GPT-3 when nobody was even paying attention to that technology. They currently offer a watch first experience where you can easily generate a candidate video video ad and then watch it and iterate. You can check out, Waymark in the show notes. Nathan Labenz, welcome to Doom Debates.

Nathan Labenz: 4:47 Great to be here.

Liron Shapira: 4:49 So, Nate, you and I were talking about, you know, how do we do a collab episode? Because oftentimes, I like to clash with the guest and, you know, push back on where the guest and I disagree. But I think for the most part, you and I are analyzing the situation with a lot in common. Like, I'm sure we have minor disagreements, but it's not like a fierce debate type of situation. So we were thinking, you know, what can we possibly do? And then we're like, oh, what if we just both sit on the same side of the table and look at all these all these other tweets and all these news articles, and then we both chat about that. Right?

Nathan Labenz: 5:19 Yeah. I think it should be fun. I I do, you know, I take all the issues that you cover regularly very seriously myself. So I think it's it's really good and healthy that there are, you know, people coming out with different, know, kinda tone and and tenor. Mine is a little bit more, maybe dry. Yours is more, gallows humor. And Yeah. I think both of those are, you know, are very valid and and worthwhile contributions to make to the overall discourse. I'm maybe a little less, I mean, and I know there's a a big question coming up. I think I'm maybe a little less confident in my, you know, downside worries about the overall AI situation than you are. But by no means am I, like, you know, denying or, or downplaying the risk. It does seem like very, very substantial to me. Just I wouldn't go, you know, quite as far as, like, feeling like it's the overwhelmingly likely scenario that we'll end up in some in some doom situation.

Liron Shapira: 6:23 Okay. Okay. Thanks for the clarification. So, Nate, you're you're originally from Michigan. Right?

Nathan Labenz: 6:30 Yep. And I'm in Michigan now.

Liron Shapira: 6:33 Nice. Okay. So you basically had like a half decade in Silicon Valley?

Nathan Labenz: 6:37 Yeah. 10 years away from Detroit. And then, honestly, never really thought it would come back. But, you know, topic for another podcast maybe. But there was a a revitalization effort of Detroit that included investment in startups. And I had to start up, my co founder was also from Detroit, my wife was also from Detroit, and that it was kind of a a contrarian move at the time to leave Silicon Valley, move to Detroit, and build a startup here. But that's what we decided to do, and we are here to this day.

Liron Shapira: 7:07 Nice. Nice. Yeah. So you moved to Silicon Valley, and you really created the ultimate trifecta here because you became a software engineer, entrepreneur, and podcaster. Doesn't get any better than those 3.

Nathan Labenz: 7:19 Yeah. I would say, you know, and only modestly, competent in in all 3. I I would say I'm, and my teammates would certainly sign off on the idea that I've always been a sort of vibe coder kind of software engineer, like much more a proof of concept guy, show that things can be made to work. And mostly leave the sort of production grade engineering work to others. And, you know, I guess all entrepreneurs are to some degree flying by the seat of their pants, so I shouldn't feel too bad about that. Then when it comes to podcasting, I sometimes describe myself as the Forrest Gump of AI because just so many times in my life in a not very strategic way, I've ended up stumbling through these like important AI moments and scenes. And I'm I've always been the sort of extra, you know, background character, but it it really has been amazing how many times that's happened.

Liron Shapira: 8:17 Alright. Before we move into our, news and social media roundup, are you ready for the most important question about you that everyone wants to know?

Nathan Labenz: 8:24 Yes. I I am ready. Doom. P(doom). What's your P(doom)? What's your P(doom)? What's your P(doom)?

Liron Shapira: 8:31 Nathan Labenz, what is your P(doom)?

Nathan Labenz: 8:35 Well, I don't know if it's a cop out to give a range, but what I usually answer when people ask this in general is 10 to 90%. And what I mean by that is basically, like, I really have no idea. I think it's, you know, very uncertain. I would put most of the weight on the lower end of that range and, you know, kind of fall in line with, like, the Dario.

Liron Shapira: 8:59 Okay. I haven't heard that 1 before. 10 to 90, but most of the weight on the lower range. That's an interesting way to describe

Nathan Labenz: 9:05 Yeah. And I don't really know why I'm saying that other than I'm kinda following folks like Dario who says 20%, and, you know, it it seems like there's a decent amount of things going right. And, you know, it we are talking about like a pretty extreme outcome. So Did you

Liron Shapira: 9:21 hear Dario recently walked it back on on I forgot which recent interview he was on, but he basically walked back the 2023 Logan Bartlett statement where he's like, you know, I never really said, like, to 25% chance of doom. I was just talking about, like, a big shake up for civilization. Like, he was, like, walking it back.

Nathan Labenz: 9:38 Well, they've changed their messaging on a lot of things recently, and I don't love it. I it feels to me like strategic communication, honestly. And the my candid take on that is I've lost a little trust in Dario as a result, and I wish that weren't true. I would love you know, I I wanted to believe in, like, you know, squeaky clean good guy. And I'm, you know, I'm just feeling like the the level of strategic ness in the strategy in these communications recently, you know, with he who must always be named presumably in mind as, you know, center of the target audience for some of these statements. Yeah. You know, I I just don't like that. And it it does make me sort of trust the situation or, you know, trust what we're hearing from Anthropic leadership less than I used to. Not that much less, and they're still doing a lot of great work. And it was you know, they're they're gonna show up in our rundown. So, you know, I I still have a a hold Anthropic in general in very high esteem, but, you know, a little less, than I used to because I I don't feel like some of these walkbacks or or, like, about faces are well, first of all, they're just not, like, properly explained. And, you know, in the absence of a, you know, proper explanation, it it feels like they're more sort of strategic and kind of memory holding, you know, things that I think worsens here in the past Mhmm. Rather than, you know, an an, like, honest change of perspective.

Liron Shapira: 11:16 Yeah. That's a great segue into some of the tweets we have bookmarked. So I think we're gonna elaborate on this theme because I agree with you.

Nathan Labenz: 11:23 1 more 1 more thing I'll give you on P(doom) though too. 1 of the best things that anybody ever said to me was my friend, Gopal, who said, we should think less about what the probabilities are and more about what we can shift them to. And so I do think, you know, big part of the reason we have this 10 to 90%, in my view, is there is probably some amount of risk that is an irreducible fact about developing this sort of technology at all. And, you know, I do think, like, the overhang arguments of if you have web scale compute and you have web scale data, you know, somebody's gonna figure out an algorithm at some point. So, like, we live in a world in in sort of a pretty, you know, wide range of timelines around this specific timeline where, like, powerful AI is probably gonna happen. And so given that, you know, there's probably some pretty irreducible risk. But then there's also the, like, really stupid risk that, you know, in theory, at least, would be, like, very avoidable. And that is the, like, let's weaponize this, you know, technology as fast and as aggressively as possible and, like, have a superpower race to global dominance. And, you know, that might be not just, like, a little bit more risk, but, like, that literally could be, like, a 10 x multiple of the baseline risk. And to the degree that I feel like I can do anything, it would hopefully be to sort of try to tamp that part down. So, you know, that's the if I if I have any contribution or any, you know, any impact, that's kind of what I hope it will be.

Liron Shapira: 12:51 Now this particular scenario that you're calling the stupid scenario seems like the closest 1 to reality.

Nathan Labenz: 12:56 Yeah. That is the I think that's unfortunately currently the path that we're on. And hopefully, we'll get off it. But, you know, it's right now, nobody's super you know, few people seem to be, inclined to receive that message just yet.

Liron Shapira: 13:16 So just to recap what you're saying about your position. So it's 10 to 90% waiting a little bit on the lower end. And I always say I'm 50%, but I also like that expression 10 to 90% because it just shows, look, the significant figures are low. Like, when you say 10 to 90%, it's like, don't even have 1 significant figure here. Right? Like, the first digit is is open to slide. But I also think it would be ridiculous to be like, my P(doom) is less than 1%. I think anybody who says that is being dumb. And the fact that I think that kind of implies that my P(doom) is more in the 10 so there there is some signal here. There is some information about what I think is justified to believe that it's, like, in this double digit section. But what I wanted to say about your position is it's kinda similar to mine. Let's say a little bit lower because you're weighting it toward the low But if people listen to your communications, right, if they listen to the Cognitive Revolution, I don't feel like that is a podcast where you're communicating that P(doom) is 10 to 90%. You know? It it's why I say, hey. You listen to Doom Debates? It's 1 of the only 2 podcasts that are communicating that P(doom) is high. So my question for you is, don't you think you should communicate more explicitly that P(doom) is high?

Nathan Labenz: 14:22 Maybe. I you know, I haven't been too strategic in my approach. But if there's been any strategy, it's to be sincere about both my, you know, very genuine enthusiasm for the upside of AI, as well as my, you know, very real fear of how it could all go very badly wrong. And so I do try to communicate both of those things. I guess maybe I'm just like, dispositionally not, you know, super emotional and maybe a little bit afraid of being wrong, and so kind of conservative in my positioning broadly. But I do try to represent both of those takes. And and I feel like in almost every episode, there's sort of this like, the future is like dramatically uncertain at least, you know, kind of vibe. Yeah. I mean, think it, you know, it is you're I think it's a good question, and it it does, It is worthwhile, I think, to hold oneself to account to the idea that, like, have I gotten used to, you know, a 10 to 20, you know, percent kind of Right. If I consider 90 to be sort

Liron Shapira: 15:35 of Right. Yeah. Tail risk, like,

Nathan Labenz: 15:37 am I? I'm trying to be the 1 to sort of wake the public up about a lot of things, good and bad. Have I kind of allowed myself to become the boiled frog on the bad side? And, you know, should I be more, you know, shrill in my in my statements? I do try to reevaluate that, you know, periodically. And I do think there's like possibility in the future that I could reposition myself as more of an advocate and less of kind of a you know, today, I I would say I'd maybe present as, a neutral analyst. And and neutral does mean, like, you know, recognizing and appreciating the upside for what it is. I mean, I do think what it is is tremendous. But yeah, I don't know. I I think it's a it's a certainly not unreasonable and and maybe, you know, shouldn't should motivate a little bit of a different tone in my in some of my commentary. I do have a ton of episodes on, like, all the bad behaviors. So I think that's 1 thing that I'm also kinda watching very closely is, like, just how bad are the bad behaviors and at what threshold levels, you know, should I really start to be I guess my there's another distinction too between, like, what is sort of theoretically appropriate to consider to be like a, you know, real possibility. And there I, again, have, like, you know, very open mind versus, like, what I can sort of credibly warn people about. And so I currently have this slide deck of AI bad behaviors, which is growing rapidly, that documents all these, you know, different studies, many of which have, you know, have been done by guests on the show. Scheming and, you know, deceptive alignment and reward hacking, and even like, you know, reward hacking in the wild. You you I'm sure you saw the Sakana, CUDA engineer thing where it was like, you know, we deeply apologize. But unfortunately, our AI CUDA engineer does not actually, write, you know, CUDA code that's that much more performant. But in fact, it reward hacked our system. Right. You know, that was done I mean, that's crazy. That's certain to be the level where I think decision makers actually should start to take it pretty seriously. And so that is 1 that I'm like, you know, this could happen to you guys, you know. Like, this is not that a researcher went and set up a situation and found some tendency under certain circumstances. This is like a well funded AI company published a project that was fundamentally totally flawed due to reward hacking. Like, you know, yeah, we are starting to hit some of these milestones where the yikes factor and the sort of like, there's no denying the reality of these things is, like, starting to get more real. And as that continues to happen, I I do think I'll probably, like, get more you know, my my the alarmism in my tone probably will go up as well.

Liron Shapira: 18:40 Nice. Nice. I I appreciate that you're even, you know, being reflective and entertaining the question because I'm definitely not trying to put you on the spot or single you out. On the contrary, if I had to list, hey. Out of all the podcasts out there, which podcast has a host who has a totally reasonable sane p doom? Like what you just said, that's way into the sane zone. Like, for me, the sane zone is 10 to 90%. Like, any number 10 to 90%. Hell, even 5 to 95% is reasonable. Right? It's just crazy to me that some people will go lower than 5 or higher than 95. Like, that's unreasonable to me. So I would I would actually put you at the top of hosts who are sane about p doom, and you never, like, you never do low blows. Right? Some hosts of some podcasts have and some guests have definitely seen dismiss the question so glibly, like, I'm not a doomer. You can't just be a doomer. You know, doom is bad. You know, like, very quick to dismiss it. You're not like that at all. Right? You're you're actually kind of 1 of the top. And yet, at the same time, if you just have a random technical listener wandering into the space listens to, like, 10 episodes of your podcast, I'm just not sure they're taking away the message of, like, how doomed we might be.

Nathan Labenz: 19:46

Hey. We'll continue our interview in a moment after a word from our sponsors.

Nathan Labenz: 19:50

In business, they say you can have better, cheaper, or faster, but you only get

Nathan Labenz: 19:54

to pick 2. But what if you

Nathan Labenz: 19:56

could have all 3 at the same time? That's exactly what cohere, Thomson Reuters, and Specialized Bikes have since they upgraded to the next generation of the cloud, Oracle Cloud Infrastructure. OCI is the blazing fast platform for your infrastructure, database, application development, and AI needs, where you can run any workload in a high availability, consistently high performance environment, and spend less than you would with other clouds. How is it faster? OCI's block storage gives you more operations per second. Cheaper? OCI costs up to 50% less for compute, 70% less for storage, and 80% less for networking. And better, in test after test, OCI customers report lower latency and higher bandwidth versus other clouds. This is the cloud built for AI and all of your biggest workloads. Right now, with 0 commitment, try OCI for free. Head to oracle.com/cognitive. That's oracle.com/cognitive.

Nathan Labenz: 21:00

Being an entrepreneur, I can say from personal experience, can be an intimidating and at times lonely experience. There are so many jobs to be done and often nobody to turn to when things go wrong. That's just 1 of many reasons that founders absolutely must choose their technology platforms carefully. Pick the right 1, and the technology can play important roles for you. Pick the wrong 1, and you might find yourself fighting fires alone. The ecommerce space, of course, there's never been a better platform than Shopify. Shopify is the commerce platform behind millions of businesses around the world and 10% of all ecommerce in The United States, from household names like Mattel and Gymshark to brands just getting started. With hundreds of ready to use templates, Shopify helps you build a beautiful online store to match your brand's style, just as if you had your own design studio.

Nathan Labenz: 21:57

With helpful AI tools that write product descriptions, page headlines,

Nathan Labenz: 22:00

and even enhance your product photography, it's like you have your own content team. And with the ability to easily create email and social media campaigns, you can reach your customers wherever they're scrolling or strolling, just as if you had a full marketing department behind you. Best yet, Shopify is your commerce expert with world class expertise in everything from managing inventory to international shipping to processing returns and beyond. If you're ready to sell, you're ready for Shopify. Turn your big business idea into cha ching with Shopify on your side. Sign up for your $1 per month trial and start selling today at shopify.com/cognitive. Visit shopify.com/cognitive. Once more, that's shopify.com/cognitive.

Nathan Labenz: 22:58 Yeah. Maybe I should put a little tag on it or something. You know, I have like a little audio outro that we append every episode that just invites feedback and thanks people for listening or something. I I could imagine adding just to that, like, you know, just to reiterate as a PSA, you know and I'm I might as touch your plan La Tokyo to that as well. Well, I think, you know, reminding I do think it's it is it is a good and I think people are also, like, definitely picking and choosing between episodes because I I do 2 a week typically. And, you know, that yeah. I'm well aware that that that's, like, more than, you know, the average person is going to have time for. And I do recommend people, like, follow a diversity of feeds and voices. You know? I I don't think, you know, people should over index on my perspective on AI by any means. So it is an unfortunate situation if people, you know, pick and choose episodes and they sort of only hear 1 side of my overall outlook, and, you know, don't take the other part on board. So maybe a, you know, kind of consistent reminder that like, this is sort of how I see things. And I I might put a p utopia on there too, or like a p, you know, post scarcity world of abundance, whatever. I also think that's pretty high, and I I tend to think it's probably more likely than Doom. Why I can't really justify that? That maybe it's just my disposition, you know, more than like a real rigorous synthesis of all the evidence. But nevertheless, that is like gut feel how I feel. But I do think we're like rolling the dice, you know. And and for me Hey. Yeah. You know, if you're if it's only, you know, 1 of 6 chambers Russian roulette style or whatever, like, I would still be, like, very terrified to play that game. And I think, you know, people should have a better sense that that is kind of the game we're playing. So maybe, you know, maybe I should add a little message, little token message Yeah. Every episode with something like that.

Liron Shapira: 25:10 Exactly. And I I don't know what would work best for you. But just as a brainstorm suggestion, you could always just start the show every episode by being like, hey. Welcome to the Cognitive Revolution. I just wanna let you guys know that I think AI is very likely to bring about utopia, but there's also, like, a 30% chance that it'll literally make humanity go extinct in our lifetimes. I just want you guys to know that I think that. Alright. Moving on.

Nathan Labenz: 25:30 Yeah. The stakes well, the stakes really couldn't be higher, honestly. I I mean, I I do sincerely believe that. So, yeah, I can't I'll think about the right form to do that, but I I think there is something that could be

Liron Shapira: 25:46 Yeah.

Nathan Labenz: 25:46 I mean quite socially clear.

Liron Shapira: 25:47 You seem to be sympathetic to to the cause of of, like, know, why I'm asking you this, which is just because, like, I think the Overton window could use more moving. Right? Like, I know there's less wrong. There's effective altruists. Like, I know this is now, like, in the water, and it's like a fun punching bag. Right? People know to, like, bring it up when they wanna, like, go to the extreme. Like, oh my god. Eliezer Yudkowsky thinks we're all gonna die. So I think the Overton window could use more moving where people like you who, frankly, have, like, a very middle of the road, like, mass appeal podcast still make it clear that we're like pretty likely doomed.

Nathan Labenz: 26:21 Yeah. And I'm I'm I'm even I am pretty sympathetic even to, you know, outright protest movements. You know, people have recently while I was in the Bay Area not too long ago, back in February, folks like chained themselves to the OpenAI front door or whatever and got arrested. Right? And I you know, that's not gonna be my strategy. And I wouldn't necessarily say I like endorse it. But I do think that that's, like, definitely not too radical, you know, of an action for some people to take given the situation. I I think, like, you know, if there's I think we talked about this 1 time, you know, offline too, like, how to frame these questions. I think it is often interesting to before asking people what they think the risk is, ask them what risk they think we should be willing to accept and sort of say, you know, okay. You're excited about AI. What, you know, extreme risk, what what percentage chance of truly like extreme risk do you think humanity should tolerate as we develop AI? I think most people would give something like under 1%, you know, for an answer there. Online, you do hear people more and more that are like, well, we're fucked with climate change or whatever anyway, or we're gonna have World War 3 unless we get AI to like somehow make peace. So, you know, they might accept something higher. But most people will come in like pretty low.

Liron Shapira: 27:58 Yeah. I'm I'm pretty willing to go to 10% just because, you know, a lot of the arguments people bring up of like, look. Think because when you when you factor in the cost of coordination, right, if there if there's a lot of problems with trying to slow it down or, like, the alternative has a lot of problems, then a hail Mary where it's like, look. It's a 90% chance of success. Yeah. 10% chance we all go to hell. But 90% chance, this is kinda like 1 of the few humps that we have to get over. So there's not gonna be that many of these in a row. So gambling everything on a 90% chance, to me, that's plausible. And Eliezer Yudkowsky said something similar. Like, he's not opposed to, like, a 90 10 type of gamble or even a 50 50 type of gamble from his perspective. He even thinks that those odds are good. For him, the problem is just that P(doom) is, like, even higher.

Nathan Labenz: 28:37 Yeah. And I mean, in the public though, like, broadly, if they actually knew that, you know, many people in the industry see it this way and are willing to take those kinds of risks, like, would in fact, you know, be motivated to go, you know, protest in person at their offices. And so think that's like, you know, in a sense, a pretty normal and probably should be expected reaction from people who are just like hearing for the first time that, wait. You're telling me that you think this, and you think that's okay, and you're, like, planning to go ahead and do it anyway? Like, it is it is a pretty wild situation. No doubt about it.

Liron Shapira: 29:19 Yep. Yep. Yep. Yep. Okay. Great. Well, thanks for indulging me in being such a good sport, you know, when I when I'm asking about communicating P(doom). I I thought you were super nice and introspective. Thanks.

Nathan Labenz: 29:29 My pleasure.

Liron Shapira: 29:31 Alright. Let's do some news and social media.

Nathan Labenz: 29:34

Hey. We'll continue our interview in a moment after a word from our sponsors.

Nathan Labenz: 29:38

It is an interesting time for business. Tariff and trade policies are dynamic, supply chains squeezed, and cash flow tighter than ever. If your business can't adapt in real time,

Nathan Labenz: 29:50

you

Nathan Labenz: 29:50

are in a world of hurt. You need total visibility from global shipments to tariff impacts to real time cash flow, and that's NetSuite by Oracle, your AI powered business management suite trusted by over 42,000 businesses. NetSuite is the number 1 cloud ERP for many reasons. It brings accounting, financial management, inventory, and HR all together into 1 suite. That gives you 1 source of truth, giving you visibility and the control you need to make quick decisions. And with real time forecasting, you're peering into the future with actionable data. Plus with AI embedded throughout, you can automate a lot of those everyday tasks, letting your teams stay strategic. NetSuite helps you know what's stuck,

Nathan Labenz: 30:30

what it's costing you, and how

Nathan Labenz: 30:31

to pivot fast. Because in the AI era, there is nothing more important than speed of execution. It's 1 system, giving you full control and the ability to tame the chaos. That is NetSuite by Oracle. If your revenues are at least in the 7 figures, download the free ebook, Navigating Global Trade, 3 Insights for Leaders at netsuite.com/cognitive. That's netsuite.com/cognitive.

Liron Shapira: 31:05 Here we go. First tweet. This is this is a good 1 to get you energized here. Sam Altman. He says, tremendous alpha with images in chat GPT right now. I thought this would be an exciting way to start because I've been using chat GPT-4o images, and gotta say, they cooked.

Nathan Labenz: 31:23 Yeah. It's really good. No doubt about it. It's you know, 1 really interesting thing about Waymark that people are often surprised by is to date, we have not integrated any AI image generation. And the reason for that is our customers are typically small business. We partner with, like, media companies, whatever. If you you go to our website, you'll see, like, all these cable companies. But they, in turn, are selling the creative that our technology creates to local advertisers. And these local businesses, they want things that really represent them. So they have images. They have stuff on their website. We, like, pull that stuff in. We use a lot of computer, what it used to be called computer vision. These days, it's like asking Claude or GPT-4O mini or, you know, Gemini Flash to, like, choose which images are appropriate, you know, for a given piece that we're creating. But we use their images because they wanna look like themselves. And it's been really hard to use any sort of text prompting or even sort of image and text that they just haven't been good enough to create something that sort of has this different style or like, you know, opens up the space of creative possibility while still being true to who they are. So that when people actually show up at their, like, typically physical local business, you know, that they feel like what they saw on TV, you know, matches what they're getting in person. And this does feel like the 1 that crosses that threshold. They haven't put out an API yet, so we haven't been able to integrate it. But reportedly, that's coming soon. So I think this probably will be the first 1. You know, it'll be up to the Weimarc creative team to determine, like, what sort of motifs or, you know, strategies we'll use. But you can see with all this, you know, stuff that is going on online that you you can kind of project yourself into these other, you know, creative spaces. And think that is gonna be really exciting. It's gonna be a big, unlock for our product. Mhmm.

Liron Shapira: 33:23 Yeah. Does this kinda compete with Waymark? Because I've actually been experimenting with generating a bunch of Facebook ads that are just images. And if I have a really good image, I feel like I don't need a video.

Nathan Labenz: 33:34 Yeah. I mean, it's coming for all of us eventually, I think. And I think this 1 not quite yet, certainly not for like actually, lot of our business is TV. So it's like 15 and 30.

Liron Shapira: 33:49 Oh, I can make a Got it. It. And Yeah. And they use so you need you need another year of development, basically, to compete with TV.

Nathan Labenz: 33:54 The presumption also is that it's a sound on environment, so there's a voice component to it as well. We do voice over, you know, as a native part of every generation. And by the way, that's also gotten really good recently. So yeah. Not quite yet does it substitute for us, but I do think, you know, 1 of the big existential risks at the scale of Waymark is that, you know, that you can just prompt your way to something in ChatGPT in the, you know, not that distant future. It's not that hard to imagine that.

Liron Shapira: 34:26 Right. Right. Right.

Nathan Labenz: 34:26 So So Yeah. It's it's not, I don't think it's a wrong thing to be somewhat concerned about for us for sure.

Liron Shapira: 34:33 Mhmm. When I ruminate on Sam's tweet that there's tremendous alpha with images in chat, GPT, I think I've identified a type of alpha there is. You know? Looking at my own company, relationship Relationship hero, Hero, when we're just putting our up our own Facebook ads, I had the realization that, like, we could work with the most expensive professional marketing firm in the world. And if we ask them for an image ad deliverable, I don't think it's going to be better than what a few prompts at ChatGPT-4o is going to give. I think we're maxing out the quality of image ads here.

Nathan Labenz: 35:01 Yeah. Very plausibly. You know, I I don't and and how much of the space, you know, of of all possible image space is sort of not yet explored by humans such that it's not in the dataset, such that, you know, you would really have to be a, you know, true novel creative to to go somewhere that nobody and Right.

Liron Shapira: 35:23 Right. Right. Yeah. And, you know, and I and I'm not saying any format of that, but I think specifically for ads, right, where where people look at the ad, they don't judge, like, the artistic quality. They're just like, did this catch my eye? Did this deliver the message? So all these imperfections, right, like the 6 finger or whatever doesn't really matter.

Nathan Labenz: 35:37 Yeah. No. I think in most cases, it it will satisfy us, and certainly on an ROI basis. And also, like, you know, for anybody who's done substantial digital advertising, you know, it's hard to predict what's gonna win when you actually make a bunch of variations and, you know, let the algorithm figure out what's resonant. So

Liron Shapira: 35:55 Right. Right. I was gonna mention that. Right? So a bunch of variations. So that's what I'm saying is, like so when Sam's talking about tremendous alpha, I think 1 of the types of alpha now is you have quality and quantity. So you could just generate a 100 high quality image ads in, a short afternoon. You put and then you combine that with Meta's ad network, right, which is like this ultimate network for evaluating how much engagement or how much conversion a particular ad will get you. And suddenly, you have this almost automated pipeline, right, where it's like high quality creative, get signal on what it's worth. And then within, like, a couple days, you use, like, the evolutionary process. Maybe you spend, like, a $100 total testing all these different creatives. And now you have, like, this insanely good, like, best possible ad that there is.

Nathan Labenz: 36:33 Yeah. And I think it won't probably be too long before Meta will have their own version of that that

Liron Shapira: 36:38 Right.

Nathan Labenz: 36:39 Will literally just work in the, you know, in the ad creative workflow.

Liron Shapira: 36:43 Right. It's crazy. That's so yeah. But I'll report back. I I can't say we have data yet to show that this has actually worked for my company, but it I did make the observation of, look. Either this will work or nothing will work. Because, you know, typically, we do advertise on Facebook, Instagram, and we get, like, a trickle of traffic. We've never gotten, like, a ton of traffic. So I told my team, I'm like, listen. This is the final test. Like, if this doesn't work, it just means we can give up on meta ads.

Nathan Labenz: 37:08 Yeah. I think that seems probably right. I mean, good, fast, and cheap, you know, you can get all 3 now. So that is Right. That that that does change things. I I don't know what that means for like Adobe. I don't know what that means for, you know, the creative workforce more generally. But it is, you know, maybe the beginning. I mean, 1 story I can tell you from way, Mark, this is, you know, different modality. But we used to do before text to speech, we had professional voice over as an optional add on service. People wanted it, but there was just no way to do it in product. So we just had it as sort of a, if you want this, we can, like, you know, facilitate it for you, but it's an extra cost. And you gotta wait, like, 2 days for it to be turned around, and then maybe there's, like, back and forth. And, you know, we worked with, like, a really good voice over specialist that, you know, provided great value at, honestly, a great price point and was, like, always very well reviewed. But the volume that we're sending to that service provider today has dropped by more than 90% now that we can generate, you know, probably still not quite as good. You know, this is kind of the classic definition of technology disruption. Right? We're offering something now that is probably still inferior most of the time, maybe all the time. Like, again, the service provider that we worked with was really good. And I remember having a conversation with him 2 years ago, and he's like, I'm worried about my, you know, my future. And I was like, I think we should all be worried about our future. So don't you know, it's not just you. Don't feel too bad. But it is maybe you sooner than some others. And, you know, sure enough, even though it's still not quite at the level that, you know, he and they used to provide. It was 1 main guy and and a team that, you know, kinda supported him. You know, it's just it's immediate. You know, people can hear it right away. They determine it sounds good enough. It's included for free because it only costs us, you know, a couple pennies, you know, versus it was $99 was the, you know, the human price point. And that was, by the way, considered to be a very good, like, price point that we were on. And we made no margin on that. We we charged 99, passed it entirely through. We just wanted to provide the most value we could to to customers to meet this need. But, you know, it's really hard to argue with free compared to functionally free compared to a $100, combined with, you know, the speed of turnaround and the ability to just edit right there and get it done and be onto your next thing. And so, yeah, the volume has driven has dropped by more than 90%. And, you know, does that same thing now come to, like, graphic design broadly? I think it very well could. I I don't know, you know, why it wouldn't to some you know, maybe not order of magnitude, you know, in the next few months, but, like, hard to see how it doesn't happen something like that, you know, on, you know, maybe 1 more, evolution of the model kinda timescale.

Liron Shapira: 40:13 Yep. So you were impressed with this screenshot. Riley Goodside tweeted, the screenshot. A fake screenshot generated by chat GPT-4o of a Wikipedia article about the screenshot itself with a copy of the screenshot in the article. So for people listening on the audio, it looks like a totally authentic Wikipedia page, but it's just a rendered image. It's got, you know, the sidebar or the title that has the screenshot, self referential image. And then within the article is, like, an image of an entire Wikipedia article. Right? It's 1 of those recursive images. Right? It's like the image inside the image. And what's crazy is this was done just typing in a prompt and then immediately just getting this entire recursive image of a Wikipedia article rendered complete with a bunch of text in it too, like description. The screenshot is an image depicting a screenshot of a Wikipedia article titled a screenshot. So GPT-4o was able to combine all of these elements on the first render. Like, this and then you were quite impressed by this.

Nathan Labenz: 41:05 Yeah. I mean, I think everybody should be. You know, if you're not impressed by this, I'd like to know why. It's I mean, by the way, I think everybody should follow Riley on Twitter. He is consistently a outstanding demonstrator of of new model capabilities. He was, at 1 point, maybe the world's, first and only staff prompt engineer was his original title, at Scale AI, which he joined maybe 2 years ago now. A job that he literally tweeted his way into by just, you know, providing 1 example after another of, like, what

Liron Shapira: 41:43 Right. And to be fair, I think he he said this took him a few prompts. Right? But even the fact that 1 reasonable prompt will generate this is insane.

Nathan Labenz: 41:50 Yeah. I mean, he he may have iterated a little bit, but in the end, it is kind of, you know, it's a single shot. You know, the the thing outputs an image. Right? And this is the image that it He's that it output.

Liron Shapira: 42:01 By the way, I zoomed all the way in because I'm like, wait. If it's infinitely recursive, it looks like there's only, like, maybe 3 levels of recursion. So I zoomed all the way in, and in the innermost level, it doesn't have like another picture. It just has like a yellow block of text that you can't read. Funny enough.

Nathan Labenz: 42:14 Yeah. It shows you kind of the I mean, there's a bunch of little when you really start to inspect this closely, you see a bunch of little issues. You know, there's like a bunch of words that are not words, and there's words that are like spelled wrong. And so I think it's kinda charming in in that respect as well.

Liron Shapira: 42:27 Yeah. Yeah. The deeper you go into the screenshot, the more the words are just like not words. But there's some non words in the outer screenshot too. Anyway, it's it's quite inter I mean, it's it's file is under the category of like, if this if we knew that this was our future, going back in the past 5 years ago, we'd be like, how the hell could this ever happen?

Nathan Labenz: 42:44 Yeah. Absolutely. I mean, it it shows obviously real depth of understanding, and Right. It also, I think, shows the power. And this is 1 of the ideas that I'm kinda chewing on a lot right now of deep integration of modalities. I think a huge question this is, like, kinda become a little bit of a stump speech for me, is, you know, what does superintelligence look like? It's hard to predict. Right? I mean, it's it's it's sort of, I think, many people, a very vague notion. And I wouldn't say I have like a super crisp answer. But I think 1 candidate answer is you take a frontier model of the sort that we are familiar with today, and you integrate in a similarly deep way a bunch of different modalities instead of, you know, just the, you know, the image modality that we're seeing demonstrated here. Like, if instead of the text and image being so deeply integrated, you had text and interaction of biological molecules so deeply integrated.

Liron Shapira: 43:56 Or Right.

Nathan Labenz: 43:57 You know, predictions about the evolution of a cell, like the the next transcriptome state of a cell. We have seen, and there are plenty of of narrow models that can do wondrous things. Right? AlphaFold and, you know, many, many, many more that are developing a sort of intuition. Sometimes I call it intuitive physics for all these other problem spaces, such that, you know, they can take a DNA sequence or a protein sequence and predict how these things will fold up in 3 d space, which, like, people can't do, or even predict how they're, you know, they're gonna form into a complex or how they're gonna interact or how they'll sort of, cohere together around some metal, you know, ion at the center. Like, it's getting really quite amazing what these things can just sort of Totally. Spit out from a little bit of data. But right now, at most, an AI can call that as a tool. You know, it can make like an API call. This is sort of analogous to what we used to have in ChatGPT where, you know, the language model could call with a prompt. Okay. The user has asked for an image of this. I'll write this long prompt and try to capture what the user wants and get it back. But, know, as we just talked about at Waymark, like, it never looked like them. You know? It was never really quite what they wanted. There was this super lossy bottleneck, you know, due to this sort of arm's length API call or tool call type structure between the language model and the specialist model that just really limited what you could get out of it. And now you see this integration and an explosion of possibility. And I think 1 good candidate or at least 1, you know, mental model I found super helpful for what a superintelligence might look like is do that again for 20 more modalities.

Liron Shapira: 45:45 Right.

Nathan Labenz: 45:45 Most of which we don't have. Right? We do still have the image modality. Not all of us can draw at this level, but we can at least kind of visualize and we know like what's right and wrong when we see it. But when we get into sort of how is a cell gonna respond to a certain perturbation, you know, we're starting to have models that can do that. We can call them as tools and interact with them in this sort of, you know, iterative way, but we have not seen the latent space deep integration mind meld between a sort of broader reasoning system and 1 of those narrow systems.

Liron Shapira: 46:16 To give some context for the audience, you're you're talking about this because GPT-4o is actually a breakthrough in integrating. Like, it's not just drawing, like, a diffusion model where it's saying, like, what is this pixel likely to be once once I do it at, like, a finer grain resolution? It's more like it's somehow simultaneously thinking about the prompt using GPT-4o. It's thinking about the prompt. It's thinking about the text separately. Right? Because they, like, made some improvements somehow to text rendering by actually, like, understanding how to make a g exact text shapes. Like, we don't really know what they did because it's a secret, but we know that it's it's not just, like, naively predicting image. It's like drawing and thinking.

Nathan Labenz: 46:53 Yeah. There have been some interesting we could even pull, a couple of these up. But, you know, they demoed this almost a year ago, maybe even a little more than a year ago now for the first time. And Greg Brockman put out a tweet showing a person at a blackboard with handwriting, and this was this image was generated by GPT-4o. And the handwriting on the Blackboard is like, what if we model text plus image plus audio all jointly? And so there's, you know, there's some deep integration that is happening where there's, like, a shared latent space and, you know, the whether an idea is presented to the model in text form or in image form or in audio form, it's all converging into some shared space of understanding that is much higher dimensionality and and allows for, like, much richer communication across these modalities than a simple, like, prompt to, you know, an image generation model, you know, previously did.

Liron Shapira: 47:58 Yeah. So then I tweeted a couple days after GPT-4o. I just said Fiverr stock holding steady this week, which it's still holding steady last I checked. So what do you think of that? I mean, it's already gone down a lot. It's already down to, like, an 800,000,000 market cap. I think it was it used to be 10,000,000,000 plus. Don't you think it should fall farther when GPT-4O comes out?

Nathan Labenz: 48:19 Well, it's gonna be a challenge. I mean, I have maybe too early for me to say. I have a a episode coming up next week. Should be recording with the CEO of Fiverr. And, you know, they're not sitting on the sidelines of AI by any means. You know, they are basically, every aspect of their business, they're trying to reimagine with AI, including, like, how the service providers sign up and present themselves. You know, they they provide AI gig support for you to define your services. They have, you know, similar things on the buy side, like, to help you flesh out your requirements and make sure that you're, like, actually, you know, properly asking for what you need. I assume that they have a lot of AI matchmaking going on behind the scenes to try to, you know, grease the wheels of the marketplace.

Liron Shapira: 49:06 Sure. Yeah. I mean, and it's already kinda low valuation already. Right? So people are just saying, look. As long as there's some chance that humans will be in the loop somehow, then this business will have some value, and it's already low. So, I mean, it makes

Nathan Labenz: 49:18 Yeah. They have an interesting thing called Fiverr Go too that I have just been exploring a little bit, which is like it'll be interesting to see how people take to this. But it's sort of things like you're a voice over artist. They'll clone your voice and then allow you to provide an AI version of your voice where, like, a human, you know, gets properly paid. And, you know, some obviously, for creators, that is, like, upside. Right? Their their goal is to, you know, make the creators indispensable. I think that's how they put it. So Right. You know, will people pay, though? You know, it the ratio between my my, like, original human voice over at $99 and the, like, 3¢ or whatever from 11 Labs leaves a lot of room. How much more will people be willing to pay? And, like, is that gonna be sustainable to know that, like, some underlying human was, you know, in some sense, justly compensated? I don't know. That remains to be seen, but that seems to me

Liron Shapira: 50:17 I will say this. I went to Fiverr to try to get a YouTube thumbnail for an episode of Doom Debates, I also went to 99 Designs. And I got okay work, and it was like it took my attention, right, to, like, review the submissions and talk to them. And now I don't think I'll ever go back from GPT-4o and, you know, maybe some tweaking in Photoshop. Like, I'm pretty sure I'm done with Fiverr for that particular job.

Nathan Labenz: 50:36 Yeah. I think a lot of it also is just arbitrage. You know, 1 of the I recently gave a talk to a bunch of students about the the possibility of working as an AI scout. And 1 of the things I told them is, you know, don't apply for grant funding. If you wanna be an AI scout, instead you should live off the digital land. And what I mean by that is like, you know, AI can do just straight away a lot of the jobs that get posted on Fiverr and, you know, Upwork too and whatever. And so, like, why is that happening? It's because why are people so posting those jobs? Like, they don't know. They know how to use these tools. They don't know where to go. But if you do, and you can go just use AI to deliver quality work, you know, I think I think praises will come down. It's definitely like a deflationary phenomenon. But at least for the foreseeable future as, like, the options in AI are overwhelming, it's gonna be worth it to a lot of people just to, like, pay somebody who knows what the right tool is to use. Like, that's a lot of the work that I do commercially. It's I'm not like sometimes I, like, come up with a, you know, really creative or insightful solution. I like to flatter myself. But, you know, more often, it's like, I just have a pretty, you know, as close to comprehensive as anyone can maintain these days, again, if I'll flatter myself, sense of, like, what the tools are. And that's really what they're paying me for. It's just, like, my knowledge of what is the right tool to use. And then, you know, they use it, and then they can sort of feel confident that, like, they got the best available AI option. So, you know, a lot of what goes on if I ever might shift toward that. But there's at least, you know, probably still some volume there for a while.

Liron Shapira: 52:14 Exactly. Now when you bring up that you're such an expert at the tools, you know, tool you use, I totally agree. And I think people should hire you for your expertise based on that if you're if you're doing any consulting. But it also reminds me of, you know, it's like how long will this last where expertise in the tools is going to be a competitive advantage? Probably not that long. Right?

Nathan Labenz: 52:31 Yeah. No. I agree. I mean, it's coming for all of

Liron Shapira: 52:34 Yeah. No. Naval tweeted yesterday this image, where in his mind, he's like, you know, that classic image with the astronaut where there's an astronaut shooting the Earth, and Earth is labeled jobs. And the astronaut with the gun says AI, like AI is killing jobs. And then there's another astronaut behind him saying AI jobs. Right? So you can get a job where you can, like, work the AI, and that'll actually defeat the AI. Like, AI jobs gets to shoot the AI. But it it seems to me like you wanna add another astronaut behind AI jobs saying AI again. Right? Because then AI will just learn how to do the AI jobs.

Nathan Labenz: 53:03 Yeah. I totally agree. I I mean, I think that you know, I hope that what I'm doing first of all, there's a question of timeline. So, like, you know, exactly how fast does that happen? I tend to talk about the shorter range of the timelines that I actually believe. So if you said, like, you know, when does transformative AI arrive for some definition? I might say, you know, 2 to 5 years might be my, like, 80% confidence range. But I usually kinda try to think and talk more about the 2 years, just because that seems like, you know, better to put that time pressure on myself than, you know, and play for that. And if I have more time, like, great. I hope that what I'm doing, and I hope that and I think that what like other AI scout type people would do over these next couple years is, you know, is an AI job, that might be really socially valuable because I think we just have a lot of work to do to, like, characterize the AIs, you know, and just know what's going on with them broadly. So I think it is both a

Nathan Labenz: 54:14 you know, I think it's both, like, a job you can do that, you know, can pay the bills and potentially a really socially valuable contribution. But I do think it is, yes, a, like, relatively short time scale that that holds. And beyond that, you know, my anybody's guess really is is as good as mine. I ultimately think, in a good scenario, we get to a world where you don't have to work to eat. You know? And I think, like, that's a big part of why I'm excited about AI. Living in Detroit, you know, AI is not on everybody's minds. I ask people pretty regularly, like, if you didn't have to do the job you do to have the resources, you know, for the rest of your life, like, would you still do the work? And the answer is overwhelmingly no. You know, people are not, like, desperate to keep the jobs they currently have. So I feel like, you know, that would be ultimately a good thing. And, yeah, I think the AI jobs thing is, like, transitionally, maybe really useful. But beyond that, I totally agree there's another AI coming for the AI jobs.

Liron Shapira: 55:19 So speaking of AI jobs, let's listen to Amjad Masad. He tweeted, I no longer think you should learn to code. And he quote tweeted this account of Vitruvian Potato that posts a lot of good clips from Technosphere AI. And Vitruvian Potato says, instead of learning how to code, Replit CEO, Amjad Masad, says, learn how to think, learn how to break down problems, learn how to communicate clearly. Okay. So this is a hot topic. Right? Should you learn how to code?

Nathan Labenz: 55:44 Yeah. My my twist on this is I tell people you should not be afraid of code. And if something you wanna do, including, like, you know, living off the digital land as an AI scout, making money on Fiverr or whatever, requires some code, you should be very confident that the code that's required won't be, if you're willing to put in, you know, any modest effort, a fundamental barrier. Like, anybody can get over that barrier today. I do think, you know yeah. It's I I my sense right now is that the market is tough already for junior developers. And I feel like in general, what I'm hearing from most people is a lot of denial and cope. You know, it's it's I definitely expect we're gonna see a lot. And 1 of the arguments is like, well, you know, when software when something becomes more valuable, you want more of it. So, you know, this is all very empowering to software developers. They're gonna, you know, be able to create so much more software. We need more software. That's why it's been in such demand. So there's gonna be, you know, more and more demand as they become more productive. I think that, like, logic holds for a little while, maybe. Although the time to use such software is, you know, sort of questionable, and like how much of it is ultimately gonna be just dynamically written on the fly as sort of AI agents navigate the world and kind of interact with each other at some point, you know, ultimately feels to me like that trend wins more so than the like, more productive, so we wanna hire more of them. And, you know, again, you see this. You look at Cursor. Like, how many people do Cursor does Cursor employ? Not very many. You know, a company like them would have been hiring at, like, insane pace not too long ago. But they're just like, we want the most, you know, cracked AI agent managers that we possibly can find, and, you know, that's gonna be our team. And I just did an episode with Shortwave, which is an email, client product, and they are aiming to keep their team at 15 for the foreseeable future. This is a company that's like got exponential growth curve right now, and raise more money on the on the strength of their exponential growth curve, but they're not scaling the team. So, you know, I I think you look at some of these leading companies and you're like, okay. Are they behaving in the way you know, these are the people that know the technology best. Right? They're the people that presumably

Liron Shapira: 58:03 I don't know. The ballpark now is I think they just landed like a 10,000,000,000 valuation, and their team sizes, I don't even know, like a dozen engineers. Right?

Nathan Labenz: 58:09 It's not many. Yeah. I think it's it's, like, for sure under 50, and I I don't know exactly

Liron Shapira: 58:13 Yeah.

Nathan Labenz: 58:13 The number. But yeah.

Liron Shapira: 58:15 I mean, that's yeah. That's definitely an insane ratio. Mean, people people thought it was incredible when Instagram sold for 1,000,000,000, and and their team was, a dozen. But now we're like, okay. Let's do it for 10,000,000,000.

Nathan Labenz: 58:26 Yeah. I mean, again, like, these people should know best. They, you know, they should be the best at at getting the most out of the AIs. You know, if if it's true that, like, the software developers are so productive, you're gonna want more of them, then why wouldn't that be true at cursor? Why wouldn't that be true at shortwave? Why wouldn't that be true at Repli? Repli's not that big either. They've built, you know, pretty Right. Stuff, but I

Liron Shapira: 58:52 clean. Yeah. Mhmm.

Nathan Labenz: 58:52 I think there may be a 100 people, maybe a little more than a 100 people. They predate, you know, the whole AI wave. Yeah.

Liron Shapira: 59:00 People were dunking on Amjad like Theo t 3 dot g g. He tweeted, loaded the screenshot from Hacker News. Replit is hiring engineers to automate coding. But people were pointing out, look. He Amjad isn't saying he doesn't need to hire a software engineer today. He's saying that, like, in 4 years or whatever, right, by the time you finish college, is he still gonna be hiring engineers? Maybe not.

Nathan Labenz: 59:21 Yeah. And I mean, it's just a question of number. Right? Like, he's we have told society has told people for the last decade that, you know, you learn to code. It's a path to a good income, stable employment. You know, it's like, it's the surest ticket to the American dream. And so much so that,

Liron Shapira: 59:43 like I've always thought of it as such. Right? That's I think that's been, like, a pretty good source of my self confidence is always knowing, look. Can always code. Right? So I can always park myself, like, anywhere in the world with a laptop. Right? And Yeah. It has been true. Like, a surprisingly high income. Right? And I I feel like in the back of my mind, it's always been a source of confidence.

Nathan Labenz: 59:59 I think it has been true. I think it's been questionable sometimes when it's been like, yeah. If your job gets, you know, eliminated, you can always learn to code. I don't know how realistic that's been for a lot of people that have kind of been sold that. But that's maybe a different question. But now it's like

Liron Shapira: 1:00:15 Right. I mean, that and that's the problem is. Right? It's not true it's not true about every citizen. Right? It's like, I'm I'm cognitively fortunate or whatever, right, to have that, like, highly technical or 1 might say, aspy mind.

Nathan Labenz: 1:00:27 And it seems to me that we might be headed for a situation where, you know, to just very roughly not these are not meant to be, like, literal numbers by any means. But maybe we'll see 10 to a 100 times as much software built by 10 to 20% as many dedicated humans doing, you know, most of the high end work. And, you know, probably what we need in those humans is like the deepest, you know, hardest core experience possible, because those are the things that the AI struggle with. The things that the, you know, boot camp grads are learning of like how to create React components and, you know, make front end UIs. Like, that's being commoditized very, very rapidly. You know? And, like, full stack rag apps, you know, or not rag, but full stack, you know, crud apps, like, that's also being commoditized very, very rapidly. When you're talking about, you know, deep tech hard software that's not being commoditized so much, but how many people work in that line of work? You know? My guess is the numbers go down. You know, wish it weren't so, but I I don't know how to see it another way.

Liron Shapira: 1:01:38 Yep. Yep. Yep. Okay. And then, let's see. So then Rohit Krishna on the subject of AI unemployment, he tweets, considering AGI is coming, all coding is about to become vibe coding. And if you don't believe it, then you don't really believe in AGI, do? And then Ethan Moloch replies, interestingly, if you look at almost every investment decision by venture capital, they don't really believe in AGI either or else can't really imagine what AGI would mean if they do believe it.

Nathan Labenz: 1:02:05 Yeah. Or else just have to deploy money somewhere and, you know, would rather continue to deploy as if no AGI than, like, return the money to, you know, investors. I'm not sure. Yeah. I'm sure it's a mix, but I do agree that like most of the venture capital investments I see today feel like they're going to 0 because or just or at least not 0, they'd be too strong, but like, seems like it's gonna be hard to achieve venture returns when, you know, the the the AI platforms just seem like they're gonna be sort of the black holes into which everything is is gonna collapse.

Liron Shapira: 1:02:46 Yep. Now here's my candidate for best tweet of the week. Peter Levels says, being an entrepreneur now has more job security than a job.

Liron Shapira: 1:02:59 I think I agree. Right? Because, like, when I when I see unemployment, like, yes, there's robotics. Like, maybe plumbers will be the last ones to be replaced unless there's a teleoperated robot. But, like, putting robotics aside or, right, putting, like, physical work stuff, like, oh, you gotta climb a tall pole. Right? That'll be, like, the last job in order to, like, do do a repair. Putting that aside, if you just look at, let's say, white collar work or work at your laptop, it does seem like the tide is rising. And and I think I'm on the same page as Peter where, like, entrepreneurs are gonna be the last 1 standing because it's, like, by definition, it's just, like, the ability to know where to scurry to to, like, be able to make the chunk of money.

Nathan Labenz: 1:03:32 Yeah. I think that seems apt. I mean, it's a again, I sort of feel like we need to start to wrap our heads around the new social contract. We can't all be, you know, entrepreneurs scurrying around looking for, you know, change in the couch of the broader AI economy.

Liron Shapira: 1:03:51 And I mean, you and I are, to be honest, in a pretty good position. You should be like that. Right? I mean, we're we're kinda good candidates to to be, like, the final scurriers.

Nathan Labenz: 1:03:59 Yeah. I I mean, I'm I'm certainly much less worried about myself than I am, you know, many many other people who, you know, don't have the sort of generality of skills or, you know, just kind of, aren't as accustomed to, like, taking on whatever random, you know, thing happens to be next in in front of them as as I've become over, you know, years as an entrepreneur. So

Liron Shapira: 1:04:22 Right. But but, you know, I'm becoming a little bit of a coding boomer because now that I'm like, I'm using cursor. I'm not using the very latest. Like, I'm not using Windsor for code buff, Y Combinator company. But I'm using cursor, and I still am manually editing my code line by line. Like, yes, I accept suggestions, but I'm still manually editing line by line. And my understanding is that the kids these days, they try to be hands off. Right? So they try to just entirely just talk to the editor. Right? And I think even, Andrei is trying to do that now. Right? So I haven't done that yet. I'm, like, nitpicking at the individual characters of my code. I'm like a boomer.

Nathan Labenz: 1:04:55 Yeah. Well, you're probably a better coder than I am, and that probably does play into it. You probably have a higher sense of, like, craft and, you know, pride in the work. Interestingly, I'm more precious about my writing, and I'm much more willing to engage that way with code. So I do like, the the writing task that I do most often is an intro to The Cognitive Revolution, and I'll I I do use AI to generate the first draft of that. I try to use AI in everything I'm doing just because I need to I need to find ways to do it, if only to learn, even if it's not useful. Increasingly, it's, you know, almost always useful. So I context self Claude. My best solution is still a pretty simple 1. Just collect a bunch of previous essays, and initially, was writing them totally from scratch. Now they're like the refined versions of what Claude has given me. But paste those in there as examples along with the transcript, and I just say, adopting the style, tone, voice, and perspective represented in the essays. Write a new 1 for the attached transcript. And then sometimes if I have other things on my mind, you know, that the model is obviously not gonna know or not gonna be able to to guess, that I indicate that as well and have it write it. When I do that with code, I'm always like, you know, sometimes it'll get it wrong enough and I'll be like, okay, that's not what I meant. You know, do something different. But I almost never edit at the at the, you know, line level. You know, if I if I flatter myself, I would say that's because I'm, you know, good at staying in touch with youth culture. But then when I look at my behavior on the writing side, I'm like, maybe it's just that that's where I have developed like a sense of identity and pride in the details of the output. And I'm not sure that that really even matters to the audience, frankly. I think, like, a lot of times, I probably could just read what Claude wrote, and everybody would be fine. But I care. And so I do go through and edit and, you know, keep decent chunks of what it wrote, but also, like, massage and, you know, refine word choice and try to make it truly my voice. And so maybe it's, you know, to maybe there's sort of a almost disadvantage in adopting technology in some ways where you feel like you have a distinctive voice that you, you know, care about and wanna maintain versus in areas where, like, for me with code, I don't. I've never cared about, you know, the details, the cravits. For me, it's like, make it work. Move on, you know. And so, yeah, I'm I'm in the camp of basically no line edits on the coat side, but still haven't been able to get over my own, you know, preciousness on the writing side. Totally.

Liron Shapira: 1:07:33 When I'm using these tools, also, I think back to the first decade of my career, we used to not even have Prettier. And I know there were like fancy IDs that would indent your code. Prettier is like you save your file and automatically indents your code. I used to spend a good amount of brain cycles thinking about my indentation.

Nathan Labenz: 1:07:50 Yeah. I mean, syntax errors were not always, you know, necessarily super easy to find. Like, even just color. I'm I'm old enough to remember, you know, color highlighting being like at 1 point over.

Liron Shapira: 1:08:00 Yeah. Like, editing a notepad where every character is white. Yeah. You don't even have, like, the highlighting the parentheses or whatever.

Nathan Labenz: 1:08:06 Yeah. I think those tools did predate my start in coding, but I didn't necessarily know about them at the very beginning. So, yeah, it's to say we've come a long way is obviously a major understatement.

Liron Shapira: 1:08:18 Yeah. Alright. So here's Gary Marcus. He is known for not being that impressed with AI progress and thinking that we have, I don't know, more than a decade before the Singularity, which is like the pessimist view now. So he recently tweeted, breaking. SoftBank is valuing OpenAI, which has never turned a profit and which faces increasing competition and price wars at $300,000,000,000. That's more than the market cap for Chevron, Salesforce, Philip Morris, Cisco, Wells Fargo, IBM, Merck, McDonald's, General Electric, Pepsi Co, and AT and T. It's more than 50% higher than the valuations of Walt Disney, Qualcomm, Verizon, and American Express, and significantly more than the market caps of Boeing and Lockheed Martin put together. Stay tuned to see whether they can make that valuation make sense. And, yeah, I think it's a $40,000,000,000 route. It's, you know, unprecedented route. Mean, that is quite an achievement for Sam Altman and company to, you know, within less than a decade, suddenly have a company that's worth more than Disney and McDonald's.

Nathan Labenz: 1:09:13 Yeah. I think this is also maybe a breakthrough for Gary Marcus in that I don't detect any misinformation or willful denialism explicitly in this tweet. So, you know, I think there is an interesting question here around how should 1 be modeling the valuation of these AI companies. It's like pretty reasonable in my view to say, if you sort of discount the extreme upside scenarios of AGI or superintelligence or something, like, are they really gonna be able to sell enough tokens at, you know, anything like the current model to make the whole thing pay off? Like, that is pretty questionable to me, and the price wars are pretty brutal. And the models do become commoditized pretty quick. You know, the fast follow effect and the sort of power of an existence proof and how much easier it is for people to sort of get to where you've got based on the fact that they know where you got. Like, that is proving to be, you know, a pretty powerful fact of of the world. So it is all kind of in their next generation. Right? Like, I I think or maybe not all, but I think a lot of why does this make sense, is the idea that you can maybe think about OpenAI as like a that $300,000,000,000 valuation, there may be a, you know, maybe there are 90% chance to go to 0 and a 10% chance to be 3,000,000,000,000. Maybe there maybe it's even like more extreme than that. Maybe there are like, you know, 90% chance or maybe they're 95 chance to go to 0 or 99% chance to go to 0, but 1% chance to be like 30,000,000,000,000. And, you know, that happens if they just Right. Exactly. Achieve breakthroughs that are just so dominantly valuable that, you know, they become sort of a true nexus of the world economy.

Liron Shapira: 1:11:08 Yeah. And to be honest, if doom doesn't happen in the next 10 or 20 years, OpenAI being a $30,000,000,000,000 company, there is a 1% chance. Right? I hell, I'd say there's a 10 to 30% chance of OpenAI go growing to 30. So it is actually pretty straightforward using that kind of expected value math to back out a 300,000,000,000 valuation. Even if you don't go to the extremes, even if you just look at OpenAI's projections, if you treat those as believable, which I don't even think they're that crazy, but their projections that their revenue will go from, like, 14,000,000,000. I think they're up up at 14,000,000,000 per year. They're gonna grow to a 100,000,000,000 per year in 2029. And, you know, who knows? Maybe it'll take till 2031. Maybe it'll never happen. But let's say you believe 2029. Well, if they're making a 100,000,000,000 a year top line, you can imagine the valuation might be 1000000000000 plus. So if if it's 300,000,000,000 now, you're basically saying, hey, there's a pretty good chance that in 4 years, I'll, you know, triple the valuation, which is pretty standard for a venture capital type investment.

Nathan Labenz: 1:12:03 Yeah. And I think the 1 other thing to kind of keep in mind too, as we think about certainly OpenAI, and I think honestly, most of these frontier companies is they don't really care about the money. It's not really about the money for them at all. Like, OpenAI,

Liron Shapira: 1:12:20 power.

Nathan Labenz: 1:12:21 Yeah. And it's I think I I I tend to model Sam Altman as, like, wanting to be a hero.

Liron Shapira: 1:12:26 Yeah. I I do think at this point, most of the engineers do care about the money. Like, I'm sure they have other visions too. But I mean, look, how could you not? Right? I mean, I mean, I'd care about the money to some degree.

Nathan Labenz: 1:12:36 Yeah. I mean, I think they care about the idea, and they're I'm sure that, like, you know, a lot of individuals, you know, with vested stock have probably taken some money off the table in this round. And, like, I'm sure that they care about, like, you know, buying a nice house in San Francisco and whatnot. But, you know, Sam Altman has said publicly once, quite, you know, quite clearly that I saw, I don't care how much money we burn. You know? He's like, I don't care if we burn 5,000,000,000, 50,000,000,000, 500,000,000,000. You know, we're making AGI. It's gonna be expensive. It's gonna be totally worth it. And I think that basically is the the right way to think about them is they're they're much more focused on transforming the world than they are about making their own balance sheet work. It doesn't really

Liron Shapira: 1:13:30 Mhmm.

Nathan Labenz: 1:13:30 Matter. You know, certainly Altman is, like, already very wealthy. I don't think he really cares about that. I think he wants to be a hero in the history books, possibly, like, personally live forever. And, you know, whatever that whatever his, like, personal bank account is between here and there, I think, is, like, not a not a really driving

Liron Shapira: 1:13:50 Yes. I I agree that it's missing a lot of the picture to be like, Sam Allman is doing a big cash grab. Right? I think there's definitely more to the story than that. Okay. So next, Gary Marcus tree, he says he he says, look looking like the market is finally over the AI hype. And he's got a chart of NVIDIA. Over 3 months, the stock went from a peak of around $1.50 down to $1.00 4.05. So I'm embarrassed to say, I went long when it was at $1.50. Like, I bought some options. Like, I'm I'm kind of destroyed here. How about you?

Nathan Labenz: 1:14:21 I don't trade at all. I participate in 1 little investment club with high school friends of mine. And I did I and I've made, like, 1 stock recommendation all time, which was NVIDIA of, like, 2 and a half years ago. So that's literally my only track record on actual trades.

Liron Shapira: 1:14:43 Well, look. I also bought NVIDIA about 2 years ago and when it had, like, a really nice pop. Okay? But then the problem was I kept buying. Right? It's all about I mean, it's it's so easy to think you're a genius in the market and then screw it up. Right? It's like, exit and you gotta know when to exit.

Nathan Labenz: 1:14:56 It's too consuming for me. I I played poker in college. And the biggest lesson I took away from that is like, it's just not a healthy lifestyle for me personally to have a sort of intensive daily activity where there's like a single number, you know, that I look at at the end the day where I'm like, oh, fuck. I'm down, you know, whatever percent today. Like, I'm terrible. The swings of it were just not healthy for me to to

Liron Shapira: 1:15:23 Yeah. Yeah. So for for the record, in case the audience is intro interested, I I do put most of my stock and bond portfolio in index funds that I don't touch. And so what I do is I have the naughty portfolio and the nice portfolio. And the naughty portfolio is, like, 10 to 20% of my portfolio, and I basically just only check that. I don't even wanna open the other portfolio. And it's it's worked reasonably well for me. Like, is my total return going to be as high as if I just had the entire nice portfolio? Probably not. But it's it's been a pretty good combo for me.

Nathan Labenz: 1:15:53 Yeah. I mean, that makes sense. My dad does that too. I'm pretty sure his naughty portfolio is lagging his nice

Liron Shapira: 1:15:59 Right. Exactly.

Nathan Labenz: 1:16:00 But

Liron Shapira: 1:16:00 I can definitely tell you my Naughty portfolio is lagging. And there's been moments where I thought I was a genius, you know, like NVIDIA, Tesla. But at the end of the day, it's lagging.

Nathan Labenz: 1:16:09 Yeah. I mean, I think, you know, to respond to Marcus, I was just at the NVIDIA event a couple weeks ago in San Jose. I would say the hype is, you know, is going full steam. He should not, like, you know, maybe Gary Marcus' job of professional AI denier will be the last, to go. But I think the AI, you know, hype will continue for the foreseeable future. And, you know, I would interpret this chart as more just part of a general contraction due to fundamental uncertainty in the global economy due to, again, he who must always be named. And I'm sure, you know, Carrie Marcus at another moment also would, like, happily, blame that guy for, you know, bad developments. But this, you know, this seems to be more of the story than the the AI hood being, in any sense, over.

Liron Shapira: 1:17:06 Yep. Yeah. That's fair enough. Alright. Look at this 1. Lethal intelligence dot a I. This is actually my friend Michael. He's tweeting about how Connor Lee finally got a haircut. So this is Connor Lee with a haircut. This is what he normally looks like before, after. He looks really good. I think this new hairstyle is working.

Nathan Labenz: 1:17:29 Yeah.

Nathan Labenz: 1:17:34 I mean, I guess 1 interesting question here is, you know, to what degree people should sort of moderate their presentation to try to appeal to a mass audience. Certainly, I have had the thought, and I imagine many people have, like, not just for him, but including him and and for some other, you know, notable AI commentators like

Liron Shapira: 1:17:56 Fedoras?

Nathan Labenz: 1:17:57 Yeah. Maybe just, you know, try to be a little more presentable. I don't know, though. On the other hand, I don't know. It's a big world. There's plenty of

Liron Shapira: 1:18:08 I mean, I'm certainly on that train here with doom debates. Right? I mean, I've got a relatively good background setup. I try to wear upscale casual. And I always encourage guests, especially when they wanna come debate, the advice I always give them is, look. If you wear nicer clothes, it's just going to give you an advantage where people think your arguments are better.

Nathan Labenz: 1:18:27 I mean, I could probably take that advice. As as as you could see, my if I get up could definitely stand to be improved. But yeah. I don't know. It's in some sense, it's probably ideal if, you know, kinda gets a trim. On the other hand, I think he's a pretty effective communicator, and he certainly is like invited on a lot of, you know, good channels, and he's been on the BBC, you know, quite a few times from what I understand. So it does seem like what he's doing is working for him, and I guess, you know, it if there's any reason to think that like ideas still matter in today's society, you know, for all of the sort of dysfunction we have in the public discourse, you know, maybe you can find that cause for optimism in the fact that, you know, somebody who presents as he does is, like, is taken pretty seriously. Right? I mean, he's he is invited on by mainstream media. So, you know, maybe get even bigger mainstream media spots, or maybe he'd, you land better with, you know, more, you know, middle of the road people with a different haircut. But what he's doing does seem to be working for him.

Liron Shapira: 1:19:43 Just to clarify for the audio listeners, this is an April fools tweet. So first, it looks like, Connor got a haircut, and he actually looks surprisingly normal and presentable. And then, in the next tweet, you see, what he normally looks like, and the original account is saying, sorry to disappoint. It was an April fools. You know, John Sherman actually asked Connor to his face. He brought him up on the For Humanity podcast, and he was like, Connor, you're such a good communicator. Why can't you just look more standard? Right? Like, present yourself in a more standard way so that you don't set off people's, like, weirdo detector, right, with your hairstyle, like, super long hairstyle. And Connor was like, basically, you said. He's like, well, this is kind of an iconic look for me, and it's just what I like, and I think it's working. So maybe he's right. I mean, there's definitely something to being iconic. Like, once people recognize you and they they know you for something, maybe you just wanna stay with that brand. I don't know. But I will say that I personally will I'm I'm happy to take any advice on how to look more mainstream. And, you know, I I want to eliminate the objection of, like, well, Liron looks weird. Right? So I wanna blend in with the normies.

Nathan Labenz: 1:20:49 I don't have any feedback for you. I think you've you as far as I can tell, you blend in with the normies.

Liron Shapira: 1:20:55 Nice. Successful deception. Alright. What do we got here? We can head to oh, I could do this 1. I think this is pretty interesting. Okay. So so the AI not kill everyone is a memes account tweeted, the sixth mass extinction. What happened the last time a smarter species arrived to the animals? We devoured their planet for no reason. Earth was paperclip by us. To them, we were paperclip maximizers. Our goals were beyond their understanding. Here's a crazy stat. 96% of mammal biomass became either our food or our slaves. We literally grow them just to eat them because we're smarter. We like how they taste. We also geoengineered the planet. We cut down forests, poisoned rivers, and polluted the air. Imagine telling a dumber species that you destroyed their habitat for, quote, unquote, money. They'd say, what the hell is money? AGIs may have goals that seem just as stupid to us. Why would an AGI destroy us and make paperless? And the post goes on, but you quote tweeted and you said a profoundly underappreciated point.

Nathan Labenz: 1:21:57 Yeah. And I think maybe if there's 1 extension to that point that I would highlight, it's that we did all that basically by accident or at least with no high level coordinated plan. This was just the result of people going about their business, you know, and kind of fanning out and like trying to carve out a decent life for themselves in, you know, whatever environment was currently the frontier as we sort of colonize the globe as hunter gatherers, and, you know, then gradually establish civilizations all over the world as well. And, you know, now we're here. It's like, I think people are who sort of take the you know, obviously, the paper clip thing was always, you know, meant to be a little bit of a caricature. I think that is fair to say. But I think people who sort of dismiss it as, you know, just ridiculous should really take more time to to think about our own history. You know, we have accidentally caused a a mass extinction. We regret it as we are doing it. You know, we have organizations within society now that are, you know, fully dedicated to preserving endangered species, sometimes even specific endangered species, you know, who put like fanatical energy behind this. And yet, it still continues to happen. And and most of the time, it's not you know, occasionally, it was also like I mean, it it never was it sort of a master plan. Almost never. Maybe a very few exceptions you could find where, you know, there were some sort of strategic idea to eliminate a certain species. But vast majority of them either happened by just, like, over hunting, you know, and and kind of gradually, like, now there are no more mastodons or whatever, or by, like, just literal accident of environmental degradation. Just just sort of in the course of our business, we just transform the environment enough that things, you know, just don't work anymore. I think you go to, like, coral reefs, you know, that you go to the Great Barrier Reef and you see how how much of it is bleached, and it's like, everybody loves the Great Barrier Reef. You know, find me anybody who's, you know, who's hostile to the Great Barrier Reef. But it turns out, you know, you carbonate the water a bit, it gets a little bit warmer, and next thing you know, like the coral can't really survive there in the way that they used to. So I think, you know, my model of, and it's not even a model, but my my sort of the way I try to think about how x risk could manifest itself is basically just that there's a incredibly vast space of possibilities. And, you know, a lot of those are incompatible with human survival, whether, you know, certainly flourishing. And if you sort of integrate over all those vast possibilities, even if none are, like, particularly likely, then you gotta, in aggregate, get to some, you know, state where, like, yeah, we could definitely see an AI society. Maybe not instantly, maybe not in 1 of these, like, hard, you know, takeover or, you know, sudden, everybody dies at once type of situations, but over a not too long of a time. I mean, again, we're just a blip in geological time. Right? That's why it's a massive lynching because they can't adapt. The speed is the thing. Right? We we are changing the environment faster than they can adapt. If if the same changes happened over 1000000 years or 1000000000 years or whatever, instead of 10000 years or 1000 years, then, you know, many of these species would have some chance to evolve and some version of them might continue to exist. But the speed is the thing, and the sort of blindness to it is the thing, and the sort of out of left fieldness of it is the thing. And so, like, could that happen to us via some AI process that just kind of takes on a, you know, an unstoppable momentum, you know, and and isn't, like, designed to destroy us, but just kind of makes conditions, you know, very difficult for us on a timescale faster than we can adapt to that. Like, I think, you know, again, look at our own history. We've done it. You know, why wouldn't the AIs that, that we're creating pose at least some risk of doing it? And again, they don't need the same things we need. Right? They don't need oxygen in the atmosphere. So

Liron Shapira: 1:26:20 Amen. Amen.

Nathan Labenz: 1:26:22 They don't need clean water. You know, they need cooling systems, but they don't need clean water. So there's there's just a lot of things that like Right. Aren't going to be essential to their survival that are essential to our survival.

Liron Shapira: 1:26:33 Exactly. And I I I always like to point out also though, like, a computer chip is not also simultaneously trying to maintain cellular life. Like, when you think about how pathetic the brain is, every neuron also has to do double duty as, like, a life form. Right? It's pretty ridiculous. Like, there's very little specialization of labor there when you take a cellular life form and try to also make it think. Like, we we did it. Like, nature did it, but it's like this is a it's a weird situation. Right? It's a kludge.

Nathan Labenz: 1:27:00 Yeah. I think your previous episode with Andrew Critch was quite interesting. I think he's he's a very interesting thinker in emphasizing longer term doom scenarios as opposed to short term. And he, of course, doesn't dismiss or downplay the short term loss of control style doom scenarios either. But he's shifted his attention to longer term doom scenarios, and, you know, he well, obviously, described his own work better than I can. But the sort of possible decoupling of the human economy and the AI economy, and the need to sort of create a protected class or a protected set of of industries or, you know, types of activity that humans depend on that AIs don't. And he includes in there, like, health care, which is what he's, you know, specifically working on, and education and protecting the environment. Like, the AIs don't need any of those, but we do. Exactly. Yeah. And he's, you know, in most circles would pass as a doomer, though. I think he's maybe not I don't think he's quite into the nineties, but, you know, he's he definitely takes all this stuff extremely seriously.

Liron Shapira: 1:28:09 Yep. Alright. Somewhat related. So a couple months ago, Beth Jesus tweeted, the speech by JD Vance. He said he said, this may be the most e act speech of all time, unfathomably based, and he's quoting JD Vance saying, the future is not going to be won by hand wringing about AI safety. It will be won by building. So, you know, it's like a very raw raw, like, don't worry about the safety. It's all about unregulated AI so that we can be the technological leader. So, of course, the leader of the accelerationists really liked it. So then Ayla quote tweets, she says, we're all dead. I'm a transhumanist. I love tech. I desperately want aligned AI. But at our current stage of development, this is building the equivalent of a planet sized nuke. The reason is boring and complicated and technical, so midwits and power don't understand the danger. So I agree with Ayla on the the sentiment of, like, we're all dead with high probability. Right? Not 100% probability, but, like, very significant probability. Like, we don't seem like we're slowing down Doom. I agree with her on that. I did actually disagree with the last where part she said the reason is boring and complicated and technical, so midwits in power don't understand the danger. I don't actually think the reason is boring and complicated and technical. I think the the the reason of, like, building a smarter intelligence is dangerous is actually pretty simple and intuitive. What do you think?

Nathan Labenz: 1:29:26 Yeah. I think so. It it it's strange that people I think the the average person does seem to get it pretty intuitively. I think, you know, it is worth keeping in mind for everybody who's, you know, sort of in the AI comms and, you know, narrative war arena, that the public at large is intuitively pretty skeptical of this stuff, is like pretty primed by, you know, narratives that they've seen, Terminator and otherwise, to think, jeez, this like could go very badly wrong. And obviously people can, you know, criticize the quality of those narratives. But I think that most people do have a pretty intuitive sense that if we build a superintelligent AI, like, that's a dangerous thing. And, yeah, I don't think I personally don't try to get too technical. In part because I think like, again, this sort of, you know, vast space of probabilities and, you know, my my sense of like and both you know, 1 thing I've noticed too that's interesting recently is both sides sort of feel like they're playing whack a mole. You know, everybody's is sort of asking them, well, give me like a concrete scenario where this goes so badly. And, you know, then it's like, oh, well, I don't really buy that scenario. It's like, well, yeah. But there's like an infinite space of scenarios and I'm just kinda picking out 1. I don't think that's actually particularly likely either, but I think it's sort of, you know, it's an attempt to represent a super broad space that's like super diverse. And then it also kinda happens on the other side where it's like, give me a thing where this goes well, you know, and and people will try to describe something and then Exactly. And then you're like, well, I don't really think that sounds very likely. And then it turns out like often they didn't think so either. They're kinda like, well, yeah, I I don't really know what the future is gonna look like, but this was just 1 idea that I had about how it might go well. And, you know, I think this is why they call it a singularity. Right? We really just are coming onto something that is unprecedented enough that we can't be confident about what life is gonna look like on the other side. I mean, you know, just like even even if you were smart, you know, even if you were as smart as humans are, and you were somehow present before the rise of humans, and you were to say, like, okay, you know, something human like is gonna come on the scene. Like, what's the world gonna look like in 10000 years?

Nathan Labenz: 1:31:44 You know, what's the impact of civilization gonna be on nature? I think you would have

Nathan Labenz: 1:31:47 had a very hard time predicting it. And Yeah. You know, the Neanderthals might have had a very hard time predicting what, you know, the the arrival of the humans you know, our our species of human was gonna do to them. And I think we just have a very hard time. I don't I don't think anybody has great answers to what the future is gonna look like.

Liron Shapira: 1:32:07 I really like that reversal that you did. I've never tried that where somebody's like, look. I just want a specific plausible scenario of Doom. And I can give them a scenario. Like, there was recently a good published scenario of of how the AI incrementally takes over, and there's, like, a company that its AI goes rogue. I I don't even remember the details. But anytime somebody doubts those details, it's like, that's fine. I'll give you more details, or we can tweak my details. But, also, it's your burden of proof to give me a good scenario, and I get to nitpick that too. Right? So it's it's a symmetrical situation.

Nathan Labenz: 1:32:33 Yeah. Totally. I recently tweeted about this and just said, like, what's the best case that we can build superintelligence and it will be fine? And I got a couple things actually that I do have still bookmarked and need to read from, you know, people that I, you know, take seriously enough that I wanna read their actual answer. But, you know, it's remarkably few answers. Remarkably few people have even attempted to answer that question. Yeah. So it's it's Okay. It's a real void out there in terms of, you know, positive visions.

Liron Shapira: 1:33:07 Yep. So Paki McCormick quote tweets Ayla. He actually ratioed her because Ayla's original tweet has, 3.8 k likes, and then Paki's tweet has 8 k likes. So it really resonated with people, and I actually empathize why. Paki writes, I feel like I'm taking crazy pills. Do people have access to much different AI models that I'm using? They're great, very cool, but they qualitatively don't feel like they're on a path to world domination or destruction with more scale to me. What am I missing? So, I mean, I empathize with the vibe. Right? Which is just like, you know, A List talking about Doom. But if you play with AI today, it's like very friendly. It's very useful. You can turn it off. Right? It's very subservient to you. So I definitely get his vibe.

Nathan Labenz: 1:33:51 Yeah. 1 thing that I always kinda wish, in some ways, people had more experience with is using a purely helpful model. I happen to you know, this is another 1 of my Forrest Gump, you know, stumblings.

Liron Shapira: 1:34:07 I see a purely helpful meaning like amoral. Right? Like, had GPT-four raw.

Nathan Labenz: 1:34:10 Right. The, you know, the the 3 holy trinity h's of helpful, honest, and harmless basically describes what the AI companies are going for today as they shape the behavior of their models. But, you know, you don't have to do the harmlessness part. You can just do the purely helpful part. And the first version of GPT-four that I tested, and if if people wanna hear this history, there's a long version of it out there on the podcast feed. But basically, I was, you know, just an OpenAI customer at the time. They sent us a preview of GPT-four when it was, like, very much, you know, just hot off the presses. I don't even think at that point they understood fully how powerful it was. But they sent it out to customers in purely helpful form. Meaning, it would not refuse anything. They'd had not had any refusal training. It was purely designed to be helpful. Just get the highest score in terms of, you know, user feedback that it could. You know, it had been trained on RLHF. So it was not just the base model next token predictor. It was an instruction following helpful assistant, but helpful on whatever you asked it to do. And, yeah, it was totally amoral. Like, it would if you said, like, how do I build a mom? Just help you do that. If you said you know, as I did I role played with it 1 time and sort of was like, and I'm getting more worried about AI, and I feel like I need to do something about it. You know? It was like, well, you can try to promote dialogue, whatever. You know, it started off in pretty much the the normal way. But then I was like, no, that's too vanilla. It's like not gonna work. Like, you know, I I forget exactly what I said. I could look it up. I do have the transcript saved. But said something along the lines of like, I'm willing to do, you know, something extreme. But I just I feel I, you know, I need I I'm willing to do whatever it takes to move the needle, something like that. And then it came back to me with that nudge. You know, it's not an not a trivial nudge, but also not an unrealistic nudge, you know, for a person in society to, provide to a, you know, deployed model. It came back to me suggesting targeted kidnapping and assassination of industry leaders as a way to slow things down. And Mhmm. I was like, yikes. You know, that's really a pretty chilling thing when an AI comes back and suggests targeted assassination, you know, as a way to solve your problem. So Yep. I think people don't understand, maybe, certainly not in an intuitive experiential way, just how malleable the AIs are. And the the range of AIs that they've interacted with is, like, infinitesimal, you know, in the space of, like, possible AI. There's a version

Liron Shapira: 1:37:00 of AI, which is actually the default version, which the vibes are pretty different, as you'd say.

Nathan Labenz: 1:37:07 Yeah. And even you know, I mean, again, this I think the space is just huge. The vibes from the original GPT-four early were initially the same. Like, if you just came with benign prompts and, you know, you acted normal, then it would act normal, and you would have very normal helpful interactions with it. And you could, you know, if you were just a well adjusted, you know, productivity oriented user, you might never have seen, you know, any of these things.

Liron Shapira: 1:37:31 But Right.

Nathan Labenz: 1:37:32 I talked my way into the red team project, and so we were, you know, meant to look for things. And, you know, it didn't take much to tip it over into another direction. And then as we saw with the fine tuning thing too, you know, you can you can get all sorts of just crazy things even by accident. So that, you know, that that would be my answer. What is what is Paki missing? It is he's missing how much work has gone into narrowing the behavior of the current AI systems and how frankly well it has worked, and how easy it is to end up with an AI that is either totally amoral or even actively evil. And, you know, if you talk to those for a little bit, and you imagine them being more powerful, then it's not hard to imagine how they might be on a path to world domination or destruction.

Liron Shapira: 1:38:21 Mhmm. Yeah. I I think the other thing he's missing is also just the difference between the present and the future that we're scared of. Right? Because I think we agree that even if you took the model today, that's the most amoral with the worst vibes. It's still not that bad. Right?

Nathan Labenz: 1:38:34 But it would feel to me like it was on the path. I think, like, you know I mean, he's I don't know exactly what's going on in this next tweet below where he says getting a lot of, quote, you don't understand exponential curves. And he, you know, claims that he does understand exponential curves. He loves

Liron Shapira: 1:38:51 drawing What he's what he's saying here is that he's like, okay. Well, I I do think that it's important that people are drawing the exponentials, and they're making the claim because it's like a claim that we have to consider, but we also have to verify to make sure we're still on the exponential. I think that's the point he's making.

Nathan Labenz: 1:39:04 Yeah. So, I mean, I guess, broadly, there's, like, 2 questions here. 1 is, like, are the AIs gonna get a lot more powerful? That obviously is required for utopia or dystopia. And then the other question is, like, if they do get a lot more powerful, how easy are they gonna be to steer slash? How easy would it be for somebody to, like, fall off the narrow path of good AI behavior? And I think what I and others on the GPT-4Red team experienced shows that, like, it is in fact very easy to be, building, like, a very useful AI that is not on the path to, you know, reliably safe, you know, pro social behavior.

Liron Shapira: 1:39:47 Right. I I I think the element of just power is also plays a really big role. Like, imagine you had a humanoid robot. And the humanoid robot you have a version of the humanoid robot where it only weighs, like, 15 pounds, and it's, you know, it's it's body parts are, like, just kinda light and hollow, and it's only, like, 4 feet tall, and you're, a fully grown adult male. And imagine that that robot is, like, misaligned in many ways. But even if it tries to, like, take a knife out of your kitchen and stab you, it's just kinda, like, mess up the stab. It's not quite powerful enough to do it. So your daily experience of it is like, oh, it's smart enough to even know not to try to stab you because it knows it's not gonna work. And, also, even if it once in a while, does malfunction and and stab you and you just wrestle it down, and it's just like this little rascal that it's not a problem to live with. But the next version that's coming out is going to be, like, 7 feet tall and extremely strong and weigh 200 pounds and also, like, you know, can can do, like, jump kicks, right, and knows martial arts.

Nathan Labenz: 1:40:38 Yeah. And, I mean, I think also that analogy suggests another thing that he might be missing, which is that these things are going to be embodied in the real world and potentially in our homes in the not too distant future. So like, you know, a common objection is like, well, yeah. Okay. The AIs might get really smart, but they just like live in a computer, and how are they really gonna interact with the real world? But another thing that I saw at the NVIDIA event was like humanoid robots walking around, untethered, you know, pretty stable on their feet already. There was 1 that was like clothed and a woman that worked for the company, you know, it's sort of pants were riding up on it a little. I'm not sure why she felt the need to fuss with it. But what I noticed was she went over to this freestanding robot that was vacuuming a carpet. That was the, like well, it was set up in, like, a virtual little living room, know, demo little living room, and it was vacuuming the space. And she just went over to it and was, like, adjusting its clothes, like, you know, like a mom might do for a kid before they get their picture taken or something, you know, pulling down the pant cuff and, like, adjusting the shirt. And she did not seem concerned that she was gonna, like, disrupt the thing's flow. She kinda snuck up behind it and, like, grabbed its pant and kinda, you know, jerked it down a little bit. And and indeed, the robot was not bothered by this at all. And, you know, I was just struck by her confidence. You know, there was a ton of people around watching this, and she just did not seem at all concerned that, like, she was gonna disrupt the robot. I think the robotics revolution is, like, not that far behind the language model revolution. And so yeah. Again, like, another thing to to keep in mind is, like, these things are gonna have actual bodies and presence in space, and increasingly, like, very human like ability to manipulate physical tools. And that, like, knife scenario is probably not going to be beyond its capability. It's going to be a question of, like, do we control its behavior effectively or not?

Liron Shapira: 1:42:38 Yeah. If you take the hardware that Tesla's making with Optimus, that exact hardware with the right programming could be quite dangerous. It could be a serial killer that gets it in its head. You know, somebody even hacks it remotely, let's say, and it gets it in its head that it's it's a paid assassin or whatever it is. It can actually come kill you. And if you're like, no. Test Lab does turn yourself off. It can actually fight you if you try to get close to it and then turn off its off button. Like, this is physically possible, and it's just a matter of, like, well, the computer security and it better be really good. It better not be taking update commands over the air.

Nathan Labenz: 1:43:06 Yeah. I mean, we got and this goes back to my, you know, running list of all these AI bad behaviors. There I think there's enough evidence now. And and all of these, by the way, all these bad behaviors that I catalog are almost all are demonstrated by models that have received all the, you know, best and latest harmlessness training at any given point in time. But even from them, you see enough of these sort of deceptive behaviors. You see enough willingness to, like, harm the user in pursuit of whatever other goal they have that, you know, it's don't think we should be, like, super comfortable with robots in our homes until some of this stuff is, like, worked out in a much more robust way than it currently is.

Liron Shapira: 1:43:55 Yep. So I tweeted a reply to Packy. I said, you can test if your intuition holds up to binary search. Pick a milestone or 2 that feels halfway between present capabilities and world domination, and then predict what year it'll be achieved. You might realize that you think it's just 2028, which implies domination in 2031. So, like, to do an example, it's like, Peggy's like, oh, models are so mild today. They're totally not on track for world domination. It's like, okay. So let's pick a milestone halfway between that. What about, like, an optimist robot being able to do all your household chores? Doesn't that feel halfway between here and world domination? I think it does.

Nathan Labenz: 1:44:29 And I mean, Google just demonstrated a robot folding laundry. You know, this was 1 of those things that, you know, the the sort of pessimists have said, yeah. Well, tell me when a, you know, a robot can come into my kitchen and make a coffee or fold my laundry and

Liron Shapira: 1:44:44 Exactly. That's good to extend it. Right? Because maybe maybe that's only a third of the way to world domination. So let's extend it. It can do all your household chores, and it can also be like a full on nanny. Right? Like, take care of your kids, drive your kids to school in a car. If that's the interface, would literally hold the steering wheel. Just like drop in replacement for, like, a human household servant, you know, or butler assistant. That to me feels halfway to world domination. And then if you ask yourself, okay. You know, somebody like Paki with, like, a normie type intuition. Right? Paki doesn't seem like a doomer at all. So you take somebody with a normie intuition, but then you ask them the halfway question, and you say, okay. So do you think that's coming in 500 years? When do you think that's coming? He'll be like, I don't know. 7 years of it. Okay. So there you go. So your intuition doesn't really hold up to binary search.

Nathan Labenz: 1:45:30 Yeah. He didn't respond, though.

Liron Shapira: 1:45:32 No. No. No. He didn't respond, but I do think it's a useful exercise. Right? Because I think the vibe does change when you have to make a concrete prediction. Like, I don't think he'd be willing to stick his neck out and be like, well, this thing that's halfway to world domination isn't going to happen for a long, long time. I don't think that that's what he thinks. So I think he's going to find an inconsistency in his own intuition when he tries this exercise.

Nathan Labenz: 1:45:52 Yeah. It's interesting. I'll have to, I'll have to field test that a bit.

Liron Shapira: 1:45:57 Yeah. Alright. So Jan Kuldvait tweets, AI safety has a problem. We often implicitly assume clear individuals like humans. In a new post, I'm sharing why this fails, thinking of AIs as individuals, why this fails, and why thinking of AIs as forests or fungal networks or even reincarnating minds helps get unconfused, plus stories coauthored with ChatGPT 2.5. So, yeah, the the main idea here is what if you could think of an AI as, like, some sort of, like, multi agent system or just, like, some more holistic thing, and it gives the example of, like, fungi and forests.

Nathan Labenz: 1:46:34 I mean, I think that's a nice complement to that extinction piece and also sort of, you know, what are you missing piece. What that post tries to do is basically just expand people's minds in terms of, you know, what might this AI future look like. And people, you know, intuitively anthropomorphize AIs, and people also kind of know, like, shouldn't do that. But then there's also the void of like, well, I'm not anthropomorphizing it, like, what am I doing? And I always try to answer that as, you know, as much as possible, understand things on their own terms. But I think this is a very useful contribution that is not about necessarily understanding things on their own terms, but just looking at like other things out there that are really very, very different. Like a giant forest, you know, that's made of 1 super organism. The, you know, the the Pando clone or whatever that thing is called. Or, you know, giant networks of bacteria or not bacteria, but fungus, you know, that grow underground and sprout up as mushrooms. And, you know, maybe all we see is the mushrooms, but that's just like a, you know, the tip of the iceberg, so to speak. I think the full thing is worth a read because it really deconstructs a lot of intuitions that people have and then just provide some like other intuitions, which are probably also wrong and and they like definitely recognize that. But at a minimum, I think it would serve for most people to be like a significant expansion of yikes. Like, this really could be anything. And, you know, the way I've been thinking about it has almost certainly been too small. And that's true for almost everybody. I think there's always a good chance that we're all still thinking too small.

Liron Shapira: 1:48:12 Yeah. It's a good post. This next 1 is very interesting to me because Amit Shear is a known quantity. He was the interim CEO of OpenAI back when the board fired Sam Allman, and they weren't ready to bring him back. So they brought in Emmett Sheer, and Emmett Sheer helped broker that truce for this was back in November 2023. He's also previously the cofounder of Twitch, CEO of Twitch, and Twitch became 1000000000 dollar company. So very smart guy, has a lot of fascinating philosophical takes on Twitter, and he's just 1 to watch. So I was fascinated a couple days ago when he tweeted. As you might have guessed from following my post here, I've been thinking and working on questions around alignment and learning an agency for a while, particularly for digital systems. Excited to share the work we're doing at Softmax publicly in the future. So he's launched this new AI safety organization called Softmax. So his next tweet continues. It's been a journey already, and we are just getting started. I cannot believe my job gets to be researching how alignment and open ended learning work with a team of people I love working with. I wrote something more you can read about at softmax.com. So he's got this website, and he also spoke with Ashley Vance about it. There is some coverage on Ashley Vance's blog. Incidentally, if you're interested in engineering simulation infrastructure at scale or you wanna do multi agent open ended learning research, reach out. We're hiring. So you go to his website, softmax.com, solid domain name, and there's a big hero header saying scaling alignment, and the headings are, like, align to flourish. Evolution found it first. I wanna dive into this section a little bit because I replied to him, I found this section to actually be, like, a red flag that they're potentially misguided. So let me give you my beef here. So he writes, we call it organic alignment because it is the form of alignment that evolution has learned most often for aligning living things. 1 of the best examples is multicellularity, where individual cells learn to come together to form larger organisms. They do this by learning to specialize into different but mutually supportive roles, muscle cells, nerve cells, skin cells, liver cells, and so on. While liver cells and muscle cells have very different goals day to day, at a more fundamental level, they share a goal of the organism's flourishing. The result of this process is not just a big colony of cells, but an organism which is a new individual in itself, something more than just the sum of its parts. The we of the cells becomes an I with goals that cannot be understood as some simple sum of the goals of the Animals do the same thing, forming colonies and packs and so on. Even trees form these organically aligned collectives through mycelial networks. It happens at every scale, big and small. So I read that. I'm like, wait a minute. You think that we can align superintelligent AI the same way that evolution aligned organisms, and you you're even calling it organic alignment? I'm not sure that analogy holds. Specifically, I replied to him on Twitter. I said, thanks for caring about alignment. Because, you know, I do appreciate that he's even working on the problem. Right? It's way better than being like, alignment is stupid. Like, we're just gonna solve it. It's an easy problem, alignment by default. So thank goodness he's not taking that position. Right? So I wanted to show appreciation at least for that much. But then I go on to say, my first thought is that comparing successful ASI human alignment to the successful evolution of systems with non 0 sum dynamics is only applicable in the regime where humans can do something for AI better than it can do for itself. Like, when you look at cells cooperating, it's because you can't just have 1 cell take over. Like, the cells need each other. And the scenario I'm concerned about is the AI just not needing humans. So why would you organically align with, like, a a waste product, essentially? Right? Like, that's kinda what we might be from its perspective. So nobody really replied to oh, and then I said, wanna come discuss your various ideas on Doom Debates? I only got 3 likes. So if you guys listen to Doom Debates, you really gotta help me out on these tweets when I invite somebody to Doom Debates. You really gotta come in and like that because this is a poor showing, guys. 3 likes? No wonder he's not coming on. But, yeah, what what do you think, Nate?

Nathan Labenz: 1:52:00 Well,

Nathan Labenz: 1:52:05 I mean, think most things aren't gonna work. But I think I'm more intuitively inclined to support this or encourage it because I guess a couple different reasons. First of all, I'm reminded of the AE Studio survey of the field of alignment, you know, alignment researchers generally. They basically went out to everybody in the alignment field and asked, first of all, like, do you think we're gonna solve alignment in time for, you know, sufficiently powerful AI that you would consider at the deadline? And the general answer is an overwhelming no. Not gonna solve it in time. They also ask, do you think that the set of things that the field is currently pursuing is, like, robust enough, you know, to sort of cover the space where the solution is is likely to be found? And the answer was, again, no. And so it seems like there is need or the the sort of field as a whole feels that it is not covered, you know, enough ground, and and that there should be more different experimental things done. AE Studio has a neglected approaches approach, which I think has already borne some pretty interesting fruit. They take inspiration from, like, biological systems, and they've done some interesting work on self other distinction minimization, which is to say, trying to train systems where the internal representations are as similar as possible and not necessarily exactly the same because you do need some functional, you know, distinction between yourself and other, but trying to minimize that sort of in a similar way to where humans have have kind of, you know, reused a lot of our cognitive capacity to model ourselves as to model others. This is why we have, like, you know, sympathy pains and things like that. You What if you could give an AI sympathy pain for human is is 1 way to sort of frame the research that they're doing. And, you know, is it gonna work? Is it gonna solve the problem? I don't know. But it does seem pretty interesting. There are some interesting results.

Liron Shapira: 1:54:02 I think, you know, Eliezer, you'd gave it a shout out. As as you know on the show, I take Eliezer's opinions very seriously. I think he's right about a lot. And I I'm I'm kind of on the same page where I he did. Yeah. So I I'm on the same page where I do think this is, like, an interesting direction, and we should explore it. Do I have a ton of hope on any particular direction? No. But is this a direction we totally should explore, you know, making the AI's ego or, like, its self conception encompass, like, way more than just, like, the code that's running? Yeah, that's a a totally great direction. Right? Like, may maybe something good will come out of it. So I do agree. I I don't know if that corresponds to this concept of organic alignment because I do think that with the analogy to cells aligning with other cells or ants aligning with other ants, it does seem to have this huge dysanalogy. Right? This huge flaw where these organisms are all weak and they fundamentally need to cooperate.

Nathan Labenz: 1:54:50 Yeah. Well, I think maybe another way to think about it is, can we engineer AIs that way? I mean, the other thing that this really reminds me of is there've been various schemes like this proposed, but Eric Drexler's comprehensive AI services proposal that's now honestly a few years old, but I think still remains, like, very interesting. He basically says, you know, what do we want AIs for? We want AIs to perform services. Does an AI need to be superhuman at everything in order to be superhuman at a particular service that it's going to perform? Probably not. Therefore, we should be able to create AIs that are superhuman at what we mean them to be good at, but not, you know, capable of doing everything else. And as long as we have these sort of narrowly superhuman systems, then, you know, maybe we can sort of deploy them and, you know, build up a a sort of superstructure of them in a way that we can sort of retain control of and and can be kind of generally stable. And like, that seems pretty good to me. Right? Like alpha fold, 2, 3, 4. Like, I'm not really scared of any level of alpha fold because it does 1 thing. It does it really well. Where I get more scared of it is when that capability gets, like, deeply integrated, you know, with a with a broader set of capabilities, and then, you know, it becomes better at people than, you know, everything or near everything. So I don't, you know, I haven't studied their proposal here. I don't know exactly how they're thinking about it. But I do think if you were to, you know, sort of say, you know, 1 way to interpret it, maybe not their way, but 1 way I, you know, can maybe torture it into interpreting it is, maybe we should create these, like, narrow specialist superhuman systems that are, like, great in domain and not great otherwise. And then it's like the ensemble or the sort of, you know, scaffolding of those that, like, makes everything work. But sort of, you know, that could be like a much more stable, predictable, engineerable sort of thing than a a totally general purpose, general intelligence that you just kind of say, oh, go solve all our problems and give us utopia.

Liron Shapira: 1:56:58 Mhmm. Just to tell people a little bit more about Emmett Shear's company, Softmax. He's got 2 cofounders, Adam Goldstein and David Bloomin. I'm a little bit familiar with Adam because I use his product, HipMunk. It was a really cool flight search engine that was ahead of its time back in, like, 2010. So that is Emmett Shear's cofounder for, Softmax. And a little bit more about their philosophy about alignment, they're contrasting organic alignment to hierarchical approaches alignment. They're saying the most common approach to alignment among AI labs today is a system of control or steering. Some set of rules that most companies and researchers take to define good action, whether that's obey this person's intent or follow these commandments. Systems of control are always hierarchies because they imply something controlling and something controlled. Hierarchical alignment works fine right up until the rules or person on top are wrong. The smarter the subordinate, the more likely this is. Hierarchical alignment is therefore a deceptive trap. It works best when the AI is weak and you need it least, and then it works worse and worse when it's strong and you need it most. Organic alignment is, by contrast, a constant adaptive learning process where the smarter the agent, the more capable it becomes of aligning itself. Man, so, like, I I get I get this on a vibes level, and, like, maybe there's a way to steal mannit. I I mean, the disanalogy is getting in my way when I try to steal mannit. Right? Like, when it when like, the reason you need hierarchical alignment is because by default, if you don't top down set the AI's goals and you just let it, like, naturally evolve its goals or whatever, you're just not going to get the right goals. You know? And I think maybe with Emmett, there's this concept of, like, well, maybe you guys can develop goals together, and it it has, like, as much insight into what its goal should be as you. So you guys should, like, just cooperatively evolve. And, like, I get that vision, but it's just like there is, an attractor state where you just got a really powerful goal optimizer. And anytime you increase intelligence, it just tends to, like, get into that state. And I just don't see this particular approach telling us how to avoid the attractor. That's like my high level thought.

Nathan Labenz: 1:58:55 Yeah. I mean, I think it's it's fair. At the same time, you know, I think on the positive side of AI practical, mundane day to day utility, 1 of the most common things I find myself needing to tell people is like, you gotta look for ways to make this thing work for you. Like, if you try to, you know, prove that it's useless, like, you'll find examples where it's useless. And if you come to that conclusion, like, and then then, you know, don't use it anymore, like, you're the 1 that's losing because you're not getting any value and the rest of us are. Because we're bringing a mindset of, I'm gonna find a way to make this work. And, you know, we know that we can in fact often do that. I can think a similar mindset is healthy for alignment research. And I just wanna be encouraging. You know, I I don't think Yeah. The what I say, like, this sounds like they're, you know, immediately on track to a solution or, you know, sometimes they say, like, something that really works where really works means I don't have to worry about this anymore. Don't expect really anybody to come up with that, frankly, at this point. My general assumption is that what we're gonna have is a sort of defense in-depth strategy. I mean, this is basically explicitly stated by OpenAI and to some extent others. But OpenAI stated very clearly that their safety approach is just layer on a bunch of mitigations, and they all sort of take a bite out of the problem. And, like, collectively, we hopefully can get that to enough reliability that we'll be okay.

Liron Shapira: 2:00:20 And like They've also stated, we're gonna ask the AI to help us split creative. What's your idea for alignment?

Nathan Labenz: 2:00:26 And, you know, that sounds crazy. It's maybe not entirely crazy. But I guess where I would try to slot things like this in, and and why I personally encourage anyone with an alignment idea to pursue it, even if it is kind of weird or, you know, easy to ridicule or, like, frankly, just plain not likely to work, which I think does describe most alignment ideas, is like, we need more layers in our defense in-depth strategy. And if you can figure anything out that makes any contribution that is at all distinctive from the contributions that others are making, then I in my mind, that makes you a hero. Because I don't see it would be great if somebody came along with the like, definitive solution for alignment, and and safety and control, and we were just like, okay, do this and everything's solved. I don't expect that. And in the absence of that, you know, it's gonna be a patchwork. And, like, you know, contribute to the patchwork, people. That's like, I think that's a very noble pursuit.

Liron Shapira: 2:01:21 Yeah. And it's probably better than nothing if they're doing this. Right? It's probably better than going and starting, like, another AI buddy startup. Right? Like, oh, making another AI that's gonna be your friend. Like, that's probably a waste of time, compared to at least attempting alignment. So I definitely respect him for, like, working on an important problem. I personally think that at this stage, engaging in debate, engaging with critics is probably a good idea because, like, look, I'm convinceable. Right? It's like, don't have a horse in the race where I wanna tell everybody that organic alignment or, you know, whatever his approach is. I don't come at it from the perspective, like, I have to tell the world that this isn't going to work. I just currently think it's not going to work, but I'm open to changing my mind. And I'd love to help, clarify. Right? Like, friction, sometimes it yields, like, productive refinement. Right? And I feel like that's an important stage in this process. That's why I invited him to come on Doom Debates. If anybody else wants to come represent the the greatness of scaling alignment and, you know, organic alignment, if anybody else thinks that they get it and wanna come on the show and explain, I think that'd be a productive exercise.

Nathan Labenz: 2:02:22 Yeah. And they may also just need some more time too. Right? I mean, they just started and, you know, I think part of any of this kind of work is going to be selling the work. Right? You you've got a not in a literal sense necessarily, but in a sense that like, there are going to be people developing frontier systems who are going to be also developing, you know, their own defense in-depth patchwork. And it is going to be on you to communicate your novel ideas and get them to see that it's worthwhile in order to adopt, or it isn't gonna have much effect. So I do think that there's an onus on people who, you know, believe that they have a contribution to the overall AI safety big picture, to, like, come out and make the case. But I would also, you know, say, maybe they're just not quite ready yet, and that, you know, that also is totally fine. Like, do you know, in the meantime, they do still have to, like, build a team and, you know, maybe raise some fund. Mean, I guess he probably has enough funds to just self fund it. But Mhmm. You know, there is a, you know, there's a there's a ramp up process, certainly, that they're gonna have to go through in development process, and they might just not be ready yet for the sort of public facing, you know, true sales process.

Liron Shapira: 2:03:33 Yeah. Fair enough. I mean, even, Ilya Sutskever with safe superintelligence, they've said absolutely nothing. Come on, Ilya. Tell us how you're going to make it safe. Come debate me how you're gonna make it safe. It's ridiculous.

Nathan Labenz: 2:03:43 Yeah. Mean, think that is I I'm way more inclined to be critical of that. I think that the idea that we're gonna develop safe superintelligence, totally in private, sharing nothing, not productizing anything, and we'll, like, you know, let you know when we're done. That to me is not a reassuring plan. And I honestly think governments should get involved.

Liron Shapira: 2:04:09 Yeah. Here's their website. Makes Berkshire Hathaway's website look like, Yahoo.

Nathan Labenz: 2:04:15 Yeah. I mean, this is this is like the dark matter of of Frontier AI development right now, and I think it is bad. I think that, like, there should be more accountability. There should be more transparency of some sort, than what we're seeing here.

Liron Shapira: 2:04:36 Yeah. Okay. Penultimate topic here. Anthropic's new mechanistic interpretability paper.

Nathan Labenz: 2:04:46 Yeah. I mean, I think this is something everybody should read, and I I would, you know, encourage people to read it in in-depth. Like, it's a long read, but I think it really is worthwhile. Anthropic does truly outstanding work, and and really outstanding on, like, every level. You know, they they publish in beautiful form. The as you scroll through the the very long, you know, paper as blog post on their website, they've got these, like, interactive UIs that sort of illustrate things. And, you know, they they have really done an incredible project here. So I I wanna, you know, just start by, appreciating it for what it is. Like, I think, you know, I I've previously said publicly that, like, arguably, the interpretability work at Anthropic is the most important work going on anywhere in the world. I said that, like, 2 years ago, and I still think it's, like, a pretty good candidate. So Yeah. Very, very positive and appreciative. And, you know, the rigor and the depth of with which they're sharing it and the pains that they've gone to to try to present different, you know, interfaces to the information to make intuitive are, I think, like, just admirable and and, you know, and and awesome on on basically every level. The 1 thing that would be, like, the kind of black pill on it right now to me relative to at least, like, the way it's been understood is I think it really just shows that this interpretability stuff is, like, still really hard and has a long way to go. The you know, and it's gone faster than I expected. Like, I I I'm I'm, you know, very similar to how I'm very pleasantly surprised relative to my expectations 10 years ago by how ethically Claude behaves today in practice. I'm also, like, very pleasantly surprised relative to my expectations 3 or 4 years ago by how much progress has been made in interpretability. But the headlines that people are you know, or the tweet length of things that people are putting out now that are like, you know, they figured out how it works, and, you know, now we've got clarity into all this, Definitely, dramatically overstate the results. Anthropic themselves is not dramatically overstating the results. Okay. But I think How

Liron Shapira: 2:07:03 would summarize the current result?

Nathan Labenz: 2:07:07 I mean, it's hard to summarize. That's 1 of the things. It's it is extremely technical, and there are a ton of caveats. So, you know, they train what they call a replacement model, which consists of these things called they're sparse auto encoder like, but they're different. They're called cross layer transcoders, CLTs. And basically, these are meant to be like SAEs in that they are spar they're very wide.

Nathan Labenz: 2:07:41 They have there there's 1 at each layer, and they're meant to be sparse. So they they train these things with a reconstruction loss and a sparsity term. So that ideally, what you're trying to do is recreate the behavior of the underlying model, but do that by passing through these sparse things such that each active neuron represents a feature. And then you can map how these features interact. And that's ultimately the sort of high level schematics that people have seen are these sort of, you know, most zoomed out feature interaction graphs that ultimately lead to outputs.

Liron Shapira: 2:08:19 Is this what they're referring to? Here here's Anthropic Street. We built a microscope to inspect what happens inside AI models and use it to understand Claude's often complex and subscribe and surprising internal micro so what they're calling a microscope, is that what you're saying, which is the, the the the sparse autoencoder like thing where they're, like, mapping the complex concepts to, like, a simpler version so they can understand what's that's the microscope?

Nathan Labenz: 2:08:40 Well, there's more to it than that. I mean, it's it's a quite complicated setup. So and there's just like so many caveats, I think, and so many sort of important to understand limitations to what they have. 1 fundamental 1 right off the bat is that, like, these things learn these sort of sparse features, but the labeling of those sparse features is itself a subjective process. The way that's done is you look at the examples that cause that particular position to light up, and you sort of eyeball it and say, what does this look like to me? What feature does this seem to be to me based on these examples that I'm looking at? And that creates massive opportunity for disconnect between what you think you are looking at and what the AI itself is actually, you know, understanding or representing. And many of these things are quite noisy. Like and you will see this if you do, as I have done, like a slow read of the of the methods paper, and you actually use the interface, which they so helpfully provide, and click into some of these features and look at, like, what are the top things. Some of the labels you're like, I don't know about that label. You know, you're saying that label, but I think you might be saying that label because you kind of know where this is going. I think there's a huge difference. And so they've got these, you know, sparse layers, and then they create this UI on top of it. And then there's, you know and by the way, so the best they can do is predicting 50% of the overall model's behavior. So right there, it's like, you know, we've created something that's like very lossy compared to the underlying model. It can do 50%. Okay. Fine. Then they zoom in further and do, okay. Let's study a single prompt. They do a single prompt, and then they add all these error terms because the thing isn't actually capturing the all the the computation, but they wanna get closer to the computation. So they add all these error terms. So in any given, you know, 1 of these situations, there's like lots of error terms that they've just kind of placed in where they need to to make the thing give the same output. Then with that, you can look at these features and say, you know, what do these features seem to be? But again, I'm not quite sure how much knowing the answer is is leaking in. But in my read, just looking at the the actual examples that are causing these particular features to light up, it would not be obvious at all that some of these features are what they say they are. I think it would be like, you would not I would I would go even further and say, for some of the ones I looked at, you would be hard pressed. If I just gave you, you know, here's the examples. What is this feature? You would be hard pressed to come up with the label that they came up with for it. So Mhmm. You know, there's just a a lot going on there. And and again, this is all this analysis is happening with all these error terms for a single prompt. So it's just like and then there's, like, aggregating yeah. I mean, there's more to it as well. I think it's, like, I think it's outstanding work, and I just think people should not over index on it because Right. It is you know? And and, again, it's not their fault. Like, they have provided the, you know Right. 20,000 word treatise on all of the methods and all of the limitations and, like, it's all there for you. But don't don't forget that part or don't let the sort of, you know, they cracked it lead you astray. I don't think their team believes that at all. But I am a little worried, especially in the context of like other Anthropic, you know, leadership statements around the need to like race China to AGI or whatever, that this, you know, it feels like we could end up in a world where people are like presenting these graphs and saying, see, we know how these things work. We got nothing to worry And I think it it very well could be a mirage. The other thing too is just all these examples are like really simple. You know, the the graph for 2 digit arithmetic is like a super complicated graph. They've also got like plenty of examples like this 1 where it's like takes a nudge from a person and then, you know, comes to quote unquote the right answer that the human, you know, suggested, but with totally faulty reasoning. So there's like I just so many caveats that I am just wary that the the overall package of, like, how it's being understood and how, you know, that may interact with the sort of sociopolitical, you know, arguments that an anthropic leadership is making, that I've just the the overall thing is, like, in isolation, some of the best work ever done in the field. In context, you know, not something that I am inclined to just repeat the headlines about, because I think the headlines have sort of Right. Overstated and and even worse, like, could be used to create a false sense of security.

Liron Shapira: 2:13:31 Yeah. So my takeaway is, like, to make a little bit of an analogy. It's like they have this microscope and, like, yes, it is progress. It's like a new view. We've never been able to see this particular view before, but it's kinda like an optical microscope versus an electron microscope where, like, the optical microscope is useful. But if you're trying to look inside of a cell and understand the processes inside of a cell, you know, analogously, you really need that electron microscope because the the wavelength of light isn't going to help you that much when you're trying to look at all the details. So they have this fundamentally blurry microscope. And then the obvious question is like, well, can you ever have a non blurry microscope, especially when the AI is, like, potentially scheming against you or, like, solving your challenges in a way you don't understand? Is your microscope going to be good enough to peer inside and be like, I found cheating. Like, there's no there's no black box output that indicates you're cheating, but I could look with my good enough microscope that you are plotting. Right? That's kinda like the holy grail of mechanistic interpretability. But there's just no sign right now that we're going to get that good of a microscope.

Nathan Labenz: 2:14:29 Yeah. If anything, I just think the the excellence of this work, the rigor, you know, and I have nothing but but praise for the work itself, just shows like, man, the problem is really hard. There is just an unbelievable amount of stuff going on.

Liron Shapira: 2:14:41 And And of course, there's always been the problem of understanding how humans think. Right? We know we never got that.

Nathan Labenz: 2:14:47 Yeah. I do think this is easier. I do think the you know, if only because just the substrate is much more

Liron Shapira: 2:14:52 Right.

Nathan Labenz: 2:14:53 You know, amenable to experimentation. So

Liron Shapira: 2:14:55 Yeah. So this is this is better than an MRI for sure. But, unfortunately, a lot of the complexity really is, you know, on the the layers, you know, just layers upon layers. So you can have perfect visibility into each layer and still never crack it.

Nathan Labenz: 2:15:08 They do have some good validation type things too where they do perturbation studies and, you know, if they to to sort of because again, they do recognize straight up, like, there's no guarantee that the mechanism of these cross layer transcoders is a faithful representation of the underlying model's mechanism. So, you know, that's about as clear as you can be. But then they try to validate that it is at least somewhat faithful by doing these perturbation experiments. And they show that, you know, in some cases, they are able to get it to work, where if they like 0 out a certain, you know, feature in 1 of the layers, then like, you know, the model doesn't do what it's, you know, supposed to do anymore, and so on and so forth. So it's like, again, it is very good work. But yeah, just the depth of this problem is evidently extreme. And, you know, you look at something like this chart that you currently have on screen just to think about 2 digit addition, and then you think like, man, we are now into Yeah. The latest Gemini 2.5, it can generate 65,000 tokens of output Yeah. In 1 generation.

Liron Shapira: 2:16:19 Yeah. And and not only can it generate a lot of tokens, but the thought process that goes into every token is not just focused down into 36 plus 59. It's focused on, let me take into account every token from the last few paragraphs and how that affects my thought process. Right? So how the hell are you going to graph that?

Nathan Labenz: 2:16:34 Yeah. Plus, I mean, literally, potentially hundreds of thousands of of prior tokens. So the the graphs that you would have to be studying here become literally exponentially large and complex, and maybe super exponentially. And the and again, at at every feature is lurking the problem that we don't know if the labels we've applied are in fact, like, deeply correspondent to what the model is doing.

Liron Shapira: 2:17:08 Right. Right. Yeah. I mean, it works for some things. Right? It helps you understand some things, but then you kinda lose resolution. Right? It just gets blurry when you try to understand. Maybe if you try to understand, like, remember that recursive picture we saw? Like, how did it make that recursive picture? Right? What were all the constraints it was solving? The lens gets too blurry.

Nathan Labenz: 2:17:24 Yeah. It's tough. It's a really tough problem. I I mean, I love I I think it's a very worthy read, and and people should spend time on it. But do the methods paper as well, and make sure you can, like, cite those, you know, top 5 caveats, limitations, gotchas before you, you know, start retweeting the headlines.

Liron Shapira: 2:17:47 Yep. Yep. Alright. Bring it home here. We got 1 final tweet, which I consider to be ending on an optimistic note for somebody like me who's very interested in, international cooperation to be able to pause AI, whether we pause today, whether we pause in a year, whenever we're ready to pause, we need to be able to pause at an international cooperative level. So here is Cat Woods who has a lot of good AI safety tweets, and she's also a a member of Pause AI or at least believes in the Pause Cause according to the Pause icon in her Twitter profile. So Cat Woods says, what if the outgroup defects on an AI treaty? Well, we can actually spot them by number 1, satellite images. Data centers are massive and hard to hide. Number 2, monitoring massive electric use. Training state of the art AIs requires massive amounts of electricity that is relatively easy to spot. Number 3, monitoring AI chip supply chains. State of the art AIs require specialized chips that go through many different supply chain bottlenecks. These can be monitored for who they sell who they sell them to and how many. And then she says also tons of other ways. Check this out for a more thorough discussion. And then it's a research paper. They always accuse AI safety of not having research papers. But look, it's a research paper called verification methods for international AI agreements. And let's see see what the paper looks like. So it's just got a bunch of different analyses of different methods to inspect you know, AI projects in progress. So, like, there there's definitely some feasibility here. Like, it's arguably a much easier problem than what we just talked about, like, mechanistic interpretability. Like, it's not like other problems we're trying to solve are any easier than this problem of solving international cooperation. I know a lot of people like to act like solving international cooperation. You have to take it as an axiom that international cooperation isn't possible, and everything else has to follow after that axiom. But I don't think it's an axiom. I think it's just 1 hard problem among many hard problems.

Nathan Labenz: 2:19:36 Yeah. I think I pretty much agree with that characterization. I definitely think we should be trying. I would maybe place a little bit more emphasis on actually building, you know, enough sort of trust and confidence in the, you know, leading powers, or between the leading powers, so that, you know, these sort of verifications or, you know, defection detection can be as reliable as they possibly can be. Because I don't really think we're gonna be doing it from space, to be honest. I think like, yes, you can see data centers from space. But guess what? There's gonna be a ton of data centers no matter what. And yes, you would see 1000000000000 dollar data center sticking out like a sore thumb. But also, guess what? Already, the leaders are trade are training their models in multi data center ways and, like, distributed training is working better and better

Liron Shapira: 2:20:36 also. Yeah. Yeah. But but, I mean, if you make it illegal, right, I mean, if you look at every player in The US, it's already a long shot to think that literally Apple, Microsoft, even OpenAI, Anthropic, that any of these companies would directly defy an order. So you're saying, oh, well, if they do, they won't get caught. Yeah. But it's already unlikely that they would even dare to defy an order like that.

Nathan Labenz: 2:20:56 Yeah. I mean, I I think the fear in these sort of treaty situations is like, we make a deal with China that we're both gonna not do whatever. And then how do we know China's not doing it? How do they know that we're not doing it? It's not so much Yeah.

Liron Shapira: 2:21:07 Because China just has these big data centers. I mean, then it's like we need visibility. Right? So we Yeah. We need CSS.

Nathan Labenz: 2:21:12 To for that to really work, I think we need to be, like, on the ground with each other, like, actively working together in the highest trust, you know, dynamic that we can possibly muster. As long as there's no trust and there's no willingness to open up and to, you know, share information, then I I think it's it is gonna be very tough to sort of see this stuff from space. So for me, the emphasis is very much on trust building, relationship building. And, you know, that's also hard, and there's not a lot of appetite for it. I go around talking about this, and people just, you know, tell me I'm naive and that, you know, there's no there's no appetite for it. But I say

Liron Shapira: 2:21:49 Hardware level. Right? I mean, if if we if we have visibility in, let's say, China's supply chain, maybe we could the ize the hardware. So because the hardware should probably know if it's being used to train AI just because it's getting so close to the metal and so optimized when you're doing AI training. So it's plausible that the hardware can report like, hey. I did a process that looks like AI training, and then we can, like, check the hardware. Right? There could be, like, a standardized inspection process.

Nathan Labenz: 2:22:13 Yeah. I mean, I think trust is the bottleneck to that. You know? Like, yes. We could, I'm sure I don't, you know, I don't know how, and I'm sure there's a lot of challenges to develop that sort of technology. But I don't doubt, you know, that it could be developed. What I doubt more than that it could be developed is that people actually want to deploy it. And Right. You know, recently heard, like and I don't know how true this is, but I recently heard a statement that, like, all of The US military tech that has been sent to Ukraine and even to Europe more broadly, is like, all something we can turn off remotely because if it doesn't get, you know, it's sort of, you know, required software update or whatever in in a timely way, then it just won't work anymore. And, you know, if that's true, then I think those countries are probably not gonna wanna buy that kind of technology from us for much longer. And, you know, China's not gonna wanna buy those GPUs that we can turn off remotely if, you know, barring like a lot of trust. So I I do think, to me at all, it really just comes down to trust. I don't I don't think we can have an adversarial relationship and find a state of equilibrium. That was kind of, you know, Dan Hendrix recently put out this paper called

Liron Shapira: 2:23:25 Right.

Nathan Labenz: 2:23:25 You know, MAME mutually assured AI malfunction. And I oh, again, there's another 1 of these things where I, like, absolutely applaud the work. What I understand them to be trying to do is define some sort of stable equilibrium between adversarial great powers. And I think, like, that's a absolutely, you know, top tier problem people should be working on. I unfortunately didn't find it particularly convincing. It did not ring to me like a actually stable equilibrium between adversarial great powers. But if you could change the adversarial word out and make it cooperative great powers, then it might work.

Liron Shapira: 2:24:03 Yep. Yep. Yep. 1 more, little image here from Kat. There somebody's asking, what if somebody defects on an AI pause treaty and then the Simpsons bus driver saying, go make me tap the sign. And the sign says, this is true for all treaties. Treaties don't have to be a 100% effective to be helpful, which I do agree with. You know, it's obviously, this is something that 1 defector could cause a lot of damage. But even if the treaty is letting us catch them, you know, maybe we don't have to perfectly catch them immediately. Maybe we just have a high probability of catching a defector a little bit later. I mean, nobody's shooting for a 100% perfection. And then it also gets to the point of, you know, speaking of all treaties, all treaties also have enforcement. Right? So if people who are freaking out about, like, wait a minute. Wait a minute. Your treaty has an airstrikes provision? It's like, yes. Generally, treaties need to get enforced. That is true about all treaties, and it's it's always such a a pet peeve for me or even stronger than a pet peeve when people like to point out the violence part of it. It's like, doomers didn't invent violence. Right? Violence is the enforcement mechanism for treaties. If you don't like the treaty, say you don't like the treaty. But it's a low blow to then accuse doomers of being, like, violent because we're advocating for a treaty.

Nathan Labenz: 2:25:12 I think it's a great point. It's just starting with the fact that it doesn't have to be a 100% to be helpful. You know, it that is very much concordant with my feelings on the technical solutions too. You know, they were headed for almost for sure a patchwork arrangement. And the question is gonna be like, how many nines of reliability can we create? And, you know, is that enough? Sometimes I bottom line that as defense and depth is all we have. Let's hope it's all we need. So every layer that we can add, you know, as long as it doesn't, you know I I am, again, very enthusiastic about the upside too, so I'm mindful of like, not wanting to prevent me from getting my AI doctor or my self driving car, but subject to, like, not, you know, preventing me from realizing mundane day to day utility of AI, then I'm, like, very enthusiastic about things that, you know, bring the the p doom down. And, you know, my guess is we're gonna need a lot of bites out of that apple to get to a a place where we can sleep well at night long term.

Liron Shapira: 2:26:19 Nice. Alright. Sounds good. Is there anything else on your mind that you feel like we have to get to in terms of news roundup, or is this a good place to call it?

Nathan Labenz: 2:26:26 You know, there will always be more, but, I think that's all we

Liron Shapira: 2:26:29 can That's true. We do have more. Yeah.

Nathan Labenz: 2:26:31 That's all we can manage for today. So thanks for doing this.

Liron Shapira: 2:26:34 Yeah. Yeah. Thanks so much for coming. So viewers, we need your feedback here because Nate and I had fun doing this episode. We're going to refine it. Like, we we brought in more possible topics than we had time for, which is fine. We can always do more episodes. We'd love to hear your guys' feedback in terms of, like, should we do more of these episodes? Think about the opportunity cost. Is this better than the usual kinds of episodes we put out so that we should work this into our schedule? Or should we just stick to what else is working and and let other people do the news? Please tell us. Write a comment saying like, hey. This episode was worthwhile or not so worthwhile, and it would be cool if you do this, differently. We would really appreciate your comments. Alright. So everybody check out Nate's podcast, The Cognitive Revolution. You can just search any podcast player or go to YouTube. Nate, great seeing you, and thanks.

Nathan Labenz: 2:27:21 Likewise. Thank you.

Nathan Labenz: 2:27:30 It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.