AI 2025 → 2026 Live Show | Part 2
Live show with rapid conversations on AI's 2025 and prospects for 2026, including Alex Boris on the RAISE Act and catastrophic-risk debates, Dean Ball on federal policy and preemption, and Peter Wildeford on chip controls, agents, costs, and robotics.
Watch Episode Here
Listen to Episode Here
Show Notes
This year-end live show features nine rapid-fire conversations to make sense of AI’s 2025 and what might define 2026. PSA for AI builders: Interested in alignment, governance, or AI safety? Learn more about the MATS Summer 2026 Fellowship and submit your name to be notified when applications open: https://matsprogram.org/s26-tcr. New York Assemblymember Alex Boris breaks down the RAISE Act's bid to curb catastrophic AI risks, the governor negotiations, and why a16z-backed ads are targeting him. Former White House AI adviser Dean Ball maps the emerging coalitions on AI, federal preemption, and what fast-improving coding agents could mean for policy and jobs. Forecaster Peter Wildeford debates selling vs "renting" chips to China and offers a 2026 outlook on agents, costs, and robotics. Recorded live as part of our AI 2025→2026 series (Part 2).
Sponsors:
Gemini 3 in Google AI Studio:
Gemini 3 in Google AI Studio lets you build fully functional apps from a simple description—no coding required. Start vibe coding your idea today at https://ai.studio/build
MATS:
MATS is a fully funded 12-week research program pairing rising talent with top mentors in AI alignment, interpretability, security, and governance. Apply for the next cohort at https://matsprogram.org/s26-tcr
Framer:
Framer is the all-in-one tool to design, iterate, and publish stunning websites with powerful AI features. Start creating for free and use code COGNITIVE to get one free month of Framer Pro at https://framer.com/design
Shopify:
Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive
Tasklet:
Tasklet is an AI agent that automates your work 24/7; just describe what you want in plain English and it gets the job done. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai
CHAPTERS:
(00:00) Sponsor: Gemini 3 in Google AI Studio
(00:31) RAISE Act status
(04:52) Catastrophic risk focus
(10:47) Super PAC backlash
(16:17) Data centers and grid
(19:47) Palantir and surveillance
(23:42) Dean Ball joins (Part 1)
(23:46) Sponsors: MATS | Framer
(27:05) Dean Ball joins (Part 2)
(32:00) Social media lessons
(36:22) Trump preemption and chips (Part 1)
(40:51) Sponsors: Shopify | Tasklet
(43:59) Trump preemption and chips (Part 2)
(44:50) Structural US-China decoupling
(49:39) Peter on chip ban
(55:43) Threat model and renting
(01:02:55) Cost drops, revenue
(01:16:03) Forecasting paradigm shifts
(01:20:38) 2026 agents and robots
(01:31:41) Show wrap and markets
(01:42:28) Outro
PRODUCED BY:
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://linkedin.com/in/nathanlabenz/
Youtube: https://youtube.com/@CognitiveRevolutionPodcast
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
Transcript
Main Episode
speaker_1: Bye. Next up, we have Congressman, no, New York State Assembly Member, Alex Boris. Future.
speaker_2: Just a little ahead of yourself, yeah, with Congressman there.
speaker_1: Future Congressman, hopefully. And he is a computer engineer by training. He was a Palantir for four years. I am definitely glad that we have engineers now entering public service, which hopefully will at least make our leadership cohort younger and more tech savvy. He is also the author of the RAISE Act, which is now, I think, the president is trying to preempt with his executive order. So there's lots to talk about. Alex, great to have you on the show.
speaker_3: So glad to be here.
speaker_2: Cool. So welcome. First question, you sponsored the RAISE Act. We did a whole Cognitive Revolution episode about it some months ago. It has, at the time, it honestly didn't seem like it was going to be a super controversial whitening rod sort of thing. it seemed like it was less that than SB 1047 was. That has not necessarily held over the intervening months. Where are we? I know there's been sort of, you know, let's say, edit to the bill from the governor. Now we have an EO that sort of targets bills like this. And then we also have super PACs that are coming online to target people like you for bringing these bills forward in the 1st place. But just for starters, like where are we in the life cycle of the RAISE Act?
speaker_3: You've hit all of the major ingredients. So the main aspect is that we're in active negotiations with the governor right now. New York has a fairly unique process called chapter amendments, where after the legislature passes a bill, The governor can negotiate with the sponsors of the bill for changes that she would like to see. And if we reach an agreement, she signs the original bill, but attaches a memo that says, I'm signing pursuant to this agreement that we've reached. And then we, the sponsors, will introduce all of the amendments early in the next session. About 1/3 of the bills that get signed in New York, it varies year to year, but roughly 1/3 of the bills end up going through this chapter amendment process. We always knew RAISE would be a part of that. And so we're actively in that process now. The governor, or I should say the bill was delivered to the governor last week, and the Constitution gives her 10 days, excluding Sundays, to sign. So that means she has until Saturday to sign it or veto it, although the message that we're getting is they're trying to shut everything down by Friday night so people can actually have some holidays. So in the next 48 hours or so, we should know if we have agreement on a revised version of the RAISE Act or if unfortunately we don't.
speaker_2: And what are the kind of key outstanding points of negotiation right now?
speaker_3: There's A lot. And I don't want to get too into sort of conversations that are happening because you want people to have the freedom to express, hey, what if we tried this? What if we tried that? So I have to say somewhat above the specific details, but there's been a lot of reporting this week that there's a lot of pressure to move to something that more matches SB 53 versus what was initially in RAISE. You know, curiously, there hasn't been a push to match the European code of practice. So it seems like it's less about maybe standardizing everything that has to be done versus weakening it. But the point that I keep making is that there's no company that would be subject to, if we were to pass the exact same bill as SB 53, there's no company that would be subject to it in New York that isn't already subject to it in California. So it's not like passing a data privacy bill where each state that passes it, you give new rights to the citizens of your state. If you just reaffirm the exact same bill, you haven't changed anything. It's like being the 39th state to ratify an amendment to the Constitution, right? 38's enough. It's in the Constitution. The 39th, it doesn't make practical changes. And so we're having real conversations about, you know, SB 53 was always meant to be the floor. Are there steps we can take that advance AI safety that we can all agree on? Or are we destined, unfortunately, to have to try again next year? Or both.
speaker_1: Let me take a step back and like, what do you think are the dangers that are trying to be addressed by the AI safety envisioned in the RAISE Act?
speaker_3: There's a number of them. And to be clear, this is not, there's a lot that we need to address with AI. whether that's its effect on kids or the environmental conversation or its effect on democracy or the workforce, right, discrimination, all of these things. The RAISE Act in particular is focused on really extreme risks, on the type of catastrophic risks that would signify something going really, really wrong. It's a set of risks that should be addressed at the federal level, but frankly, have not been. And in fact, it seems like most of the push is the other way about stopping the federal government from taking action. And certainly, there's a push to stop the states from taking any action. But because the federal government hasn't, states have been stepping up. And there's eight states that had some version of a frontier model safety bill this year, and only two managed to pass it. And California had a very, very strong version that then SB 53 became that compromise at the end of their session that is focused mostly on transparency. There aren't really safety standards in the bill, but there are transparency standards and that's very important. And then the RAISE Act, which passed in New York, which put real safety standards on what could be developed and requiring the labs to have a safety plan that they make public and actually stick to. to disclose critical safety incidents, things that show a massive increase in risk of something bad happening. And if a model fails their own tests, that they shouldn't release the model. And that's designed to stop the situation that we had with the tobacco companies, where they were the first to know that cigarettes cause cancer, but continue to deny it publicly and release their products. We're saying in many cases, we recognize the labs are the experts in what's happening. They're at the frontier. And if they discover an extreme risk, they have a duty to act upon that.
speaker_1: But what is an extreme risk? Like what are we talking about?
speaker_3: So specifically in the RAISE Act, it sounds almost silly to describe how extreme it is. We are talking about 100 people dying, killing 100 people, or a billion dollars in damage. specifically via helping to build a chemical, biological, radiological, or nuclear weapon, or by committing a fully automated crime. So this is not someone who's using the AI to commit a crime. It is you tell the AI to go do something totally legal and it goes off on its own and commits a crime. And even if you meet that bar, you then have to prove that it was materially better than just using the internet and that risk was in some way foreseeable.
speaker_1: I think one of the things that strikes me is that a lot of these, a lot of developers take these models and they apply them, the 1st place they apply them to is like crypto trading. And like arguably all of crypto is money laundering anyway. So they're all, it's all illegal anyway, right? Like it's all, it's all, we all know it's actually all illegal, but we like, we have this like, we all kind of agree not to look at it for now until like the Dems come back into power, but it's all money laundering anyway, right? So it strikes me that all of these models are capable of and are going to do it and developers are going to apply it to crypto trading of various kinds and, you know, wash trading. And I used to be a financier, so I know all the tricks that you have. And crypto, in crypto, people apply all of them. It's like the 1920s, pre-SEC stock trading was like that in the United States. So I often wonder, to what extent these bills kind of do not recognize that is the state of play, that all this illegal activity already occurs everywhere. And it's just that prosecutors have difficulty prosecuting. The political winds are in the opposite direction for the moment, and which will swing at some point. And so I wonder, how these regulations take into account the reality of the state of play on the field.
speaker_3: There's a lot there. So let me say, first of all, I don't think all crypto activity is money laundering. I just want to say that I do support stronger regulation and I think we should have some clarity there. But the key in the RAISE Act is if you were to you as a user were to use a model to commit a crime, that is still a crime, right? And that is still prosecutable. That is, we wouldn't need additional legislation for that. The point of the legislation, and to be clear, does not change downstream liability, does not change what is a crime. Like all of those laws still apply. This is saying like, what is the subset of extreme risks that are so bad that if they got out, if they happened, we wouldn't be satisfied to just deal with it later in the court process, right? You want to actually prevent the bad thing from happening. I mean, you can think about it in terms of, say, national security, which is different than law enforcement, right? National security is about preventing something really bad from happening, even if it hasn't happened yet, and figuring out what are the ways it could go wrong and getting ahead of that, where in law enforcement, you're saying, okay, a crime has occurred, how do we deal with it? This is more on that national security side. There are things that are really bad. A bioweapon being created is exceptionally bad. Loss of control of an AI model is exceptionally bad. Those are the sorts of things we want planned for ahead of time and not dealing with later via lawsuits. And all the RAISE Act is saying is you as a developer should have a meaningful plan that reduces the risk of those bad things happening.
speaker_2: Can we? Touch on the politics of this a little bit. Totally. I'm struck by the fact that, first of all, most of what you said there, and obviously that was in the details and all that, but like the general framework is very much an echo of the frameworks that the companies have put out themselves for the most part, including OpenAI. And One would think that the, if you're a naive simpleton like myself, you would think that the tech industry would look at somebody who has a master's degree in CS and who even worked at Palantir for a few years and think like, hey, this is a guy we should be able to work with, right? And yet you've obviously come under some, I don't know, pressure, you know, political attack ads, whatever, from, as I understand it, a super PAC backed by A16Z and Greg Brockman. And I guess I don't really understand like, why. I'm curious as to what you think their ultimate like reason or objection is. But then I'm also really curious as to how is this playing out now in your race? Like, are they running attack ads on you that are on topic on AI, or are they running things that are off topic and, you know, kind of just trying to cause you trouble in general? And like, how are the voters responding to this? You know, I kind of wonder if the AI industry might create a Streisand effect for themselves, where they sort of end up doing their own cause more harm than good. Because I think they should maybe look at some of the polling numbers on just like how popular AI is before they get too confident with some of their. messaging. So a lot there, but what do you, what's your side of that story?
speaker_3: Yeah, you're spot on in your analysis. So I am running for Congress in a very crowded field. There's, I think, 10 declared candidates already in the middle of Manhattan. And this super PAC called Leading the Future was formed over the summer and was given $100 million funded by Andreessen Horowitz, by Greg Brockman, by Joe Lonsdale, by Ron Conway. and said they would go after basically anyone who was imposing regulation on AI. And there was actually funny politics over the summer where Trump initially opposed it and said, don't support any Democrat that is like pro-AI, like you should just be a Republican. And so, but anyway, they came out with two people so far who they've named, me, who they named public enemy number one. And then there's someone running for Congress, I think in Texas, a Republican, who they really support. And his, the ad in support of the Republican barely mentions technology. It's just, this is a Trump-loving Republican and vote for him there. For me, they're actually very straightforward. And to be clear, these are smart political operatives who are doing it. I do not think they are dumb. If, but if you were just trying to bury someone, What you would do is you would wait until three weeks before the primary and you would put a bunch of negative ads on whatever the polls told you was most salient, not about the issue that you actually care about. And you do it right at the end so that disclosures would come out after the primary when it's too late for anyone to take that into account. What they did here is the opposite of that in every way. They announced early on who they were, that they were targeting me, that they were targeting me specifically about AI. and started putting ads out saying, don't vote for Alex because he'll regulate AI. And you're absolutely right. These ads, I think, are net positive for me, not only boosting my name ID in a crowded field, but saying that I want to regulate AI, which is a very popular position. So the question is, if they're smart, why are they doing it this way? And there's two clear answers to that. The first is, that they want to discourage any other candidate from making AI part of their platform. It is clear that regulating AI is popular. People are figuring out what they're going to run on. They want to put a stake in the ground that they are coming after you if you ever want to regulate it. And the second reason is because the RAISE Act is currently before the governor. Their ad against me talked a little bit about me. It talked a lot about the RAISE Act, and it had a lot of images of Albany and said, Albany is dysfunctional. Albany is dysfunctional. It was a very obvious message to the governor that they would come after her if she ended up signing a strong version of this bill. And so it is often difficult to separate the politics from the policy, but in this case, they're being explicit in tying those two together.
speaker_2: Fascinating, and you think it is? I mean, I don't know if it's too early to have metrics on this, but you think it is actually benefiting you in your race so far?
speaker_3: Now, listen, they originally said they were gonna spend 2 million against me, then... I got a bunch of free press out of that. I got a surge in donations. I got a surge in volunteers. So they then upped it to they're going to spend at least 10 million against me. And I think if I keep at this pace, I could use up all 100 million of the super PAC and have to spare any other candidate from dealing with them. But, you know, it will be it will be bad three weeks before the primary when they are dumping a bunch of negative ads. But so far, It's been very, very beneficial to me. That still means I'm going to need a lot of support from people so that when all that negative stuff is on the air even more, I can get my message out and fight back.
speaker_1: So I have a question for you, because unlike some of the other anti-AI candidates, I think you have been also clear that there are opportunities in AI. Totally. And I think this morning, Bernie Sanders put out a little bit of a spiel. about how they should block data centers and data center construction should be blocked, and this is something of the 1%, et cetera, et cetera, et cetera. So far, very, very negative reaction among the tech crew, both left wing and right wing. What do you think about that? How do you connect? How do you connect that, a lot of the economic growth in the country right now is being generated by construction for AI data centers with, and a lot of the jobs are being generated, and a lot of them are union jobs too, electricians, construction workers, et cetera. And how do you connect that with, on the left wing of the party at least, this, you know, want to total shutdown on data centers?
speaker_3: I don't support a total moratorium on data centers, to be clear. What Senator Sanders is reflecting, though, is that people feel like this is moving too fast, and they feel like they don't have a say in how the technology develops. And it's being decided by 5 people, and Americans deserve a democracy where we actually get to influence this. So, you know, what I would do on data centers, and to your point, I see the potential gains from AI. I use AI all the time right now, but I also, especially when it comes to medical research and curing diseases. I mean, my mom has multiple sclerosis and autoimmune diseases are some of the most difficult to understand. I'm really bullish on the ability of AI to have medical discoveries. It's just that the same pathways that allow you to have a medical discovery could allow you to build a bioweapon. And it's because I'm so bullish on the capabilities that I think there needs to be regulation. On data centers in particular, the American grid is so old and so in need of upgrades, and we need way more power as well. And we have stumbled into a situation where there seems to be near unlimited private capital willing to pay for a specific thing if we set up the incentives correctly. So what I would love to see the federal government do is instead of allowing states to be played off each other, And instead of allowing long permitting times to have clear incentives set up where I can't get new renewable online for four years, that's way too long to benefit from a data center. But what I can do is go to a... old coal plant that still has approval and fire it back up again. Or I can go to a nuclear plant and say, hey, sell less to the grid and give me some directly for a data center. And so that's what's happening right now. And that's why we're seeing even dirtier and dirtier power support this. I would love to see the federal government say, we will streamline permitting. We will make it much easier for you to connect if A, you use a certain percentage renewable, right? It's tough to say 100% because you need base loads and storage, but like a certain set renewable and You pay for the upgrade, you pay for the interconnect to the grid so that doesn't get passed to ratepayers and ratepayers aren't subsidizing this. I think if you did that, you could have a huge benefit to our grid that helps everyone and not require a lot of federal money in order to do that. So I think there's a lot of creative things we could be doing here.
speaker_2: It's time to build, as they say. Dean is here. So just maybe one more question. I really am super curious about your answer on this one. With respect to Palantir, you worked at the company. I find, AI scrambles all kind of politics. Everything is kind of topsy-turvy. But I'm struck by the sort of coded politics now seemingly being simultaneously super anti-China because they're a totalitarian state and, you know, we wouldn't want to live that way. And at the same time, bringing this sort of mass surveillance tech to our own country, right? And we've just had like an announcement, for example, that people that want to come visit are apparently going to have to hand over their social media. And I assume it's going to be AI reading, you know, into that social media background to figure out who's desirable or not to visit here on vacation. I'm not sure exactly what the, you know, what we're supposed to be getting from that, but Palantir would, you know, potentially be the kind of company that the US government would engage to do it. So if you end up winning this election and going to Congress, How do you think you will engage with the fact that, and Palinger's always quick to say, well, we're only doing stuff that's legal. But Zvi, our first guest today, very often points out that so many things in society kind of rest on the assumption that certain things are hard to do. They're not actually scalable. They're costly. So it might be legal, but people aren't going to do it all the time. AI obviously challenges that assumption in a very fundamental way. The key question is like, how would you engage with these sort of government contracting with a company like Palantir to spin up these sort of, in my view, kind of uncomfortably China-like surveillance operations? Where do you draw the line? Do you even trust Palantir? You know, do you trust Palantir, the company? Take that in any direction that you want, but I'm very curious as to what you would support and oppose in Congress.
speaker_3: There's a lot there. I'll try to be quick because I really want to hear from Dean too. So you have a great next guest coming on. So I was at Palantir for 4 1/2 years. I worked almost entirely on the federal civilian side. So I helped the Department of Veterans Affairs better staff their hospitals and ensure that veterans get the care that they need and deserve. I helped the Department of Justice tackle the opioid epidemic by finding doctors that were committing fraud against Medicare and Medicaid to do pill mill operations and just get people hooked on opioids. I worked with the CDC to better track epidemics. I'm very proud of the work that I did while I was there. And also I left in 2019 because they renewed a contract with ICE to allow the software to continue to be used for deportations. And I didn't think that was the right thing for me to be working on. And so I took a stand there. The The question on the social media coming from the airports and traveling is like, that shouldn't be the policy of the United States. We should be quite clear, not only because of basic human rights and privacy rights, but also We're a country of immigrants, and we're also a country that depends on people coming in and working here. I represent New York City, where we want people to come visit. We want people from all over there, and making that harder is such a self-defeating policy. So I think the broad policies of what we should and should not be doing and what people's rights are should be set in Congress. It's just we've had a Congress that hasn't gotten anything done for far too long, and I think we need to send people there that are actually focused on doing things.
speaker_1: Indeed. And I think we may not have the same politics, but I do want to see more people who are more technically competent in Congress. So I wish you luck, Alex.
speaker_3: Thank you. Thank you, Alex.
speaker_2: Appreciate you being here.
speaker_3: Thanks for being here. And everyone feel free to stay in touch at Alex Boris on Twitter and Alex Boris dot NYC. Appreciate it.
speaker_2: We'll be following.
speaker_1: Next up, we have Dean Ball. Let me get. Dean on.
speaker_4: Hey, how's it going?
speaker_1: Hi, Dean.
speaker_2: Welcome. Thanks for joining.
speaker_4: Of course. Thanks for having me. Good to be doing this.
speaker_1: So Dean, you were the policy advisor on artificial intelligence. You were at the White House Office of Science and Technology. And then you left to, you said, I remember your tweet, you said you were a writer at heart. and you decided to leave. So how are things going? What are you looking at right now?
speaker_4: Well, I mean, so the four months since I left government have been among the most productive in my life. There's just all these very cool opportunities to do interesting and I think positive things. And so The main thing I've been struggling with actually is like what to say no to, and sort of like what the actual, like what is it that I want to accomplish here? And so, but I've been, I've loved being able to sort of speak openly again, particularly because like, I think in my view, this year alone, we've had kind of two major moments that have redefined, at least for me, like how I, the level of utility that this technology has to me. And those were for me like 03 was one, which actually came right as I entered, same week I entered government as when 03 came out. And then in the last couple of weeks, my view is that the sort of coding agents at the frontier are like kind of there. And they can kind of do like basically anything you ask them to. And so it's just kind of been an incredible year in so many ways. And I've been enjoying being able to talk about it.
speaker_2: Can you talk a little bit, I know you heard at least the second-half of our chat with Alex. What do you make of this dynamic where, you know, it's just, it violates my intuitions in so many ways to see a CS masters worked at Palantir with, I would say, relatively light touch, you know, kind of echoing the, you know, the company's preparedness framework constructs in legislation becoming the subject of attacks, which are, it sounded pretty honorable to me in the sense that they're at least talking about the issue that they really care about in the public, which is not always the case, right, in these sort of political operative type situations. But he thinks it's helping him. What is going on? Can you make sense of this?
speaker_4: Yeah, no, it's the politics of AI, politics of AI are in a very interesting state of sort of transition right now. I think the dice are really still in the air in terms of what the coalitions are going to be. it's undoubtedly the case that like there's going to be, there's substantial public sentiment on AI, negative sentiment. You know, thus far it's negative sentiment, but low salience, so low importance. But when you do ask people, so it's like, oftentimes it'll be like the 6th or 7th priority of voters, which basically means for like electoral purposes, it's not really a priority. In certain districts, that won't be true. all politics is local, blah, blah, blah. So I think the coalitions are in a state of flux right now. And I have said, I'm sorry.
speaker_1: Just to interrupt you before we go into coalitions, what are the factions forming these coalitions?
speaker_4: Yeah, well, so I mean, I think even that is in a state of transition. But I think, you know, you have obviously like the traditional AI safety world. And then you might have like the sort of pro-AI, industry heavy type of camp. Though it's not only industry, there's plenty of just like other people in it, myself included, by the way. And then you have what I would describe as this sort of emerging anti-AI truly, not like AI safety, but anti-AI. Those are different things. I think that's really important to understand. Like, I don't think of, you know, Assemblymember Borres as an anti-AI person. I think of him as someone who is concerned about some aspects of AI safety, and I think there's pretty good reason for a lot of those things to be concerned. And so this anti-AI coalition is like, there's people on the left, you have Bernie Sanders saying we're going to ban the data centers. You have people on the right that are also doing this, saying these kinds of things. And I think then you have a bunch of people that are also worried about things like kid safety, sort of more like consumer protection style harms. And so the real question is like, there's these two groups in the middle of, I'm worried about kid safety and I'm more of a traditional AI safety person. And the question is like, which direction fundamentally are those people in the middle going to orient themselves towards? Are they going to decide? that they're fundamentally anti-AI people, or are they going to decide that they're like fundamentally, yeah, we want this technology, we want it to go well, and we think we need guardrails for that? It's a really interesting question.
speaker_1: Is there another faction, because I often note that there have been several comments, we're not going to allow what happened to social media to happen to AI. And mainly, not just about the safety aspects, But I think there was some loss of control that happened from, I think, public leaders to social media. They saw a loss of control, which I often say social media was actually the first proto-AI, and which already caused this kind of loss of control that the AI safety people were worried about, and that elected leaders could actually see it happen. as they lost control of mass media and it became a social media, which is mediated by these algorithms which delivered information to people. So to what extent do you think it's caused by this already kind of like fear of loss of control to AI?
speaker_4: Yeah, no, I mean, there definitely is some of that. And some of it, I think, is like this question of alignment, right? Of like, did the algorithms of social media, did they, were they incentivized to do things that were actually good for society at large? Or was it sort of more locally optimal, but maybe globally quite profoundly suboptimal? And I think that there's like, those are like important questions to ask. And we see the exact same questions present themselves in AI today, right? Like GPT-4O and Sycophancy is a good example of this. I think kind of a lot of the RLHF, heavily RLHF models have this sort of, especially like from the earlier part of this year had a lot of these issues. Seems like they're getting better now. But yeah, like we see some of those same issues. At the same time, it's very easy to fight old battles. And so I see a lot of people that say that sort of thing. Really, they're viewing AI almost entirely as a consumer technology like social media. And that causes, I think, a relatively myopic frame that in many ways totally misses the point. And the only other thing I would say, though, is like, you're totally right that there's ways in which social media has challenged traditional institutions of media and the state in some ways. Like, really, social media is like an authority destroyer. If you were like an old storied source of authority in society and credibility or whatever. Social media has a tendency to erode that kind of thing. And there's ways in which that's good, but there's also ways in which that's like, you know, quite bad. And yeah, I do think there's some people who really understand this acutely. I would point to like Josh Hawley. I think Josh Hawley is like one of his true underlying concerns. He only talks about it sometimes, but I really think one of his foundational concerns, Senator Josh Hawley from Missouri, a Republican. You know, I think a big part of his concern is like this balance of power between basically corporate power on the internet and the government.
speaker_1: Indeed. I also note that for some of them, it's a question of wanting to tax the income stream, which they see that the big firms have a very enormous income stream right now. And And I think, for example, you can see it in kind of Marsha Blackburn in Tennessee. You can see a large portion of the regulation that they want to put forward is to protect artists in Nashville who they're afraid are going to lose their livelihoods because of corporate power in California.
speaker_4: For sure. I mean, of course, you know, whenever people talk about, oh, we have to protect creators from AI, I think to myself, like, well, in some ways, like all three of us are like creators. of different kinds of stuff on the internet. And I speaking for myself, I can just personally say like, I don't really think any of that stuff particularly protects me. feels like it's more about protecting holders of very large IP portfolios. So it's like, the interests of Disney and the News Corporation and similar large media conglomerates, I don't think their interests, like we shouldn't factor in their interest to nothing, right? It's not like, you know, but like, my only point is like, let's not pretend this is some populist cause where we're like defending the copyright of like, we're pretending the intellectual property rights of like the proverbial musician on the street, with a dream and a guitar, right? I don't really think that's what's happening here. We're protecting Taylor Swift, who's like a billionaire, and that's fine. But come on.
speaker_2: Yeah, to protect myself, I would like to propose a ban on Notebook LM so that everybody has to listen to real human voices. Yes. Okay, here's a question I really want to get your take on. And obviously there's a certain, you know, he who must always be named that I think of as kind of a great example of this, an idiosyncratic person you've called a political genius, and I think it's hard to argue with that, but who has made some kind of perplexing moves recently in terms of an EO, which I think seems to be not popular among his base, right? If I understand those politics correctly, like my sense is that people who voted for the president did not do it so that the president would take away states' rights to regulate technology. On the other hand, we're now going to sell H200s to China. So you can comment on those particular details of those perhaps surprising actions if you want to. But then zooming out, If I think about what really matters in history, I would usually think of myself as more of a structuralist thinker that's like, what are the really kind of core technology drivers or the core economic drivers? And then, of course, you've got people sort of squabbling for position on top of those more fundamental dynamics. But I don't tend to think of the individual leaders being like super key forces that shape history. Until right now, I'm kind of like, oh my God, maybe this is a sort of moment in history where the great man of history theory could perhaps have a lot more to say for it because so much is up in the air that maybe these like individual decisions like selling H-200s to China really could become like historical hinge points. How do you think about that?
speaker_4: So I think you are, I think you're spot on that like right now we are in a moment. I mean, I think that Donald Trump, say what you want about him. I do think he will probably go down as being like a great man of history, a person who like really changed the orientation of this country, changed sort of the lexicon of our statecraft and our civic life. Like, and I think, you know, if we're being honest, like with anyone like that, where you're living through the impact of what they're doing, I think you have to be honest that history will be the judge of all this. I think that the, like, In terms of the specifics, maybe like, so I think the president is one thing on the right that I think a lot of people don't understand, like a lot of the people who would be opposed to something like preemption, or at least would say they are, is like, they have this belief that like the president has been corrupted by the tech industry, that like it's the tech industry pulling the strings here. And the president couldn't possibly disagree with them on the core of this issue. And it's like, no, the president does. Like the president cares himself about the issue of preemption. It's very clear from the way he talks about it. It's very clear from how consistent he's been on this issue. He personally cares about it. He personally wants to do this. And, you know, he's ultimately a leader. He's going to lead the party where, you know, where he thinks it needs to go, what he thinks is in the best interest of the country. And I think he's right that preemption is in the best interest of the country. In terms of the H200 thing, I mean, so much has been said about that. I think it's like, we'll see what China even does. Like, are they even going to take the chips? How many are actually going to move? It's so hard to exactly know. I, you know, I personally, like, I think we need to be balancing here between like, we want to give China, we want China to not like completely abandon. We don't want to We want to make sure that we are selling stuff that is better than the best stuff they can produce, such that like we do maintain some ecosystem advantages that are real, right? Like there's Chinese engineers who help optimize who help write CUDA kernels and do other sorts of optimizations to the basic infrastructure of machine learning. And like, that is a big part of how NVIDIA achieves performance is through the contributions of this ecosystem. And so in many ways, like those things have benefits, not just for NVIDIA, but for American developers of all sorts who are building within the NVIDIA ecosystem. That's an important thing. It's, you know, I mean, other just sort of US ecosystems in general, right? Is it the most important thing we need to be thinking about in the context of this competition? No, it's one of many important things. So you have to balance that with the national security issues. And we'll see kind of, also China has to do their own balancing act. So we'll see where they, we'll see what they actually do. It's fascinating.
speaker_2: On the China point, I was really struck recently, I ran your discussion with Max Tegmark as a cross-post on the cognitive revolution. Thank you. scrolling through the names of people who signed on to the Super Intelligence ban statement that you guys were debating, two names jumped out at me above all others. One was Steve Bannon. The other was the CEO of Zipu AI, or Z.AI, which is one of maybe the top five or six Chinese AI companies. They're the makers of GLM. The latest one is GLM 4.6. So that's kind of God. What do you make of it? I don't know. Is there any, I mean, it seems like we are maybe in a sort of Nixon goes to China proto moment here, perhaps. Is there any, do you have any optimism for that?
speaker_4: I think this is where the whole great man of history versus structural forces thing is really going to come to a head because My view of the situation, and I've written this publicly, is that we are in a state of structural decoupling from China, that it doesn't actually matter. In other words, if I'm right, it doesn't matter who the president is, unless the US government starts to take way more control over the economy, like unless we become more centrally planned as an economy. But if we don't, and we continue to have a market economy, which means that private capital allocators make choices about how to allocate capital, they're going to look at the policy landscape and it's going to be like, well, look, I can't rely on China for rare earths because they've just expressed a willingness to use this entire supply chain to derail my business. So I can't do, like I'm the fiduciary of a company. I cannot do that. We've already seen US companies do that, and a lot of them are doing much more of it right now. So we're in this sort of state of structural decoupling China is exporting all this stuff, which does lower prices, but it also competes with domestic industries, not just here, but everywhere. And so there's a lot of people that are having problems with this all over the world and are like, what are we going to do about this? Like, it seems like what China wants to do is extract raw resources from my country and transform them in its own country to things that it sells back to me. And that's basically how China views the future of the world, which is like not a very cooperative or positive vision for the vast majority of people that are not in China. And I don't even think it's that good of a vision for the people in China, honestly. I think it's a good vision for the Chinese government. I think we're in this structural decoupling. The president thinks differently. And I think, I mean, my read on the president is that like he is genuinely interested. He doesn't want to be the person who started a hundred-year civilizational grand competition. He wants to be the person who established A framework for peaceable relations between our countries and therefore really for the world. He wants world, I mean, he really does genuinely want world peace. And maybe he can do it. You know, I'm not going to bet against POTUS. He has surprised me way too much for me to bet against him.
speaker_2: One thing you tweeted recently, Dean, that I thought was quite interesting and quite true is that for the purposes of making sense of what's going on in AI, a lot of the best information is private and is sort of circulated, you know, person to person at the proverbial San Francisco parties, what have you. Could you share in sufficiently abstracted terms so as to not cause yourself or your sources problems that would help people who are not as plugged into the whisper networks have a better sense of what to expect in the next year.
speaker_4: What to expect in the next year? Well, I mean, so I think very broadly, there's a lot of things in AI right now that are just, like, there's a lot of stuff that's quiet because it's sort of politically incorrect almost. Like I felt like yesterday I made the pronouncement that I feel like Claude Opus 4.5 in Claude Code is like just AGI, basically, according to like some definitions at least. And like, that's the kind of thing that is like fairly politically incorrect to say. And might have been, a lot of people might say it like behind closed doors, but you don't, because you don't want to, there's so much career risk of like, oh, am I going to be the guy who came out and said, this thing is AGI. AGI is an unserious concept, blah, blah, blah, blah, blah. I think that one of the things I would guess is that the conversation will in some ways become a little bit more candid, I hope, over the next year, just because some of the realities are going to be harder to deny. And so there's a lot of things right now that like on both sides, there's a lot of like foundational assumptions that both sides have have stuck to throughout the year that have sort of undergirded their rhetoric and points of view. And I sort of wonder whether that will, it's not that they won't, it's not that they'll like be totally, you know, neutral or whatever. It's more that I think that the assumptions will start to change.
speaker_1: And therefore, bearing assumptions were starting to go away.
speaker_4: Yeah, I think the aperture of like stuff you can say and policies you can contemplate will widen. I basically think the Overton window will widen. And I think other stuff too, like the models are going to be really good and they'll probably be autonomous cyber attacks and all that. But yeah.
speaker_2: The first drop in knowledge workers expected Q2 2026 from what I understand as well. Amazing. Well, Dean, thank you very much for joining. I appreciate your participation in our live experiment today. And it's always great to get your take on these big picture questions. You're a unique thinker and we appreciate it.
speaker_1: Next up, we have Peter Wildeford. He is the co-founder of Metaculous. He is a policy strategist. He is the globally top ranked top 20 forecaster. This is going to be exciting.
speaker_5: Hey, yeah, thanks for having me. Sorry to take you away from Dean, but happy to be here. By the way, I'm not the co-founder of Metaculous. I wish. I'm on the board of Metaculous. I'm an avid Metaculous participant, but yeah, so I work at the Institute for AI Policy and Strategy.
speaker_2: You guys are, since we were just talking about chips, you have been a opponent of selling chips to China. I think that's fair to say. What's your take on the latest development?
speaker_5: Yeah, I mean, I think my take is that, like, obviously there's something to be said about wanting to undermine demand for Huawei chips. But I think the mistake that people make is that they assume that China is like a capitalist country and will respond to supply and demand the way that other countries do. And that's just clearly not the case. The Chinese government will make sure that Huawei has unlimited demand for their products. Right now, Huawei can only supply like 1 to 4% of Chinese demand. And the Chinese government is going to make sure that there are companies that take that demand by law, by software requirement. And so then the question is like, well, what about the remaining 96%? That's like not being able to be filled by Huawei. Well, I think in that case, like that's where Nvidia is kind of a useful idiot. Like they're kind of filling in that like missing 96%. And China's going to be like, thank you for helping turbocharge our AI industry. But then the moment Huawei can take in more and more demand, the government's going to be pushing out Nvidia and they're going to be promoting Huawei, putting really, really firm thumbs on the scale there. And that's like, that's what China's done in every other industry. Like Tesla's come to China thinking they can sell a ton of electric vehicles. They got pushed out for BYD. Lots of other companies think they're going to like, Apple they would be selling a lot of devices in China got pushed out also for Huawei. Yeah, like the Chinese government, like Uber, I think also got pushed out. Like, yeah, just like this is how it works in every industry. And so I think we need to have more of an understanding of how supply and demand actually works in China and like how these chips like are actually being used or not used to get like what is the like how is Huawei being undermined? Like I really don't think they're being undermined at all. I think China's kind of, they're having their cake and eating it too by boosting Huawei domestically and also leapfrogging off of Nvidia chips. Yeah, so I feel quite strongly about that.
speaker_1: So what would you say if the chips issue was put in as a part of strategy to kind of stress the Chinese government into overspending. So it's not that the US policymakers really believe in AGI or ASI, but it is kind of a red flag to a bull, kind of causing the Chinese government to overcommit on construction, overcommit on chips. At the same time, they're going to have to match SpaceX on launches. They have a real estate bubble, which has just burst, and they're trying to address that. So they've got all of these things going on at the same time. A lot of needs for capital, a lot of capital burn. BYD and the other EV companies are now, are all negative capital, and the government is having this anti-involution kind of campaign to prevent people from producing at massive losses. So basically, it's similar to the strategy that the US used against the USSR, which was to start off the Strategic Defense Initiative, stress the USSR on spending. At the same time, oil prices went down. The USSR then had a collapse. So in the same way, this is maybe even policymakers do not believe in AGI or ASI, but it is a very useful kind of way to encourage spending in China and overspending and overcommitment on these strategies.
speaker_5: Yeah, I mean, with all due respect, I do think that's kind of a bit of a galaxy brained take. Like it seems to be really relying on 2nd order effects and ignoring like the premise of the 1st order effects. If you believe, I guess what I do about AGI, ASI, like I think a lot of this like big spending is very rational. China might actually be underspending relative to the advantage that AI can create for a country. when it becomes kind of like super powerful, as I'm expecting. I think, yeah, like the US is, if the US is going to win an AGI, it's going to be largely by outspending China by having deeper capital markets. I think that's always been the US's strength. And I do think the Chinese government, the Chinese economy is under strain. I think that China can only prop up the government so much. And I think that they had a lot of pain from zero COVID, which I think was a tremendously ill-advised policy and I think shows that the Chinese government isn't omni competent. But I think if like if the US just wants to like trick China into spending, I mean, like why not just like race to Mars or something? Like there's like far more, there's like far more safe things we could be doing than giving China everything they need to like power their military, power their economy. And like, and like it's clear, like China's weaponizing their AI against us. They're like cyber attacking us. relentlessly, like they're not playing fair with the technology that they're using with us. And they're also using AI to do tremendous human rights abuses domestically. And I don't think it makes sense for us to be powering that.
speaker_2: So is that the core of your objection? I feel like one of the things that always gets kind of... glossed over, often gets glossed over in these sort of, why should we or shouldn't we sell ships to China debates is like, what is the actual threat model? My sense is like, they have enough production to power their drones. They probably have enough production to power their surveillance stack. And even though at one point in time, right, if we, I once ran a deep research report on the history of justifications for the chip ban, and it sort of started with like, well, we'll deny them the military applications, but then it was like, well, we can't probably do that, but we can at least deny them the ability to make frontier models. And I was like, well, now we can't really do that. And the latest version that I've heard from like Leonard Heim, who's kind of been a big advocate for this policy is like, we at least we'll have a lot more AI agents than them. So like we'll be a lot more productive economically and they won't be. And when I hear that, I'm kind of like, I don't know, man, that doesn't, that sounds like not really super consistent with my values. Like I'd like to see the Chinese public enjoy the benefits of AI agents, you know, in their day-to-day life as I hope to enjoy them in my life. it doesn't seem to me like we can really do much about their military applications or their surveillance tech or even their cyber attacks. Like those are going to happen. I also note that like my life goes on pretty well today without too much interference from actual cyber attack damages. So the other, of course, answer that people give is like, well, if they get this, then there'll be like 5 plus more companies in the AGI race. And that's just going to make the whole competitive dynamic worse. And we'll have even more of a rapid race to the bottom. And that resonates with me a bit more, to be honest. But then I'm also kind of like, well, maybe we should regulate our own companies a little bit if we really want to do something about that. So I guess all that to say, like, there's kind of a menu there. You can take from the menu or go off menu. What is the core threat model that you really worry about when China gets chips?
speaker_5: Yeah, I mean, I want to be really clear. it definitely should not and it cannot be the policy of the United States government that we want to make China poor. And we should not be trying to imply that to China at all. I think China deserves economic success as much as any country does as much as the United States. And I completely agree. Like I want the Chinese people and even the Chinese government to be benefiting from AI technology, even benefiting from American AI technology. I think when it comes to the chips in particular, though, I think the threat model is like, kind of like once you give chips, you can't really take them back. I think it also is just kind of, it's really underrated, I think, the amount of the current Chinese AI ecosystem, Deepseek, Quen, et cetera, that are being built on American chips. It was just reported by the information that Deepseek is smuggling American chips. I think if America really wanted to and like really could crack down on chip smuggling, crack down on chip like sales, like I think China would be much further behind the frontier, especially with enough time for those policies to work. And I think like the core threat model is that I think AI will be kind of like able to, like you said, drop in remote workers earlier, kind of drop in military analysts, drop in like military commanders. I think a lot of like military AI will be just like a tremendous strategic asset. And I think we want to be really careful about like, I mean, China has so much civil military fusion. I think unfortunately it's like very difficult to be giving China these equipment that then can be militarized and like, yeah, if AGI does really take off and if there is some like. God help us, some like US-China great power war. Like we're not going to be wanting China to have all those like drop in military strategists. Like we want, we'll want the American military to have that. And so I would say like we should rent chips to China rather than sell them. We should make sure China can benefit from American AI rather than have the materials to compete with American AI. And we need to be careful, I think, about the military implications, kind of like how I think the United States should sell a lot of things to China, but we shouldn't be selling our fighter jets or our missiles to China. Like, I guess that's like obviously more of like a clear example, a little bit inflammatory deliberately, but kind of like the way I see chips and the way I think military will eventually be tremendously super powered by AI. I think that like is a non-trivial portion of the Chinese AI ecosystem and eventually their military capability. And I think we obviously are doing fine now, but if we do need to be prepared for some sort of Taiwan scenario, some type of great power war scenario, and we shouldn't be suffocating China in the quest to do that, but we should be thoughtful about like, what are we or are we not selling? And like, what are those implications?
speaker_1: What is the difference between, let's say, China buying the chips and setting up data centers onshore and ByteDance buying the chips in Singapore and deploying them in Oracle data center in Malaysia across the border. What are the major, because, and as you imply in the civil military fusion, I'm sure ByteDance does some work for the government, right?
speaker_5: Yeah, I think they definitely do. I mean, I think the biggest difference with renting chips as opposed to buying is that if there were some sort of war, you could always turn the chips off. If they're like kind of more on Malaysian shores, we just have so much ability, like if they're controlled by Oracle, we just have so much more ability to shut them off if we think that they're being misused. And then I think also there's like a bit of a dynamic in China too, where the military will be tremendously disincentivized to use Malaysian data centers because they don't want the United States like spying on their military like workloads. But commercially, I would imagine Chinese companies would be happy to use Malaysian Oracle run chips. So like that, I think that actually is a great way to benefit the Chinese people economically without benefiting Chinese militarily.
speaker_1: Interesting.
speaker_2: Yeah. I like that a lot.
speaker_1: Maybe just segue to Nathan, go ahead.
speaker_2: I was just thinking, I like the rent don't sell paradigm a lot.
speaker_5: Yeah, I think rent don't sell is my favorite way of trying to square the two things of like, yeah, I mean, I am definitely like a China hawk, but like again, I really don't want the Chinese people to be poor. This is not like an economic war against China. This is just like sensible military policy. And I think permitting renting, but not selling them the chips, I think is the best way to square that. I know other people disagree with me on the renting thing, and I'm happy to have that debate with them as well. I guess this sort of feels like maybe a more moderate position in that case.
speaker_1: I'm going to add this to the stage, which is the the famous Arc AGI one leaderboard with a 390 times improvement in one year. I think that you had a tweet that this was surprising to you or?
speaker_5: Yeah, I mean, I do think it is surprising. So one of my favorite things about this evaluation is like, I mean, I feel like every eval we ever see, like emphasizes like, oh my gosh, like AI is improving so much in like ability to get stuff done and capability. And like, that's super cool. But I think what goes a bit unnoticed is just like how much cheaper it's getting to do things as well. Like so the old capabilities you used to be able to do just barely can now be done by like open source models and maybe even models that you can run on your laptop and soon models you can run on your phone. And I think that has a lot of implications as well in terms of like what we're going to be able to do with AI. And I'm very excited about that. And so I think just kind of seeing wow, not only is there tremendous increase in capability, but tremendous increase in affordability. And you kind of get to multiply that, like affordability times capability. And I think that just really shows an awful lot of opportunity, especially as that presumably will keep continuing.
speaker_1: Is this one of the trends that you foresaw in your 2025 forecast? Did this surprise you?
speaker_5: Yeah, I mean, I must admit that I didn't like look at this trend in particular, but I guess I do find myself sometimes a bit surprised by just like how quickly it is that you can take existing capabilities and just make them a lot cheaper, even sometimes only over a matter of months.
speaker_2: I looked back at our respective AI 2025 predictions and not surprisingly, yours seemed to hold up a little bit better than mine. I overestimated the benchmark scores by a bit pretty much across the board. You were a little more accurate. I think I was a little closer to the revenue number though, which was the one number that really blew out relative to the, you know, the sort of aggregate community forecast. So this is kind of an interesting tension where the benchmarks haven't gone quite as far as I expected, but the revenue has really ramped. And then when you juxtapose that against the fact that like the same capabilities are getting dramatically cheaper, and yet we are still kind of telling ourselves, all that so far would suggest that, well, we must be seeing just like infinite use on these sort of mundane tasks. Then we also have the narrative that like, well, we haven't really seen much economic impact yet. So how do we square the those three things. Like, obviously, progress has been pretty significant, but a little less than I expected. Revenue growth has been huge, even as cost has come down, and yet we still are like, where's the impact? Like, what do you make of that sort of very seemingly confusing constellation of facts?
speaker_5: Yeah, that is confusing. I must admit, I did a really, I made a pretty big mistake when trying to forecast revenue for this year, which is basically at the end of the year, companies, OpenAI, Anthropic, et cetera, all just like reported, here's how much revenue we expect to make over the next year. And I thought like, oh, well, those are the company's official projections. Like surely they know much more about revenue than I do. Like surely you can just like sum them up and that's total revenue. And like, why would you try try to do something else. And I think one thing I learned this year was like, I guess when companies put together their revenue projections, they have to use really standard economic models and math and they're not allowed to account for the craziness of AI progress because that's just like unprecedented and you're not allowed to put unprecedented stuff in your economic models or you look unserious. And so in retrospect, I think that was like a really dumb prediction on my part. I definitely should have known that like, oh, the revenue will obviously be a lot higher because I was expecting AI progress that wasn't being priced in by like the standard economic models, which I think are primarily backward looking. I think it seemed like revenue this year was kind of really explosive off of coding models in particular, like Claude Code, Cursor, Windsurf, et cetera. Like that really sort of was a breakout use case. I was expecting, like one thing I was a little, kind of you said like overestimating capabilities a bit. I was expecting more economic value just from like other sorts of computer use agents besides coding. that kind of didn't really take off this year. It wasn't really the year of the agent as much as like I or others might have said. But like I think AI just became tremendously good at coding in particular as like Yin Bal was talking about. And that just ended up being a very economically valuable use case. Like I'm always surprised like I was, I was literally a professional software engineer, data scientist for five years. I was coding every single day, hours on end. And even these days, like Cloud code so much faster, so much better than me. I basically like rarely write code myself anymore. Just like always would rather like have Cloud code spit it out for me, at least for the first draft. So I was very impressed by that.
speaker_1: Yeah, I think, you know, just to add to that point, The thing that I always point out is that the CFO of the company has to have some sense of reality when dealing with investors. And so the researchers do not. So there's this gap between the researchers and the management. And the CFO, often externally facing, has a certain number of things that they are allowed to say and a certain number of things that they are not allowed to say. And one of those things is, declaring a 10X on revenue in any year is not something that anyone will take you seriously, because they will ask you, what is the product that you have? Who are the customers that you have signed? How does this money come in? And just to take a step back, Claude Code at the beginning of this year, they launched it late last year. It had pickup, but it didn't have this kind of 10X pickup. It was growing very strongly, but took 10X in one year is amazing. It's fantastic. I also wonder to what extent do you think they can 10X again next year? Because again, projections for next year are kind of like 2X to 3X for each of these firms. Do you think they're going to exceed again?
speaker_5: Yeah, I mean, I would think yes. I mean, I admit there is a certain level of 0 to one where like people weren't using AI at all and now they are using AI and like maybe some of that can't be recaptured. And so that obviously would like slow growth down a bit. But like, I mean, I think kind of like the true like game-changing AI use cases like still aren't really here. Like if companies actually can land something that is like akin to like a drop-in remote worker or drop-in management consultant or something. Like companies pay like hundreds of thousands of dollars a year for that sort of thing. Yeah, we're not talking about like $20 a month for your Claude Code subscription, but like 20,000 a month for your like remote worker subscription. And so again, I think it really sort of depends on can this like train of increased capability continue? And I really think computer use agents could be enormously economically valuable if they could actually be delivered in a sufficiently reliable way that like meets the promises that people have, like myself, have been hyping up a lot over the past few years.
speaker_2: What are the sources that you are watching? You mentioned, obviously, like the company's official revenue projections. We've got, you know, obviously Twitter where people are signing off all the time. I do find that honestly to be still my primary source of up-to-date information. But then there's all these prediction markets as well, you know, some of which are more kind of like Metaculous is a little bit more geared toward like the true, you know, forecasters forecaster, and then others are more kind of betting markets. Interestingly, it seems like recently we've seen sort of a surge of insider activity on some of the betting markets, which has been a really interesting phenomenon. But what are the sort of, you know, SIGINT sources that you are tuning into to make sure you are as informed as possible that you would recommend other people pay more attention to as well?
speaker_5: Yeah, I mean, I think you're totally right. Twitter's my go-to as well. Also, yeah, pretty happy with V's like aggregation of Twitter on his substack. I think a lot of other substacks are pretty excellent as well. Yeah, like Tim Lee's like understanding AI or Nathan Lambert's substack, I definitely like a lot. I'm also a big fan of following semi-analysis, following Epoch AI, following Meter. I think kind of these organizations that have like a reputation for not like hyping stuff up, not vague posting, but like kind of really getting into the details of like where is AI on these evaluations, like how are things trending? And more importantly, like what are these evaluations actually mean and like what are they covering and what are they not? Something I've really liked seeing from Epoch AI, for example, is like just seeing like real professional mathematicians like comment on how useful they're finding AI math assistants. And like, yeah, like I think Daniel Litt, for example, on Twitter, like every lot of times when a new AI model comes out, he like puts it through its paces with his like professional mathematics work. And I mean, these are like AI systems that can win mathematics challenges. And they're definitely seem to be approaching some level of value, but they still seem to, and they're doing so well on this like frontier math benchmark, but they still seem to be lacking in like their ability to deliver for professional mathematicians. But like, I think as that changes, we'll be kind of seeing from these like in-person taste testers. I've also been kind of like taste testing the models myself, especially with more like writing or policy analysis tasks that I think are just like not really well covered by benchmarks. Yes, like I think getting like a holistic picture seems important. I also like that like meter looks into reliability as well, because I think it can be like one thing to do well on some like hard problems in a benchmark, but another thing to be like reliably delivering in a professional world. Like for example, if I only succeeded at 80% of my policy analysis, I don't think I would be keeping my job very long. And so like, yeah, so it's like the jaggedness of these models, like they can succeed a lot, but then just like mess up on something really stupid makes it a bit harder for them to be a drop-in worker right now as opposed to a working assistant, which I use them for in an assistant way.
speaker_1: Do you assign like a probability of being correct or wrong, like handicap some of these organizations and statistics more than others?
speaker_5: Yeah, I mean, I think so. I mean, I've always been a bit frustrated, I think, especially by OpenAI, where they do all these vague posting, hyping their models, like posting Death Star on Twitter and talking about how their model is going to be like, or like, I think Elon Musk, like every single model he's ever built, he's like, oh, this is the one that's going to be AGI. And you're just like, yeah, come on, like, stop, doing that. Like, it's just really unhelpful and like also needlessly stress inducing. for those of us trying to follow this kind of stuff. I feel like now you can just sort of tune it out. But yeah, I think like it was just like adding more noise than signal in the past. Yeah, so I mean, usually when models come out, I just advise everyone to like ignore the news for the first time. 24 hours. Like every tester is like, I got early access to Model X and it like was the best model ever. And it like did all my homework and it like built me a new website and it like cured cancer or whatever. And it's just like, well, the only reason those people tend to get early access is because they're going to like write glowing things about every model no matter what. So like, I really like again, Epoch and Meter organizations that have more of a track record of not, of just kind of calling it as it is, as opposed to hyping up the model or saying that the model's stupid when it's not.
speaker_2: As we look ahead to 2026, I think it's hard to, when you get into forecasting, which I've done a little bit, a little more about me, I participated in one of the original Good Judgment Project forecasting tournaments like 15 years ago. I did okay, but not as well as you. Actually did pretty well, but still not as well as you.
speaker_5: Nice.
speaker_2: One of the things that I've learned though in doing this is like, it is very much anchored in questions where you can get a number out, you know, in a lot of cases, right? It's sort of, it tends to lend itself toward in paradigm changes. You know, we have a benchmark, current number is X. What's it gonna be at the end of the year? How do you think about things that are a little bit more paradigm shifting. Like if I were to ask you, what are the odds in 2026 that we get a new paradigm on the order of like the chain of thought reasoning paradigm that took us from sort of language models 1.0 to 2.0, let's say, a 3.0 could be a bunch of different things. It could be a continual learning breakthrough. It could be something else. It could be embodiment in some way, shape, or form. How do you think about trying to get a handle on those things, which are not like a well-defined, scale or move, but rather something that's like kind of qualitatively different than what we're currently dealing with.
speaker_5: Yeah, I mean, I think that's definitely an astute observation. I think with forecasting in particular, there's so much fuss put on the task of take an existing forecasting question that has well-defined resolution criteria and put a number on it. And that kind of activity, I think, is an activity I'm very good at and does have a lot of value to people. But it's only a small part of the overall forecasting value chain. I think like two things that are really dismissed is first, like, what questions are you asking in the 1st place? Like, what are you actually posing to people? Like, what are you allocating your attention to? And then secondly is like, once you have this number, this prediction, like, what do you actually do about it? Like, how do you make decisions different than you otherwise would? And like, yeah, so I think like the full forecasting value chain is like way more than just like putting a number on stuff. It's also like, yeah, asking the right questions and doing the right things with the information. And I think a lot of times, a lot of AI forecasting can be really stuck in assuming current paradigm continues and current paradigm is the only paradigm when I think like looking back throughout history, like we have, I think a decent amount of evidence that we get like new paradigms, like honestly, even like every five years, like twice per decade or so. like it stands to reason like any particular year might have like at least a 10% chance of having some new paradigm developed that does like kind of qualitatively change model performance. I think like one of the interesting things also with the reasoning models is like we all kind of assumed pre-training was like the only kind of training. I think previously it was just called training and you would just have like more and more flop for models and you were expecting kind of more and more returns from that. But it kind of felt like pre-training, I guess, has like been burning out a bit. But it started burning out a bit like right at the same time reasoning models started taking off. And so you almost like continued this like linear AI progress just through like a second era where like the first thing kind of stopped working, but this new thing picked up just in time to continue it. And so I think you could actually get maybe surprisingly linear progress despite like kind of a changing guard of paradigm. But yeah, I think definitely It's like sort of difficult black swan kind of events to think about like, oh, will 2026 actually see some form of continuous online learning? Will it see some form of like. like the chain of thought's no longer in English. Now it's like thinking in like raw neural pattern and that improves its ability to like represent concepts and it's just like so much better now. Or like, will there be some new data efficiency algorithm where like, rather than requiring like 100,000 examples, it only requires like 10 or 15 like a human. And like it can like learn so much faster that way. Like I think, yeah, we really haven't like scraped out like what's possible. So I think still does kind of like widen your uncertainty definitely when thinking about next year or even the year after that.
speaker_1: What do you think the end of 2026 looks like? The most likely case in your mind can be anything that you envision, but what do you think it feels like at the end of 2026?
speaker_5: Yeah, so I think by the end of 2026, I think you'll So I mean, one evaluation I've been tracking a lot that I like a lot is like meters evaluation where they evaluate AI on their like ability to do software tasks and then they like compare roughly like how long does it take human professionals to do that task. And right now AI is achieving like 50% reliability on tasks that take human professionals about two hours. And so like that's enough to get reasonable value out of Claude code or something, but still like the human is very much involved. But I could see at the end of 2026, I think it could be possible that you could be having AI that could do like a day or two of human labor. So like it would just be like significantly more autonomous, significantly more able to just like run with things, do all the trial and error itself without having human input every like 20 minutes like cloud code currently does. And I think that really could transform, yeah, like transform kind of how we relate to AI and like what AI is capable of. I think one other interesting finding from Meter that they looked into was like also it's not just like, it's not just software that is like increasing gradually. in capabilities, but kind of almost a lot of tasks, a lot of tasks you can measure, whether it be computer use or economic analysis or legal performance or self-driving cars or anything, kind of have similar capability increases over time, just starting from much lower basis. Like right now, maybe an AI can only do like a 10 second computer task and then it gets confused or whatever. Like I would imagine by the end of the next year, like I could, I would expect AI to be like using computers, like kind of at least a human level. And so I think that would really transform again what AI can do. Like if AI can sort of do, like, I mean, the computer right now, the internet is just kind of the gateway for like a lot of economic activity. And if you could start delegating that to AI and just have even some semblance of reliability and maybe even multiple trials, it tries five times to do the thing and messes up four times, but that fifth time it knows it succeeded and then it reports back. And maybe it's horribly inefficient compared to a human doing it, but it's so much cheaper and it can also run while you're asleep or run while you're doing something else. I think that still would be pretty valuable. And I think that will again transform what we could be getting from AI. so I know I said, I kind of said 2025 would be the year of the agent and that sort of didn't really take off. But there has been a lot of improvement in computer use. And so I would think that like 2026 might be the year of the agent in some sense. I'm happy to just repeat that prediction again and see if it sticks again this upcoming year.
speaker_1: What do you think will be the political ramifications of that? Because you've already got like, I think Sandra Fetterman, was he wants, if you're going to have self-driving trucks, he wants a Teamsters representative on board. Bernie Sanders this morning with, you know, moratorium and data center build outs. What do you think as you get to the year of the agent, what do you think the politics of this looks like at the end of the year?
speaker_5: Yeah, I mean, I think the politics will be heating up. Definitely. I think we've already been seeing that. I think 2026 will also like, you'll be seeing Waymos and self-driving cars. way more commonplace, hopefully. I could see like if there was some like AI agent, even if it did like fairly rudimentary things, like if it was like affordable and fairly universally used, like I think that could be another. ChatGPT moment, like kind of again in 2023, like the first like actually usable chat bot with a good UI, the trends were all there. Like all of us studying AI, like weren't that surprised by ChatGPT, but like because it had such a slick user interface, like it created like a transformation in politics. And then all of a sudden we were having like international summits on AI and stuff when previously it wasn't the talk of tech policy at all. I could see like, crap, like the AI is not a chat bot anymore. It's actually doing things in the real world. It's like getting my Uber Eats and it's like booking my plane tickets and it's like able to do these actual practical things. Like I think you could more viscerally feel how that leads to unemployment. Maybe it actually does like tick up unemployment in like a very small amount, but like in a large amount to be noticeable. And then I think, you're heading into the midterms. And like we're already seeing just this past year, there's been a lot of populist anti-AI sentiment. Like the data centers are, there's a claim that they're like using all the water. There's a claim that they're running up electricity prices. Like people are already on the watch for affordability. And I think they like don't like big tech. It's easy to blame the data centers for your problems. It's easy to blame big tech for your problems. And maybe you don't feel like you can really do anything about OpenAI or Facebook or something, but you definitely can do something about that data center in your neighborhood. And so like it is kind of a way for individuals to feel like they have some power over what's going on. And I'm expecting, yeah, politicians to continue to use that. I definitely think, yeah, like AI, it's already been kind of a big part of tech policy this past year. I think it'll be an even bigger part of tech policy in 2026. Unfortunately, I don't feel like we're going to really land on any actual solutions that get passed. I think kind of most midterm years tend to be fairly acrimonious and not really solutions oriented. And I think that's super unfortunate and I want to work on that. I want to make sure that we're getting good use cases like Waymo's, while of course not like running up way too much risk from AI run amok, like AI with no guardrails whatsoever, like a lot of big tech seems to want, but finding somewhere in between and harnessing that anti-AI sentiment towards more productive solutions seems like a really important thing to do this upcoming year.
speaker_2: Last one for me. Do you want to venture a year when I'll get a domestic service robot that will come into my house and do my laundry and cook my meals and make my coffee in the morning.
speaker_5: Yeah, great question. I mean, I think that like robotics has always been on the disappointing side. And I think there's also just like not really good evaluations for it. But I kind of have a sense that like, once robotics starts working, it will just like really start working. And so like, we may just see kind of a lot of robotics progress kind of feel like it comes out of nowhere. Sort of like I think how self-driving cars like weren't really economically viable for the longest time. Like I remember like even way back in 2015, I thought I might have a driverless car pretty soon. And that was like just wildly off. I wasn't a good forecaster back then. I was just a college kid. But now we're finally seeing it happen about 10 years later. So like, I mean, I would do a venture maybe, by like the end of the 2030s, you have sufficient AI progress and sufficient follow on robotics progress and sufficient reliability that, and also just like sufficient cost reduction that it's just like cheaper to hire the robot butler to come instead of the human butler. Obviously, I guess that would be like, tremendous for individual consumers who then can like outsource a lot of their chores. I certainly would like that. Of course, that's like obviously very difficult for the American worker that would normally work that job. So yeah, they're definitely really important trade-offs there. And that's going to be a really important policy conversation for sure.
speaker_2: Yeah, it feels to me like we're maybe four years behind in robotics compared to the chat bot. Like we got ChatGPT late 2022. We're going to start to see the first humanoids shipped to consumer homes next year, it seems like. They're probably not going to be great. They might have a sort of GPT 3.5 level reliability. But if it keeps pace, then in a, you know, in a 2030, you can imagine getting AIs along the, you know, getting robots that are sort of performing similarly as a like Claude 4 or 5 Opus, perhaps.
speaker_5: Yeah.
speaker_2: And that would be, not necessarily perfect and able to do every last thing I might want in my home, that would definitely be a product I would be keen to sign up for, especially if it could bring the character of Claude into my home.
speaker_5: Yeah, that would be super fun. Yeah, I mean, I think we haven't really seen the ChatGPT moment for robotics yet, but like, yeah, I mean, the ChatGPT moment was only two, three years before the Claude 4.5 Opus moment. So I mean, definitely once you start hitting on something, it can be very quick until it's like very useful.
speaker_2: Yeah.
speaker_1: I feel like the physical manufacturing servos, et cetera, are actually in better shape than the AI used to control these things.
speaker_5: Like there's a lot more.
speaker_1: Yeah.
speaker_5: I mean, but one thing I did learn recently was I think like robots only have 10,000 tactile sensors on their hands, which sounds like an awful lot of sensors. And that is definitely a lot to cram into one robot hand. But I think humans have only almost a million different touch points on their hand. And so yeah, there's just like still so much scaling to be done to meet human performance. Yeah, it really does speak to that more of X paradox that like, yeah, doing the dishes is just like so much more difficult than like doing a math competition when it comes to AI.
speaker_1: Incredible.
speaker_2: That might be a great place to leave it unless you have any other bold predictions you want to give us for 2026 or beyond.
speaker_5: I'm hoping I'm predicting a successful live show and hoping you'll do another one next year. And I'm wishing everyone, yeah, a merry Christmas or a merry happy holidays of their choosing if you're not of the Christmas persuasion. Indeed. Yeah, looking forward to it. Thanks for having me on.
speaker_2: Happy holidays. Appreciate it, Peter. Happy holidays. Bye.
speaker_1: Wow. What a show. What a marathon.
speaker_2: I'm not even sure I can properly step back to reflect on it just yet, but it was certainly fun. And I do think we, up until the end there, where we kind of let the time with Peter go long, we at least stayed on time.
speaker_1: That was by choice, because you know, we'd been pressed for time the entire, you know, the entire series, so... we'd want every speaker you want to continue and then you end up having to cut them off. It's a tough one. I found Boris interesting. He's very charismatic. You know, it reminds me that, you know, politicians are much more charismatic than tech people. Very charismatic. I tried to draw him out on the crypto stuff. He didn't bite. So I think that was interesting for me. Interesting moment. I found, I think, Dean, was much more optimistic, I'd say. I found him very optimistic. So that was good. That was a good differentiation from Zvi, who started off quite, I think, quite negative. And overall, it also struck me that one of the things that struck me was how many of the developments that we were talking about had really only happened in the last couple of months. like the GPT-5.2, the 390x cost improvement, that was just like a couple of few weeks ago. Nested learning was not that long ago. It's really, when you talk to people about biotech, they talk about things that happened like a year ago or like, you know, 18 months ago, right? And you can really see kind of like the, how fast things are diffusing, at least, from the research to the community. but perhaps not quite into the economy yet. And you could feel that. You could feel that. I could feel that everyone, I think everyone on the show expected more and still expects more for the next year, which I think is a little bit different from the market right now. The market thinks AI bubble, this is it. So the market has kind of flattened out expectations for the year. It's very interesting to watch the to hear the insider views, which are quite different from what the market believes.
speaker_2: When it comes to the market, one thing I always wrestle with is, and I think actually Noah Smith had a pretty good write-up of this recently, where he was like, the tech is for real. You're not going to get a sort of bubble burst when it comes to what AI can do. Obviously, it can already do a lot and it'll continue to do more. But that doesn't necessarily mean that there will be durable high profit margins for the companies that makes the financing of all this make sense in the fullness of time. So I wonder how you think about that, because I'm kind of like pretty confident that we're going to get to some kind of, as Dean already, you know, Tyler Cowen called AGI with O3. Dean is kind of soft calling it with Claude 4-5 Opus. I, if we don't want to call it yet, I'm still confident we're going to get to something that will qualify. But I'm not sure that, because I, and this is one thing I wanted to get to with Logan, but we didn't quite have time. It seems like switching costs are rising, but they're still not that big. And part of the reason, you know, people maybe don't switch as much as they otherwise could is just because all the models are pretty cheap or we're all, you know, we're all benefiting from the competition. Nobody wants to be dramatically undercut. So they're all pricing pretty low. But it does feel like we could see sort of progress continue, but also price wars continue to the point where maybe the economics of it do kind of go haywire, even as the capabilities progress. Maybe not quite as fast as I predicted, but not necessarily too dissimilar. What do you think of that?
speaker_1: So one thing that strikes me is that the useful models may get smaller over time. Like Gemini 3 Flash, for example, is a breakthrough in efficiency, obviously. Like it's as, it's on par with 2.5 Pro, but it's so fast, it's obviously a much more economic model to serve. And so that makes me think, okay, if you have a model that can do a software engineer's job today, but it requires A Blackwell chip, And in 12 months, you have a model that can do a software engineer's job, but it requires an H100 chip, meaning you go back in the chip, you get so much more efficient that you're using an older chip. Do you still make the same economic value from the software engineer? Because it's the same work that's being done, like on the B100 or the H100 in 12 months. And so that speaks to me that, you know, this is what Jensen said, that H100s are still being used. Because as things get better, you are able to deploy more advanced, smaller models on older chips. And the economic value that those models are providing is still there. And that really confuses, that I think really confuses a lot of the assumptions that people have. It's not that you need more, like better chips for more capability, but you can also serve the same capability with older and smaller chips. And that's something that we've seen even, for example, RKGI, the tiny recursion model was a 7 billion parameter model. So much smaller. And if the economic task was RKGI, I'm sure in 12 months, you'd have a 7 billion parameter model, which was at 80% scoring equivalent to a 5.2 pro, right? So over time, a job is a job. That software engineer's job is still being done. It's being done on an older, smaller chip. that economic value is being created. If you have a lot of software engineers, the demand starts to, you get demand, you get oversupply, obviously. But in a world where we have too few software engineers and where software engineering creates a lot of value, and over time, you know, this idiot index of our brains using 20 watts to produce a software engineer and, you know, these models using, you know, 1 MW hour to produce a software engineer. And that, we're closing that index, right? That idiot index is closing over time. And I feel like that means that you get more economic activity on older chips over time. And I think that is something that will become more apparent. You can see it in kind of like the booking numbers for like some of these firms with older chips. But I think it'll become more apparent by the end of next year, because I think there'll be more Claude code type software engineers out there. and creating more code. And that, I think, might be where you start to see, okay, this is real economic activity. Even if you spend on these data centers right now, they are actually going to be useful six years from now, because six years from now, we will have so much better models, smaller models that we can still deploy on these chips and we can still get economic growth out of them. I don't know. That's what I feel. What do you think?
speaker_2: So does that bottom line out to you think the financing can be sustained because these things will have a long serviceable life, basically?
speaker_1: Yeah, I think so. I think like the way, for example, like a meta things is that if I get super intelligence, great. If not, we're going to generate lots of slop AI. And the slop AI, and you know, they're definitely going to do personalized automated advertising. It's going to come. You're going to, you're going to You're going to open up your Facebook and like, you may have like someone who looks very similar to your wife or your kid pitching you on a product. It is going to get, it's going to get real, right? And video Gen. takes a lot of, a lot and a lot of GPUs and the economic benefits are going to be there for advertising. It's clear personalized advertising will come. And that is like a fallback position for these companies, for Meta, for Amazon. Worst case scenario, do slop ads, right? You have the economic benefits there. And that's how they think, right? They're like, they build out to support that they're not going to go bankrupt and they can still build these things. Worst case, they go into junk pond territory. They live a few years with, you know, kind of junk pond type like credit ratings and, but they're confident on the cash flows because, you know, at this point, a lot of the economic activity in the US is now concentrated on these like $10 trillion companies that are really reshaping the global economy as we speak, right? It does worry me that it's a one-way bet, though. It's A one-way bet for the entire US economy right now. There's no diversification anymore. It's all in on this. China is not doing that. China is diversifying into multiple other things. They're spending money on a lot of other stuff. It does worry me a little bit that it is a one-way bet, but You have people who are willing to gamble at the top, so they're gambling.
speaker_2: Yeah, that's maybe a great cliffhanger note to leave us on. Thanks for helping put this together. I really enjoyed getting to talk to 9 different leading experts, and I think there's, you know, obviously ample opportunity to do it more. I just made a little list of things that we didn't really cover. Mechanistic interpretability, Opening up the black box, didn't touch on that at all. Alignment and control techniques, we basically didn't touch on at all. AI for science, we kind of alluded to a few times, but there are a functionally infinite number of experts at the intersection of AI and all these different sciences that we could be talking about. Media Gen. we didn't either.
speaker_1: There are these people doing reinforcement learning with science back and forth. That could be, that's another thing that we didn't explore. Nathan, thank you. This was awesome. And yeah, let's see if we can do it again. Let's see what the responses are online. I'm very curious.
speaker_2: Let us know what feedback you have. If you made it this far, bless you. And we'll definitely be very interested in your feedback. Thanks, Prakash. This has been fun.
speaker_1: Thanks, Nathan. Cheers. Bye-bye.
speaker_2: Bye for now.
Read next
