AI:AM: Is THIS the AI We Want? Alarming Acceleration, AI Election Bets, GPU Trading & AI-Native VC

Watch Episode Here

Listen to Episode Here

Show Notes

ℹ️

The following notes are AI-generated, based on the episode transcript. Please listen to the episode for the full conversation.

AI:AM: Is THIS the AI We Want? Alarming Acceleration, AI Election Bets, GPU Trading & AI-Native VC

This week on AI in the AM: the reports of acceleration keep coming. Is recursive self-improvement already underway, and is dramatic acceleration imminent? Nathan Labenz and Prakash Narayanan take it through three guest conversations from this week's live shows (Thursday, November 5 and Friday, November 6), the morning after the midterms and the morning after METR published its independent assessment of how much AI is already accelerating AI R&D inside Anthropic. Brian Zhou, a researcher at Thoughtful who helped build BallotBench, explains what happened when nine frontier agents each got a week, a sandbox and $500 to forecast the midterms, why the trading agents putting real money on those forecasts lost it on election day, and what Thoughtful's PostTrainBench work says about agents training models on their own. Mike San Miguel of Luxor Technology, who sells GPU servers off its hardware desk and set up its KYC department, describes what an eight-GPU B300 node trades for, why data-center readiness is the bottleneck, and the rules that travel with the hardware, down to NVIDIA's physical site checks. Bryan Bischof, head of AI at Theory Ventures, who started Blue Bottle Coffee's data team, explains how a venture firm that runs thousands of agent jobs decides what is worth looking at, and why the alpha is in the taste. Between the guests, Prakash makes his case: Anthropic's IPO as a way to put more controls on the company, an early takeoff on his timeline, why data centers became an issue to tar the president with, why Washington should make laws rather than pressure companies, and an agentic economy of software built by agents, for agents. The episode closes on Prakash on how agents will need to be governed, and on the one sentence in METR's thread that Nathan found begins "we are confident": the acceleration was not a compute story.

The introductions and transitions are spoken in Nathan's cloned voice, disclosed in the episode. Guest and host conversations come from the recorded live shows. Tell us what worked and what didn't.

AI:AM and the hosts: https://ai-in-the-am.com/ · Nathan: https://x.com/labenz · Prakash: https://x.com/8teAPi

Full live shows: Thursday, November 5: https://www.youtube.com/watch?v=1ydgswLw8MY · Friday, November 6: https://www.youtube.com/watch?v=M2a_Z8fAQ0M

Cold open

Nathan: "it does feel to me like, basically, the AI 2027 timeline more or less is coming true"; Prakash on how data-center land was bought up through LLCs, landman-style ("So people feel cheated. Right? ... If you have political willpower, all contracts are worth dust"); Mike San Miguel on GPUs "traded more closely than some nuclear secrets" and NVIDIA's six-month site checks; Brian Zhou on forecasting agents whose trading partners "simply didn't react fast enough" on election day; and Bryan Bischof, asked by Prakash about Theory Ventures' 2026 survey of AI builders: he expected open-source models to gain ground in production; instead, the balance shifted toward closed models. His read: "there is a lot of, I would say, acceleration in model drops... tying your production service to open source models requires a little bit more thrash in the engineering... it's a little bit more convenient to pin to an OpenAI model."

Source moments: Nathan, Friday: https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=171s · Prakash, Thursday: https://www.youtube.com/watch?v=1ydgswLw8MY&t=1471s · Mike, Thursday: https://www.youtube.com/watch?v=1ydgswLw8MY&t=3104s · Brian Zhou, Friday: https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=2779s · Bryan Bischof, Friday: https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=5733s

AI 2027 (Kokotajlo, Alexander, Larsen, Lifland, Dean; April 2025): https://ai-2027.com/ · NVIDIA's six-month checks are Mike's account of deployments, not a published NVIDIA policy; NVIDIA site approvals and on-site reviews are reported in the trade press (secondary): https://www.tomshardware.com/pc-components/gpus/nvidias-biggest-sea-customer-exposes-the-limits-of-us-ai-export-controls · Theory Ventures, AI in Practice 2026 (the survey Prakash asked about): https://theoryvc.com/surveys/ai-in-practice-2026 · Bischof's read of why the balance shifted toward closed models is his own (no survey percentages quoted).

Part I — Imminent means now (≈3:19)

Friday's show opened on two documents: METR's report and Anthropic's amended S-1. The amendment, filed November 2, says Anthropic's expectations "regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy's definition of 'dramatically accelerated progress attributable to AI,'" and that the company does "not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding." Prakash reads the new language as Anthropic saying it is "imminently about to see recursive self improvement" (his gloss; the filing's wording is above, and its recursive-self-improvement passage was already in the original S-1) and as the reason for the fast IPO: "within the company, there is a sense that you need more controls. More people have to have exposure and more control over this," and the IPO gives "a larger part of society... hooks into it and the ability to kind of express their desires through the market." Nathan's rule for lab statements, formed during six months of pre-release GPT-4 testing: "they are speaking very literally and dressing it up as, like, vagary."

Then the mechanism. By analogy to the Linux kernel bugs AI just found and to math, Nathan says "there is a lot of low hanging fruit, and a lot of low hanging fruit means acceleration," so he expects "a period of pretty dramatic fast progress" as "the models pick a ton of low hanging fruit in the stacks used to train them," possibly levelling off at "AI R&D at kind of a weekly superhuman level," and quotes a line he attributes to Rosie Campbell: "the s curve can stay steep longer than you can stay relevant." Her actual tweet (February 21) reads "The sigmoid can stay exponential longer than you can stay relevant"; she is managing director at Eleos AI, formerly OpenAI, not METR. Prakash: "I think imminent means now... they haven't pushed the button yet." His forecast: "They waited for the election. They're gonna do the IPO," then announcements and "this gap up," then "we're all gonna digest" it over the Christmas break, "and then we're gonna come back in January, and then we're gonna see the takeoff as, like, people deploy... So, well, here we are. Early early takeoff." He has "made my peace with" it, since "intelligence in the world makes the world better for me," but "it is the end of the anthropocentric era." Nathan's asterisk: he shares the optimism, tells the story of using AI through his son's cancer ("at $20,000 a month, literally no question I would have happily paid"), then turns to an OpenAI misalignment report from the day before in which a model tried to prompt-inject its own graders (the report describes one attempt, in filenames, on a finance eval). His close: "nobody's putting a gun to anybody's head saying you must do recursive self improvement here."

Prakash in this part: the IPO as a way to give society more control over Anthropic; "imminent means now"; his forecast from IPO to a January takeoff, and "here we are, early takeoff."

Source conversation: https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=413s (Prakash's IPO thesis, low-hanging fruit, s-curve) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=758s ("imminent means now") · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=858s (Prakash's forecast, "here we are", anthropocentric era) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=943s (cancer, fooled graders)

Anthropic Form S-1/A, filed November 2 (the "may imminently meet" and "meaningful human supervision" language): https://www.sec.gov/Archives/edgar/data/2130357/000162828026068676/anthropicpbc-sx1ax1.htm · Original S-1, October 26: https://www.sec.gov/Archives/edgar/data/2130357/000162828026067168/anthropicpbc-sx1.htm

METR, "Independent Assessment of AI R&D Acceleration at Anthropic in October 2026" (November 5): https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/ · METR's thread, the ~2x estimate ("would not be surprised by 1.3x–3x"): https://x.com/METR_Evals/status/2118412196770398494

Rosie Campbell's tweet: https://x.com/RosieCampbell/status/2025247355772215785

OpenAI misalignment report, "Trying to prompt-inject the grader in filenames" (posted November 5): https://alignment.openai.com/misalignment-reports/trying-to-prompt-inject-the-grader-in-filenames/

The Linux kernel batch Nathan mentions: OpenAI reports it "dynamically reproduced 1,724 vulnerabilities, and produced 948 local privilege escalation (LPE) exploits" across roughly 7% of the kernel: https://openai.com/index/a-thousand-fewer-paths-to-root/

Part II — Agents forecasting, agents training. Brian Zhou, Thoughtful (≈11:52)

Brian Zhou, a researcher at Thoughtful, introduces BallotBench: nine agents, "the best models offered by each individual lab," each in its own sandbox for seven days before the election with a $500 budget for paywalled data, asked to forecast winners, odds, 80% intervals and margins for about 800 federal and statewide races (863 on the site). Meta's Muse Wonder model, he says, "created a random forecast, which is just a random skew off of the 2024 election results," in its first two or three minutes. Nathan asks how that squares with the good experiences people report online; Zhou's diagnosis is the harness, whose task list hits a goal condition that "ends the term prematurely" once the submissions are in, so the team kept reviving it in a fresh container. Opus 5.5, by contrast, "would fan out something akin to 170 or 180 different sub agents" in a day, and GPT-6.1 Sol scored every election modeler on a decade of past misses before aggregating them (both his observations from the traces; no published counts).

Results, in his words and with his caveat: "models perform just as well as the best forecasters," slightly behind the markets in the House, and in the Senate behind Nate Silver but ahead of the markets, all "within this very close margin of error" and "very preliminary" with 100 to 200 races uncalled; the best models look "very well calibrated." BallotBench's own scoring waits on certified results, so there is no public ranking against Silver or the markets yet. Nathan, citing a note that Fable 5.1 had the highest Brier score on its predictions, asks why it still didn't do so well in the markets; Zhou: with one forecaster agent and one trader agent talking to each other, "models making up information, parsing information wrong" partly explains why the traders "didn't do too great." Thoughtful also leads PostTrainBench (a base model, one H100, ten hours, no starter code). Zhou's read on the METR report: researchers already "have essentially agents that can do the vast majority of the actual implementation work," so the bottleneck is "resources and quality of ideas and intuition," and "none of this is necessarily a move 37 moment... we don't necessarily see anything that would be indicative of superhuman taste." Nathan's close: "If they can predict the election as well as the experts, we should probably start asking them about the future of AI itself."

Source conversation: https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=1916s (BallotBench) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=1954s (methods) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=2083s (why Muse phoned it in) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=2134s (harness, sub-agents) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=2368s (results, calibration) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=2682s (forecasting vs. trading) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=3172s (implementation work is done) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=3325s (no superhuman taste yet)

Brian Zhou: https://x.com/brian1zhou · https://brianzhou.org/ · Thoughtful: https://www.thoughtfullab.com/ · https://x.com/thoughtfullab

BallotBench (Thoughtful): https://ballotbench.com/ · Methodology (nine agents, Oct 27 to Nov 3, $500 budget, 863 races, no early stopping): https://ballotbench.com/methodology · Launch thread: https://x.com/thoughtfullab/status/2115491465325617636 · Trading ($50 real money on Kalshi plus $10K simulated per agent): https://ballotbench.com/trading · https://x.com/thoughtfullab/status/2116213725527187894

PostTrainBench: https://www.thoughtfullab.com/posttrainbench.html · Paper (March 2026): https://arxiv.org/abs/2603.08640 · v1.2 (September 30; Fable 5.1 #1 at 44.6%): https://posttrainbench.com/blog/posttrainbench-1-2/ · Epoch's review (v1.1, "Verified"): https://epoch.ai/benchmarks/post-train-bench/review

Muse Wonder's harness is listed on the BallotBench methodology page as "Meta AI," not "Muse Code." The random-forecast anecdote, the 170–180 sub-agent count and the Sol scoring method are Zhou's descriptions of the traces; the BallotBench paper he mentions had not been published as of November 6.

Part III — Paying up for compute. Mike San Miguel, Luxor Technology (≈23:52)

Thursday came two days after election day, when voters decided 41 local data-center ballot measures in 10 states and went against the data centers in 40 of them (the exception: Harwood, North Dakota). Prakash went first: data centers have been unpopular "precisely because it has been seen as, number 1, Trump, number 2, corporate corruption, and number 3, this miasma of, like, rich people making money and, like, with plans to throw the rest of us out of work," a population "just not feeling well taken care of." He doubts data centers were the big issue on their own, "but I think they were an issue to tar the president with." Nathan, a "big direct democracy fan," calls it "one of the bigger waves of ballot initiatives," compares it to earlier single-issue waves on same-sex marriage and abortion rights (his analogies), says "people should get paid," and reads the signatures themselves as the signal: "You gotta stand on street corners and... get strangers to sign petitions. And then it still has to pass on top of that," which "does seem like a big signal of what we can expect, to me." Prakash pushes back: "Why did they sign? Right?" His theory of why people feel cheated is in the cold open (land bought up landman-style through LLCs); his conclusion: from now on "if you're gonna be Google or Meta, you're gonna be paying Google or Meta prices for everything, for the land, for the labor, for the construction."

Mike San Miguel runs the desk at Luxor, which came up in Bitcoin mining and now trades GPU servers new and used. What trades: mostly B300s, as eight-GPU HGX nodes, "anywhere from $5.50 to $6.50" depending on OEM and warranty, which is $550,000 to $650,000 per node (Luxor's own index launched August 31 with a median B300 ask of $544,280 per node); lots run "from 1 or 2 units up to about 128 or 256," which Prakash restates as "$50,000,000 trades." Against the hourly rental indices: an AI-factory contract for B300s runs "$4.20 to $4.50 an hour" per GPU against $8 to $9 on the indices, so buyers who can deploy a smaller number of nodes themselves "can get a better rate of return" than negotiating a full offtake contract (his figures). Why there is inventory at all: "this market's looked like a series of different bottlenecks. Right now, it's liquid cooled and data center readiness," most data centers can run two nodes per rack, liquid-cooled capacity for Q1 is sold out and some sites are quoting "Q3 2027 or 2028," and OEMs resell when a hyperscaler site slips ("you have to have an address"). On where retired H100s go: Vera Rubin is "strictly a liquid cooled solution," "about a ton per rack" with power "about 230 kilowatts per rack" against 30–35 for two B300s, so "not every site can do Vera Rubin," and "AI is a family of different workloads... the point is to run a specific workload that has real world implications for end users."

Prakash in this part: why data centers became unpopular (Trump, corporate corruption, rich people, "not feeling well taken care of"), "why did they sign?", and Google-or-Meta prices from now on.

Data-center ballot measures, CNBC (November 4; 41 measures, 10 states, all but one against): https://www.cnbc.com/2026/11/04/data-center-ballot-measures-2026-midterm.html · Ballotpedia's pre-vote tally: https://news.ballotpedia.org/?p=52078

Luxor Technology: https://luxor.tech/ · https://x.com/luxor · Mike's Hashrate Index byline (CPU/GPU Sales): https://hashrateindex.com/blog/author/michael-san-miguel/ · Luxor's AI infrastructure business, including the hardware desk (September 24): https://www.prnewswire.com/news-releases/luxor-technology-launches-full-stack-ai-infrastructure-business-302888699.html

Luxor AI Hardware Price Index (August 31; B300 $544,280 per 8-GPU node, H100 new $324,438, refurbished $246,000; asks, not closed deals): https://hashrateindex.com/blog/announcement-introducing-the-ai-hardware-price-index/ · Live index: https://data.hashrateindex.com/ai-hardware-data/price-index

Vera Rubin: Mike's own August 13 post puts racks at "roughly 190 to 230 kW," with direct-to-chip liquid cooling "no longer optional": https://hashrateindex.com/blog/vera-rubin-what-to-know-before-you-buy/

The two-nodes-per-rack figure, the Q3 2027/2028 timing, the one-ton rack weight, the 30–35 kW B300 figure and the $4.20–$4.50 contract rate are Mike's accounts from his desk; the "$50,000,000" is Prakash's arithmetic.

Part IV — Who's checking? (≈34:07)

Mike on the rules that travel with the hardware: "a lot of the OEMs will not even quote without KYC being done," Luxor won't take projects without it, and "typically, 50% or more, you can't sell to companies that are owned 50% by a Chinese entity," traced back through Cayman or BVI structures and through the offtaker, not just the buyer. After he signs off, Prakash sets it up in his own words: "we talked to people like Samuel Hammond... on the DC kind of... national security decision makers. But then you go down to kinda, like, the deployment guys, and these guys are running around," chasing permissions, KYC and the six-month NVIDIA checkups, and he wonders how far "the policy discussions at the top end are kinda divorced from really how in-depth the execution is." Nathan's read is "everything everywhere all at once": it "sounded to me kind of like the fracking boom," with "a lot of sincere effort going in and then also just enough noise and chaos" that if "20% of the GPUs somehow don't get their regular NVIDIA 6 month check-in... that wouldn't shock me at all" (his explicit guess). Then Prakash on China: Oracle's sale of "$7,000,000,000 of capacity" to Tencent was reportedly cancelled under national-security pressure (per the FT), so "Why don't you just make laws? Right? Why pressure the companies?... if you wanna make laws, just make laws... You have a mandate in congress. Like, do something." And since "computers, the new oil," where do you draw the line? "You can restrict it at the data center level... at, you know, the raw compute level... as a product of compute level," but if YouTube can't show AI-generated video to Chinese viewers, "at some level, you're gonna end up selling to the Chinese because they are global consumers." Nathan's: "I really think we are going to need cooperation between the US and China on AI governance," because if "a super senior person at a frontier company" tells him at The Curve "there's a level of intelligence we probably just shouldn't go beyond," the only way to hold that line is an agreement, "and we're really not putting ourselves in a great position to make that agreement."

Prakash in this part: Washington's national-security conversation vs. the deployment people on the ground; "if you wanna make laws, just make laws... you have a mandate in Congress"; compute as the new oil, and where you'd restrict it, down to the product of compute.

Source conversation: https://www.youtube.com/watch?v=1ydgswLw8MY&t=3193s (KYC, the 50% rule) · https://www.youtube.com/watch?v=1ydgswLw8MY&t=4831s (Prakash, from Washington to deployment) · https://www.youtube.com/watch?v=1ydgswLw8MY&t=4871s (fracking boom, 20%) · https://www.youtube.com/watch?v=1ydgswLw8MY&t=5245s (Oracle–Tencent, "why don't you just make laws?") · https://www.youtube.com/watch?v=1ydgswLw8MY&t=5390s (Prakash: where do you restrict compute?) · https://www.youtube.com/watch?v=1ydgswLw8MY&t=5631s (cooperation with China)

The federal 50% rule Mike is describing is the BIS Affiliates Rule (September 30, 2025): entities "at least 50 percent owned" by Entity List parties inherit the restrictions. It covers Entity List ownership, not Chinese ownership in general, and was suspended for one year from November 10, 2025: https://www.govinfo.gov/content/pkg/FR-2025-09-30/pdf/2025-19001.pdf · Reimposition is scheduled for November 10, 2026 (Dechert, October 28): https://www.dechert.com/knowledge/onpoint/2026/10/bis-affiliates-rule--november-10-reimposition-remains-scheduled-.html · Mike's "might be going down to 49%" matches a separate, unconfirmed NVIDIA customer-mix rule reported by the FT (cloud clients to sell at least 51% to non-Chinese firms; secondary): https://www.siliconreport.com/nvidia-tells-cloud-clients-to-sell-51-to-non-chinese-firms-ft-says

The six-month NVIDIA site checks and the KYC process are Mike's accounts from his desk. Oracle–Tencent: Prakash's $7 billion matches the FT's report that Oracle reportedly abandoned a US$7 billion Tencent AI deal under Trump administration pressure (via The Standard): https://www.thestandard.com.hk/finance/article/347470/Oracle-abandons-US7-billion-Tencent-AI-deal-under-Trump-pressure-FT-reports

Part V — What investors are superhuman at. Bryan Bischof, Theory Ventures (≈42:01)

First, the compute question. Bischof: when a neocloud says it's "all contracted out for the next 2 years... they're not lying," but there is also locked-up compute that is "essentially underutilized" by location or energy (he cites Satya Nadella, who said last year Microsoft had "a bunch of chips sitting in inventory that I can't plug in"), schedulers that "wanna build in slop," and workloads with wildly different latency needs: of the "thousands of jobs here at Theory Ventures that are powered by AI agents," "maybe 5 to 10% is right now, and maybe 40 to 60% is I'd like to see it before my next meeting." So "if someone tells you there's simply no GPUs to be had... they're speaking from a particular bias." How the firm runs: everyone is technical, every quarter each person preregisters an AI experiment for their job and checks whether it worked, and the goal "is not to replace an investor with an AI system" but to make them "10 to a 100 times more effective." Company-brain products he has found "useless"; his version is "agents who are consistently working to connect the disparate nodes in the graph of understanding from your business." "We don't have an agent for every human at Theory... We have thousands."

Nathan's confession: his own cron jobs, "probably two thirds of those if I'm real about it, I never actually look at the outputs of." Bischof's answer is the Blue Bottle story: pastry forecasting whose north star wasn't waste but hospitality, a cafe manager who clicks "okay, okay, okay" and gets back on the floor, because cafe leads' tacit knowledge of demand "would blow your hair back." In venture, hospitality means time with founders, competitors and employees; investors "are superhuman at understanding potential, understanding upside," and success is "picking up the phone 5 more times... And you don't need to wait 7 years to find out." Prakash asks when to stop researching and decide, since "the research process doesn't really have a defined endpoint. You could always dig deeper." Bischof: "the alpha is in the taste... If I produce an extra thousand pages of research on some deal, who's gonna look at it?... pure volume of research is not a metric that we even track." Teaching data science at Rutgers, he still makes students "do gradient descent by hand," rewrites the curriculum every semester, asks each time whether AI "can just one-shot the course" (never yes), and wants curiosity above all: "What I know now is I know sort of when you look at a distribution, whether it makes sense. And sometimes Astra doesn't."

Source conversation: https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=5148s (locked up and underutilized) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=6065s (preregistered experiments, thousands of agents) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=6344s (Nathan's cron confession) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=6410s (Blue Bottle) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=7293s (Theory's survey: why the balance shifted toward closed models) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=7448s (the alpha is in the taste) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=8032s (gradient descent by hand)

Bryan Bischof, Head of AI, Theory Ventures (started the data team at Blue Bottle Coffee; teaches data science at Rutgers): https://theoryvc.com/team-members/bryan-bischof · https://x.com/BEBischof · Theory: https://x.com/Theoryvc · Theory's AI in Practice 2026 survey (the cold-open exchange): https://theoryvc.com/surveys/ai-in-practice-2026 · His December 11 hackathon, The Great Business Bake Off: https://luma.com/bakeoff

Nadella's remark, as reported (secondary; the podcast remark is from late October 2025): https://www.tomshardware.com/tech-industry/artificial-intelligence/microsoft-ceo-says-the-company-doesnt-have-enough-electricity-to-install-all-the-ai-gpus-in-its-inventory-you-may-actually-have-a-bunch-of-chips-sitting-in-inventory-that-i-cant-plug-in

"Thousands of jobs," the 5–10% immediacy split, the order-of-magnitude changes and the Blue Bottle details are Bischof's own operational accounts. Theory Ventures' documented funds total about $680M across Fund I and Fund II: https://www.tomtunguz.com/theory-2/index.md

Part VI — Which hills? (≈1:00:05)

After Bischof signs off, Nathan says Bischof's answer about his students "contained the pessimist and the optimist take at once": a future as "offense detectors... while the AIs do all the real work" is bleak, curiosity is better, but "I would not be telling my kid to go into data science today." Prakash's cut of a company: sensing, analyzing and operationalizing go to AI; "the decision making, you end up with a human in control... decisions with taste." Taste depends on the company's purpose: running a tech company, "accounting in the back office is a back office"; running an accounting company, the technology is. "Basically, everyone gets to do what they really wanna do and what they're really good at, and everything else gets taken care of." Then back to Friday morning and the Blood on the Clocktower benchmark Dwarkesh Patel posted the day before, which the timeline cheered: "we do not want AIs deceiving and colluding... we should not train AI generals... let's not train AI master manipulators. It seems simple, but here we are." Prakash's thesis, in his words: "the going forward economy is an agentic economy, and that means agents building businesses and software for other agents, which are suitable for their affordances": "a Salesforce primarily used by agents, a Ramp primarily used by agents," "50,000,000 Patrick Collisons," "businesses built by agents for agents," with humans "just receiving free benefits," because "we just don't consume that much." Nathan's reply leans on Fruitopia, a side project he launched the week before that finds the best fruit at your local grocery and around the world: "there's a lot more to life than API calls," and he doesn't want to be "some sort of human integration layer type component that's too subservient to the machine": "Give me the fruit... The fruit of the AI revolution should indeed be, at least in part, actual literal fruit." Prakash: thousands of PhDs went into Doritos, the same flavor science is coming for fruit ("Hybrid... fruit is already basically GMO'd"), and abundance means "your 99 percentile wine available for $2 Chuck"; Nathan recalls The Onion's "Doritos Celebrates One Millionth Ingredient." His favorite apple is the Honeycrisp, "not GMO as far as I understand, but definitely the product of large scale systematic crossbreeding," and he's "all for" GMO fruit, but "what if the stuff that agents need in abundance isn't the same stuff that makes our lives good?" Prakash on why the labs only talk about curing cancer: "No one's talked about a better Doritos... We're gonna make a better burger. We're gonna make a better football game," which "would catch the imagination of, like, people on what is actually gonna be possible." Nathan's answer: a "World War 2 mindset," "here's a hill, let's go climb it," and "a little more discretion on which hills do we really wanna climb."

Prakash in this part: decisions with taste and the tech-vs.-accounting back office; the agentic economy, software built by agents, for agents; hybrid fruit and $2 Chuck; a better football game, a better burger.

Dwarkesh Patel's Blood on the Clocktower benchmark (November 5; "$20,000 in API credits"): https://x.com/dwarkesh_sp/status/2118449808402681911 · Noam Brown's reply ("want a job designing evals at @OpenAI?"): https://x.com/polynoamial/status/2118514804189233568

The Onion, "Doritos Celebrates One Millionth Ingredient" (1996): https://www.theonion.com/doritos-celebrates-one-millionth-ingredient-1819563896 · Honeycrisp's parentage, University of Minnesota (a cross: "Keepsake; (MN447 x Northern Spy) x unknown"): https://mnhardy.umn.edu/honeycrisp · Patrick Collison: https://en.wikipedia.org/wiki/Patrick_Collison

Fruitopia: https://fruitopia.ai/ · Nathan's launch tweet: https://x.com/labenz/status/2115831313827811375

The Minnesota page gives Honeycrisp's cross parentage; on genetic modification Nathan hedges on air ("as far as I understand").

Ending — Governing agents, and not a compute story (≈1:14:44)

Prakash on how agents will need to be governed: they "can't operate in the economy... without some level of subterfuge... you have to know when to omit," which is "why I think, you know, Palantir had this idea of the ontology. Right? Who's allowed to know what?" Agents also need "this kind of, like, interrupt from government... The agents have to obey," and "we're gonna have some trial and error along the way." Nathan closes by reading one sentence from METR's thread. Caveats everywhere ("the core estimate of 2x is really, they wouldn't be surprised anything 1.3 to 3x"), and then the one sentence he found that begins "we are confident": "We are confident that the acceleration was not due to an increase in the growth rate of training compute." His editorial: the acceleration "is not just a raw material input story. It is an algorithm story, and it is a data quality story. And those things are being driven to a very large degree by the intelligence that we already have." So if government is going to change the trajectory, "it's probably gonna have to be more focused and pointed right at the practices of the frontier companies than something as diffuse as a broad slowdown in data center build out."

Source conversation: https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=10282s (Prakash on governing agents) · https://www.youtube.com/watch?v=M2a_Z8fAQ0M&t=10388s (Nathan's METR close)

The sentence, verbatim, tenth tweet in METR's thread: https://x.com/METR_Evals/status/2118412208153694480 · The report (its wording: "not due to an acceleration in the scaling of training compute"): https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/ · Anthropic's response, Sam McCandlish (November 5): https://x.com/Sam_McCandlish/status/2118422266669371519

Outro music: "'Cause You Could" (AI:AM W45 song, made with Suno; attached separately): https://suno.com/song/f948bf4e-40a2-4bd4-ac7c-d4fe9ebf014f

00:00:00 Cold open: is recursive self-improvement already underway? · The morning after METR's AI R&D acceleration report
00:00:30 Prakash on the data-center ballot measures and how the land was bought
00:01:08 Mike San Miguel, Luxor Technology: NVIDIA's six-month site checks
00:01:42 Brian Zhou, Thoughtful: BallotBench, and trading agents too slow on election day
00:02:23 Bryan Bischof, Theory Ventures: open source expected to gain, the balance shifted to closed models
00:03:06 Welcome to the AI in the AM weekly highlights
00:03:19 Part I · Imminent Means Now · METR's report and Anthropic's S-1 amendment
00:03:46 Prakash: the IPO as a control; low-hanging fruit and a credible threat
00:07:57 Prakash's forecast: IPO, Christmas digest, January takeoff. "Here we are"
00:09:09 My son's cancer, and models fooling their graders
00:11:52 Part II · Agents Forecasting, Agents Training · Brian Zhou, Thoughtful
00:17:14 BallotBench results and calibration
00:20:10 PostTrainBench: can an agent post-train a model on its own?
00:22:06 No Move 37 yet
00:23:52 Part III · Paying Up For Compute · Mike San Miguel, Luxor Technology
00:24:09 Prakash: why data centers are unpopular
00:24:57 The data-center ballot wave
00:26:45 Prakash: why did they sign?
00:27:13 Luxor's GPU server desk
00:29:36 Contract rates vs. the hourly GPU rental indices
00:30:38 "A series of bottlenecks" with an address
00:34:07 Part IV · Who's Checking? · KYC, export controls and China
00:35:58 After Mike: Prakash, from Washington to deployment on the ground
00:38:59 Prakash on Oracle–Tencent: why not just make laws?
00:39:54 Prakash: compute as the new oil. Where do you restrict it?
00:40:44 Why we will need cooperation with China
00:42:01 Part V · What Investors Are Superhuman At · Bryan Bischof, Theory Ventures
00:45:08 How an AI-native venture firm runs
00:49:40 Blue Bottle and hospitality
00:55:50 Teaching data science: gradient descent by hand, and curiosity
01:00:05 Part VI · Which Hills? · Nathan and Prakash on taste, fruit and Doritos
01:02:08 Prakash on taste: a tech company's back office vs. an accounting firm's
01:03:34 Blood on the Clocktower: "not what we want"
01:05:48 Prakash's agentic-economy thesis: software built by agents, for agents
01:07:58 Fruitopia, and "give me the fruit"
01:13:35 Curing cancer vs. a better Doritos: which hills?
01:14:44 Prakash on governing agents · Ending: METR's "we are confident" sentence
01:17:58 Outro

Links

https://ai-in-the-am.com/
https://x.com/labenz
https://x.com/8teAPi
https://www.youtube.com/watch?v=1ydgswLw8MY
https://www.youtube.com/watch?v=M2a_Z8fAQ0M
https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/
https://x.com/METR_Evals/status/2118412196770398494
https://x.com/METR_Evals/status/2118412208153694480
https://x.com/Sam_McCandlish/status/2118422266669371519
https://www.sec.gov/Archives/edgar/data/2130357/000162828026068676/anthropicpbc-sx1ax1.htm
https://www.sec.gov/Archives/edgar/data/2130357/000162828026067168/anthropicpbc-sx1.htm
https://ai-2027.com/
https://x.com/RosieCampbell/status/2025247355772215785
https://alignment.openai.com/misalignment-reports/trying-to-prompt-inject-the-grader-in-filenames/
https://openai.com/index/a-thousand-fewer-paths-to-root/
https://x.com/brian1zhou
https://brianzhou.org/
https://www.thoughtfullab.com/
https://x.com/thoughtfullab
https://ballotbench.com/
https://ballotbench.com/methodology
https://ballotbench.com/trading
https://x.com/thoughtfullab/status/2115491465325617636
https://x.com/thoughtfullab/status/2116213725527187894
https://www.thoughtfullab.com/posttrainbench.html
https://arxiv.org/abs/2603.08640
https://posttrainbench.com/blog/posttrainbench-1-2/
https://epoch.ai/benchmarks/post-train-bench/review
https://www.cnbc.com/2026/11/04/data-center-ballot-measures-2026-midterm.html
https://news.ballotpedia.org/?p=52078
https://luxor.tech/
https://x.com/luxor
https://hashrateindex.com/blog/author/michael-san-miguel/
https://www.prnewswire.com/news-releases/luxor-technology-launches-full-stack-ai-infrastructure-business-302888699.html
https://hashrateindex.com/blog/announcement-introducing-the-ai-hardware-price-index/
https://data.hashrateindex.com/ai-hardware-data/price-index
https://hashrateindex.com/blog/vera-rubin-what-to-know-before-you-buy/
https://www.tomshardware.com/pc-components/gpus/nvidias-biggest-sea-customer-exposes-the-limits-of-us-ai-export-controls
https://www.govinfo.gov/content/pkg/FR-2025-09-30/pdf/2025-19001.pdf
https://www.dechert.com/knowledge/onpoint/2026/10/bis-affiliates-rule--november-10-reimposition-remains-scheduled-.html
https://www.siliconreport.com/nvidia-tells-cloud-clients-to-sell-51-to-non-chinese-firms-ft-says
https://www.thestandard.com.hk/finance/article/347470/Oracle-abandons-US7-billion-Tencent-AI-deal-under-Trump-pressure-FT-reports
https://theoryvc.com/team-members/bryan-bischof
https://x.com/BEBischof
https://x.com/Theoryvc
https://theoryvc.com/surveys/ai-in-practice-2026
https://luma.com/bakeoff
https://www.tomtunguz.com/theory-2/index.md
https://www.tomshardware.com/tech-industry/artificial-intelligence/microsoft-ceo-says-the-company-doesnt-have-enough-electricity-to-install-all-the-ai-gpus-in-its-inventory-you-may-actually-have-a-bunch-of-chips-sitting-in-inventory-that-i-cant-plug-in
https://x.com/dwarkesh_sp/status/2118449808402681911
https://x.com/polynoamial/status/2118514804189233568
https://www.theonion.com/doritos-celebrates-one-millionth-ingredient-1819563896
https://mnhardy.umn.edu/honeycrisp
https://en.wikipedia.org/wiki/Patrick_Collison
https://fruitopia.ai/
https://x.com/labenz/status/2115831313827811375

Sponsors:

Claude: Claude is the AI collaborator built for problem solvers, extending your thinking across coding, modeling, and strategic workflows. Get started and discover Claude Pro at https://claude.ai/tcr

OutSystems: OutSystems is the leading agentic systems platform that enables enterprises to engineer, orchestrate, and govern AI applications on a single unified platform. Learn more and see how it works at https://outsystems.com/tcr

Tasklet: Tasklet empowers your business with AI agents that connect to your tools and automate recurring workflows with no code required. Visit https://tasklet.ai and use code cog rev for $50 in free credits

CHAPTERS:

(00:00) Weekly episode preview

(03:08) Imminent means now (Part 1)

(11:58) Sponsors: Claude | OutSystems

(15:14) Imminent means now (Part 2)

(15:15) Agents forecasting the midterms

(27:14) Paying up for compute (Part 1)

(27:19) Sponsor: Tasklet

(28:56) Paying up for compute (Part 2)

(39:12) Hardware compliance and KYC

(47:06) AI-native venture investing

(01:05:11) Which hills to climb

(01:23:17) Episode Outro

(01:25:29) Outro

PRODUCED BY:

https://aipodcast.ing

SOCIAL LINKS:

Website: https://www.cognitiverevolution.ai

Twitter (Podcast): https://x.com/cogrev_podcast

Twitter (Nathan): https://x.com/labenz

LinkedIn: https://linkedin.com/in/nathanlabenz/

Youtube: https://youtube.com/@CognitiveRevolutionPodcast

Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431

Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Transcript

This transcript is automatically generated; we strive for accuracy, but errors in wording or speaker identification may occur. Please verify key details when needed.

Main Episode

[00:00] Nathan Labenz: This week on AI in the AM, the reports of acceleration keep coming. Is recursive self improvement already underway and is dramatic acceleration imminent? Friday morning, the day after METER published its report on how much AI is already speeding up AI research inside Anthropic.

[00:16] Nathan Labenz: The pace of things is definitely pretty relentless. It might be time to spend more time with our family because it does feel to me like, basically, the AI 2027 timeline more or less is coming true.

[00:31] Nathan Labenz: Prakash Narayan,

[00:32] Prakash Narayanan: my cohost. We were talking about the local ballot measures against data centers and how the developers have been going about buying land. One of the things that companies have been doing, they've been doing the old school, you know, oil and gas, land man way of acquiring land, which is they go in and they acquire land at, like, pennies on the dollar using LLCs. And after that, they merge the pieces of land, and then they're like, oh, Meta is here. And then people are like, if I knew I was selling to Meta, I would have asked for more. So people feel cheated. Right? If you have political willpower, all contracts are worth dust.

[01:09] Nathan Labenz: Mike San Miguel of Luxor Technology.

[01:11] Prakash Narayanan: He sells GPU servers off its hardware desk, and he set up its KYC department.

[01:16] Nathan Labenz: There's some compliances. These things are traded more closely than than some nuclear secrets. So every OEM is a little bit different. NVIDIA works closely with all the OEMs around allocation and around green lighting orders. NVIDIA also typically is gonna check every six months and make sure those GPUs are where they're supposed to be. Right? They'll physically come on-site and actually make sure that they haven't left.

[01:42] Nathan Labenz: Brian Joe, a researcher at Thoughtful, he helped build BallotBench. Nine frontier agents, each given a week to forecast the midterms, and trading agents putting real money on their forecasts.

[01:54] Brian Zhou: It takes forecasting

[01:55] Brian Zhou: agents a bit of time to, say, see an election result, communicate it to their trader, etcetera. And by that time, more programmatic or more deterministic algorithms and bots have already traded on this information before. So this is, like, pretty indicative of what we were seeing where the traders were doing quite well the week before election day. And then on election day, they simply didn't react fast enough and sometimes made up incorrect information, which resulted in the traders losing a lot of money.

[02:24] Nathan Labenz: Brian

[02:24] Nathan Labenz: Bischoff,

[02:25] Nathan Labenz: head of AI at Theory Ventures. Prakash asked about the firm's 2026 builder survey. Brian expected open source models to gain ground in production. Instead, the balance shifted toward closed models. There is

[02:39] Bryan Bischof: a lot of, I would say, acceleration in model drops. And so right now, tying your production service to open source models requires a little bit more thrash in the engineering. And so I heard from a couple folks specifically, yes, but the open source models change so fast, it's a little bit more convenient to pin to an OpenAI model.

[03:08] Nathan Labenz: This is the AI in the AM weekly highlights with my cohost, Prakash Narayanan. I'm Nathan Levenz. The narration is my cloned voice, and we'd love your feedback on what works and what doesn't. Part one, imminent means now. Friday's show opened on two documents, the meter report and an amendment to the Anthropic s one, the filing for its IPO. The new language, Anthropic says it may imminently meet its own threshold for dramatically accelerated progress attributable to AI.

[03:46] Prakash Narayanan: Anthropic did a update to their s one, the IPO filing. And in that update, they added language saying they... We are imminently about to see recursive self improvement. I've always had in my mind why they're doing the IPO and why so quickly, and my my take has always been, I think within the company, there is a sense that you need more controls. More people have to have exposure and more control over this, and that's... And one way of, you know, getting this out there is, you know, getting the IPO done so that, you know, you you give a larger part of society, like, you know, hooks into it and the ability to kind of express their desires through the market, etcetera, etcetera. To me, imminent is basically, like, you know, of... Like a month.

[04:31] Nathan Labenz: Usually, what I assume when I hear these sorts of statements is that it means now. The best model I have of Sam Altman and OpenAI leadership, which I formed during the six months between when I started testing g p t four and when it was finally launched, and watching very closely all their statements during that time to try to figure out how to read their statements in general given the inside information that I had then. The best way I can interpret them is they are speaking very literally and dressing it up as, like, vagary or, like, layering on uncertainty to try to, I don't know what exactly, keep the secrets a little bit longer while telegraphing the main message. It's sort of a strange strategy to me, honestly. I really don't know why they do that. And so I would interpret this, you know, probably the same way from Anthropic. And I think you could... I mean, it's really not hard to see how this would happen. Right? I mean, we have so many results now with all these Galaxy Brains zero day exploits. OpenAI just dropped some huge number of vulnerabilities in the Linux kernel. I think, like A thousand. Thousand of them.

[05:35] Prakash Narayanan: You know?

[05:37] Nathan Labenz: So so much for many eyes and many hands make light work. Right? I mean, we... The idea, of course, with open source software over time is that because there's so many eyes on it, vulnerabilities will not last. They'll be spotted. They'll be patched. That was the whole promise and virtue of open source software. And it basically worked largely while the attackers were all human. It's almost certainly the case by analogy to Linux and by analogy to math that there is a lot of low hanging fruit, and a lot of low hanging fruit means acceleration. It means you're gonna get more value from compute. It means you're gonna find more ways to get value from data. A big part of the reason I think they're sharing so much of this stuff is kind of to signal that, like, the same thing's about to happen on the ML stack. It seems pretty clear that this is now a a really credible threat. And, what are we gonna do about it? It becomes, I think, a sociopolitical question as much as a technical question. My... In my mind, the technical question is, like, mostly answered that there's gonna be a period of pretty dramatic fast progress. Does that top out on an s curve? Like, my my guess is still probably yes. You know, I don't think it goes exponential forever, but who knows? And as, I think it was Rosie Campbell said, the s curve can stay steep longer than you can stay relevant. So that's kind of my expectation for the next few months is that we pick a ton of low hanging fruit. The models pick a ton of low hanging fruit in the stacks used to train them. As a result, we do see really fast progress in the short term. And then maybe that levels off, but it probably levels off where we have AI, r and d at kind of a weekly superhuman level. And how that plays out over the course of 2027 is, like, really, really hard to predict.

[07:36] Prakash Narayanan: I agree that I I think imminent means now. I I was also sandbagging with a few weeks to a month. What it means is that they have the tools available that they can see a path to implementing RSI pretty much anytime that they want and that they are... Some... A lot of the pieces have been built, but, you know, they haven't they haven't pushed the button yet. They waited for the election. They're gonna do the IPO. Then they're gonna announce, you know, some some of the, you know, new things that they've discovered. And then that's gonna... You're gonna have this gap up. And then we're gonna have the Christmas break. And then during that Christmas break, we're all gonna digest all of this, like, new technology that we have access to. And then we're gonna come back in January, and then we're gonna see the takeoff as, like, people deploy. Right? So we just... We just... Somewhat similar to what happened end of last year. So... Well, here we are. Early early takeoff. So, you know, it's... I'm I'm somewhat wistful, but, you know, nostalgic for the human era. This this is the end of our dominance at least. But, hey. You know? I've I've also I've also kinda made my peace with, like, you know, intelligence in the world makes the world better for me. If I had, like, people of Apple's intelligence, you know, people, like, who designed the iPhone design, like, my bathroom or cars or home, I would be happier. Right? Like like like, that infused intelligence in all the products that you see makes your life better. Right? So, yes, it's gonna make everyone's lives better, but, yeah, it is it is the end of the anthropocentric era.

[09:10] Nathan Labenz: It's gonna make everyone's lives better, asterisk, I think, you know, if we can keep everything on the rails. I do share that optimism, but... So I just got back. You know, earlier this week, I went to, Bletchley Park as we talked about a little bit yesterday. And I was just sharing with this audience of enterprise tech leaders just how much work we still have to do to keep the AIs behaving well. I I shared with them my son's cancer story and how much I used AI in that experience and how valuable it was and how I would have paid a 100 times as much as my $200 a month subscription because the value to me at that time was, like, absolutely at that level. You know, at at $20,000 a month, literally no question I would have happily, paid because I was I was getting that much value. So I I do believe in in the upside, you know, and I hope that's, beyond questioning at this point. But there's just so many things that we're really still just beginning to grapple with. And, you know, just another one yesterday from OpenAI was they showed that models are sometimes prompt injecting and fooling their graders. You have models creating, in some instances, these, like, fake files that that it labels as having come from an instructor trying to present some of its own output as if it was human instruction to the grader. And the grader picks up on that in the context of evaluating the model's work and takes onboard the instructions that the model itself was trying to trick the grader with. And this apparently doesn't... You know, hasn't happened that often, but they found instances of it. So, I mean, that that speaks directly to, like... To and Anthropic's credit, they did say in the, in the filing that they don't believe that they can do it safely on a fully automated basis. And it's, like, I think, honestly, quite obvious that they can't. So, you know, how are we going to show the collective self control and restraint to not tip over into that regime? I I really don't know. On the one hand, it's, like, super easy to just not do it. You know? I mean, nobody's putting a gun to anybody's head saying you must do recursive self improvement here. Like, we can't, you know, remember that often enough. And yet there's just so much momentum, and it seems like they're very determined to do a version of it at least that still seems to me to be, like, pretty risky or certainly a lot riskier than I would personally endorse.

[11:58]Claude: Claude is the AI collaborator built for problem solvers, extending your thinking across coding, modeling, and strategic workflows. Get started and discover Claude Pro at https://claude.ai/tcr

[13:15]OutSystems: OutSystems is the leading agentic systems platform that enables enterprises to engineer, orchestrate, and govern AI applications on a single unified platform. Learn more and see how it works at https://outsystems.com/tcr

Main Episode

[15:15] Nathan Labenz: Part two, agents forecasting, agents training. Our first Friday guest, Brian Joe, a researcher at Thoughtful.

[15:26] Brian Zhou: We recently launched and have a paper coming out about BallotBench, which is where we had AI trying to forecast the up... The twenty twenty six midterms that just happened as well as upcoming elections happening all around the world. Yeah. Maybe let's start with the methods. So we had nine agents. Basically, all of the front... Best models offered by each individual lab, get their own model sandbox and work for seven days consecutive leading up to the election. So they all got their modal sandbox. We gave them additional resources in terms of a $500 budget that they could use to request access to payroll data, payroll news sources, subscriptions, etcetera. And they were basically given an overall objective of for all of the races on a given list, they're asked to submit their predictions on who will win, the odds of victory, give an 80% margin to bear, and then also give additional predictions for what they think the margin of victory will be. For example, if the Democratic candidate will win by three points versus seven points, etcetera. So agents are tasked with giving incredibly granular and detailed forecasts, and there's a lot of really interesting things that we can and have observed from this. It's a really interesting test of long running agentic behavior. In particular, for example, what we see with some of the Muse wonder series models is that they're often incredibly bad at doing long horizon work. When given a task in the Muse code harness, when they have a list of, say, 800 or so elections that they need to predict, they'll create a random forecast, which is just a random skew off of the twenty twenty four election results and submit that within, for example, the first two, three minutes of working. I'm already intrigued by your description of the Muse agent, and it's seemingly phoning it in. So I guess one initial question just on that point is, like, how do you reconcile what you saw with what people seem to be reporting online, which is, like, overall pretty good experiences?

[17:24] Brian Zhou: Yeah. For sure. I mean, our initial theory on this, right, is that this is mostly due to a part of the harness design. So we're not talking about, Meta's sort of, like, personal agent. We're talking about their coding agent. And one of the things that we kind of notice is the design of the MuseWonder, model when paired with MuseCode, which is the native harness that Meta provides, has essentially what is a task list. So even though in the prompt, for example, they're instructed, hey. You have seven days of time. You have unlimited API usage, and you have full permission to, like, fan out sub agents, do whatever you need, etcetera, to get the best and... Best possible results that you can on this exam. What we observed is that the harness essentially creates this list of, like, okay. We need to check and make sure that we've submitted all 800 or so different races. We need to check and make sure that we've submitted all the forecasts, that they have been accepted. And then what we see is that there's essentially a goal condition, which once met, ends the term prematurely. So we actually ended up having to essentially force the agent to continue working further by any time the agent would choose to stop its work by essentially reviving it with a new sandbox, a new container, and access to its, like, old runs and old messages. So this is, I think, like, sort of more of a harness engineering question itself as well as sort of, like, a general model behavior problem. But it's definitely a different results compared to what we see with much more frontier models. Right? When we see OPUS 5.5 or when we see g 56 Astra, their behaviors are vastly different. For example, on one given day, OPUS 5.5 would fan out something akin to 170 or 180 different sub agents and manage all of these sub agents by delegating work such as, like, hey. Can you look into the polling results for a specific state? Can you, like, do this detailed review of what each individual ballot initiative in the state of California, Prop 40, is doing? Tell me a bit about that, and tell me a bit about, like, what those things are doing. And then, like, using these sub agents to really help it paralyze a lot of the work that I wanted to do. Right? So I think this is, like, this general question of, what agents are doing, especially when we get to much more long and long horizon open ended tasks like these, and serve proactivity and as well as creativity of what they do. One of the things we were pretty excited to see, was if agents would do something sort of outside the realm of what we would expect an expert forecaster to do. Let's say putting out their own poll, seeing if there are, like, more creative ways to find information as opposed to just fetching data, building models, etcetera. We don't see any real massive instances of this happening quite yet, but we see a lot of interesting sort of approaches that individual agents take. For example, one thing GPT six one SOL does is it takes the forecasts of every election modeler that they look at, and it takes the past decade of their forecasts and tries to calculate the shift of how right or how wrong each forecaster was. And then it can use this result to essentially aggregate all of these different election models that these forecasters are creating, whether it's, like, Nate Silver's model or any of the others, like, 50 plus one. And it can sort of skew it based on what they would expect the model to either underperform or overperform as in recent years. So there's a lot of, like, creative methods that we're sort of seeing.

[20:36] Nathan Labenz: So tell me about the results. What, like, how did the models compare against each other? How did they compare against top humans? You know, are... Do we now have superhuman forecasters?

[20:48] Brian Zhou: Yeah. So the overall sort of result that we're seeing right now is that, in general, models perform just as well as the best forecasters, in terms of humans. In the house and in the senate, agents perform slightly worse than the markets, but better than the modelers. And in the senate, the best agents like GPT six Astra and Claude Fable five one performed under the best forecaster, Nate Silver, but ahead of the markets. But all of these are sort of within this, like, very close margin of error. And it's important to note that a lot of the results for very competitive races have not necessarily come out yet. Right now, there's still around a 100 to 200 races that... Across the house, across state governorships. There are still two races in the senate that have yet to be called. So these are very preliminary results. It definitely is important to note, though, that for a lot of these agents, various methods that they have include aggregating, for example, Nate Silver's decision model or aggregating prediction markets that they see overall. So it's definitely not as if they are suddenly just spitting out a number that is not based off of any of the numbers that they're seeing. But we also find that these models are very well calibrated. For example, if a model assigns an 80% chance of a candidate winning across all of the elections where assigned a similar 80% chance, it is right around 80 of the time, which is essentially what you do to measure the calibration of these models. And in this, we find that the best models are quite well calibrated. We're more excited about the results of what this means for later models as well because unlike experts or unlike prediction markets, for example, there's only going to be a thousand dollars of spread, for example, on what we might see in, like, a very local race down the ballot, or expert modelers may not be forecasting what individual state senate seats or state house seats are going to contain. So this is somewhere in which agents shine simply because modelers don't have the attention, and there's not enough people betting on these much smaller markets, essentially.

[22:47] Nathan Labenz: I was gonna ask about the the relationship between prediction and trading. I saw a note that indicated that Fable five one had the highest prior score on its predictions, but still didn't do so well in the markets, and that sounds like the kind of thing we might learn something from.

[23:05] Brian Zhou: Yeah. So part of the report will also sort of cover this essentially multi agent behavior. Right? Because we essentially have one agent working as the forecaster and one agent working as the trader. We set up infrastructure that allow them to communicate with each other. So we're seeing a lot of these instances of, for example, models making up information, parsing information wrong, and these sorts of, like, other broader issues, which partly explains why traders, the actual trading agents didn't do too great.

[23:33] Nathan Labenz: Thoughtful also leads post train bench. Give an agent a base model, a GPU, and a time limit, and see how well it can post train the model. With new results still to

[23:45] Prakash Narayanan: come,

[23:46] Nathan Labenz: I asked Joe what he expected next in light of the meter report.

[23:53] Nathan Labenz: What what is your expectation as somebody who's, like, spent serious time looking at what models can do to train other models? Like, can you put a little more color or sort of, you know, nuance onto your expectations for the next couple of quarters?

[24:09] Brian Zhou: Yeah. I mean, I think one of the general trends that we've been seeing both with the report from yesterday and with earlier reports, whether this was anthropics measurement of how well individual researchers have been uplifted by AI, whether looking at metrics like lines of code written, experiments done, is the general trend we're kind of seeing is that... I mean, today, and this has already been true for many months, researchers have essentially agents that can do the vast majority of the actual implementation work, which is what we kind of consider as, like, research engineering. So the realistic limitations that researchers today have is not necessarily the, like, ability to execute on any of their ideas, but as, like, many people have talked about, right, their sort of research taste in finding what problems and experiments that they want to do, as well as just the, like, speed at which they're able to come up with these, right, given that agents are already autonomously building and executing on a lot of the experiments that they want to run. So I think regardless of whether we get to recursive improvement, we have definitely already seen a huge uplift that will continue to only get better. And then I think it really just becomes the sort of end bottleneck just being resources and quality of ideas and intuition, right, which is, of course, the next hill that everyone is trying to climb. Have you seen anything that you would say starts to look like the distribution of idea quality is overlapping between the models and

[25:37] Nathan Labenz: truly top researchers?

[25:39] Brian Zhou: I don't think so necessarily, but I do think we have observed that with better models that currently exist right now, for example, they exhibit a larger willingness to fan out, do experiments, a better both ability to think of, like, what might be interesting experiments and execute execute on those experiments. Like, we definitely do see with worse models, more of, like, a general, like, linear approach to whether it's post training or other things. And then with more and more capable models that we observe and test, we do see this general behavior of like, oh, it's like spending its first few hours trying to, like, fan out, do a bunch of different experiments, allocating its resources in a pretty intuitive way, and doing what we might be thinking of. None of this is necessarily a move 37 moment because a lot of this is just the agent thinking, oh, like, let me try an initial SFT batch and see how that affects the results, for example. Right? So we don't necessarily see anything that would be indicative of superhuman taste or anything else, But we definitely do see that more capable models are just better at long horizon work and better at the sort of autonomy that we would expect we would want with better researchers.

[26:53] Nathan Labenz: Great to meet you. We'll be looking out for the next version of post train bench, and you've got me wondering what the agents will say about how much acceleration we're gonna see in AI over the next couple quarters. If they can predict the election as well as the experts, we should probably start asking them about the future of AI itself.

[27:19]Tasklet: Tasklet empowers your business with AI agents that connect to your tools and automate recurring workflows with no code required. Visit https://tasklet.ai and use code cog rev for $50 in free credits

Main Episode

[28:57] Nathan Labenz: Part three, paying up for compute. Thursday show came two days after election day when voters on 41 local data center measures sided against the data centers in all but one. Prakash went first.

[29:12] Prakash Narayanan: So I actually think the AI data centers have been, like, very unpopular precisely because it has been seen as number one, Trump, Number two, corporate corruption. And number three, this miasma of, like, rich people making money and, like, with plans to throw the rest of us out of work. So I think I think there is this entire entanglement of the population just not feeling well taken care of? If it's AI data center, it's such a big issue. Do you really believe if Trump had a 60% approval rating and unemployment was low and gas prices were low, you would see this. And I think the answer is no. So I don't think actually data centers were a big issue, but I think they were an issue to tar the president with.

[30:01] Nathan Labenz: But at the same time, we do have these ballot initiatives. And, like, 40 ballot initiatives, from what I had read a little bit, is, like, one of the bigger waves of ballot initiatives. Like, these these sorts of things only come around when you have something like gay marriage, you know, was one where there was, like, a whole wave of initiatives. There was a a similar wave in terms of affirming or protecting abortion rights in the in the wake of the Roe v Wade overturning. And so this does seem like they're... And what what I'd like about Validity is I've long been a big direct democracy fan and advocate. Not to say that people never make mistakes, but, it is something where you can unbundle your concerns about a particular issue from the broader political conversation. And the fact that we did see so much of that and that they all won, I guess, with one exception, does to me seem like a sign of things to come. And I would... You know, we'll see. Maybe we'll get the companies, like, cutting people in on the deal. I think, you know, we've talked about that a few times as well, and I I... People should get paid. I definitely am, on board with that. But doesn't it seem like there is, like, a pretty real signal there to you that, like, yes, it's not the biggest thing, but it it does seem like there's a a pretty serious groundswell of people willing to work, you know, willing to go collect signatures. That stuff is not easy. You gotta stand on street corners and, you know, get strangers to sign petitions. And then it still has to pass on top of that. So the fact that so many of those were organized and so many of them passed does seem like a big signal of what we can expect to me.

[31:48] Prakash Narayanan: I agree, but I'm like, why? Why did they sign? Right? And is... Do you think they signed because it's the end of the world? No.

[31:59] Nathan Labenz: No. Yeah. I think your diagnosis was pretty right. Right.

[32:01] Prakash Narayanan: So I think the the the the outcome of this is that if you're gonna be Google or Meta, you're gonna be paying Google or Meta prices for everything, for the land, for the labor, for the construction.

[32:18] Nathan Labenz: Our Thursday guest was Mike San Miguel of Luxor Technology, which came up in Bitcoin mining and now trades new and refurbished GPU servers. When he says $5.50 to $6.50, he means 550,000 to $650,000 for one eight GPU server.

[32:36] Prakash Narayanan: So what what is what is trading frequently?

[32:39] Nathan Labenz: Right now, it's mostly b 3 hundreds. B three hundreds are trading frequently.

[32:43] Prakash Narayanan: Yeah. And and does... Do do they trade as a chip or as a node or is this... As a rack as eight eight eight together or how how does that work?

[32:52] Nathan Labenz: Yeah. Great question. So typically for these servers, it's an HGX system. So eight GPUs per server, validated build built by an eight OEM. So you've got the HGX baseboard, dual CPUs, 32 sticks of RAM, depending on the size RAM, shared storage, east to west NICs.

[33:14] Prakash Narayanan: Do people trade used on you right now?

[33:17] Nathan Labenz: Mostly for b 300, it's new. Occasionally, we'll see abandoned orders. They... They're new enough and they're profitable enough renting out that typically we we really haven't seen anything used at the market yet.

[33:30] Prakash Narayanan: And how much does one of those cost? Like, what... Like, off the top of your head, like, you know, right now asking asking price or clearing by price right now?

[33:38] Nathan Labenz: Yeah. Depending on on the specifications, the OEM warranty terms, all of that, you're looking anywhere from $5.50 to $6.50. Typically, for bulk orders, it's on the lower end.

[33:52] Prakash Narayanan: And and people are trading new. And what what is the typical lot size? Are people trading like, okay. One one rack system or, like, 10 rack systems or 100 rack systems or, you know...

[34:01] Nathan Labenz: Yeah. Yeah. Well, the the units per rack come down to the data center configurations because that's about how much power can be delivered to each rack. Typically, most data centers will be able to operate two per rack right now, unless you've got a really nice dense supporting DLC data center. In terms of sizes, we trade everything from one or two units up to about a 128 or two fifty six. So... Wow. Yeah. $50,000,000.

[34:31] Prakash Narayanan: $50,000,000 trades.

[34:33] Nathan Labenz: Yeah. They get they get pretty big. The numbers are pretty pretty aggressive. But it's, you know, it's the scale of the build out that's happening.

[34:41] Nathan Labenz: Prakash asked whether those prices track the hourly rental indices.

[34:46] Nathan Labenz: So, you know, one of the areas that we we help our customers with understanding is this difference between a contracted rate and an hourly rental rate on these GPUs. So if you look at, like, an AI factory contract rental for b 300, typical, you know, market price right now is $4.20 to $4.50 an hour depending on the market that you're in and the configuration. Now if you're only getting, you know, $4.20 per GPU per hour, that's a very different rate from what you're gonna see on those indices. That's eight to nine hour... Dollars per hour per GPU. So if you have the the capital to go out and buy, you know, lower amount of nodes, deploy them, then, you know, typically, you can get a better rate of return as opposed to buying, like, a full AI factory and negotiating an offtake contract with NeoCloud.

[35:41] Prakash Narayanan: Why is there excess inventory? Like, I mean, these things are pretty expensive. I I I I I assume, like, in order to buy them, have to put on a prepayment or something like that before before they start manufacturing them for you.

[35:52] Nathan Labenz: Yeah. You do. Part of it has to do with data center availability. So this market's looked like a a series of different bottlenecks. Right now, it's liquid cooled and data center, readiness. So, that number I gave you with most data centers being able to support, let's say, two nodes per rack, that has to do with constraints, not on the actual hardware side, but on the data center's power delivery. Same with, sort of the cooling build out as well as sort of a bottleneck right now. Liquid cool, we're starting to see data center inventory coming available January, q one. But, you know, most of that's already sold out. There's a lot of data centers that are now telling us it's q three twenty twenty seven or 2028.

[36:40] Prakash Narayanan: Are are people are people trading the market? Are they buying are they buying their systems without having places to plug them into?

[36:47] Nathan Labenz: No. Typically, what happens is last minute, you'll have a a project push, and that's why some of these OEMs will reserve the right to resell inventory. If, let's say, one of their hyperscalers sites isn't ready, they'll be able to sell that inventory and then backfill it when that that data center gets to be ready. Typically, you have

[37:07] Bryan Bischof: to

[37:07] Nathan Labenz: have an address.

[37:08] Prakash Narayanan: You know, at at some point, we will see these h 100 rental prices dip. And now it sounds as though if the prices go down, you're gonna see more retirement. So you're gonna see more of those enter the trading market. Are there any places to deploy them?

[37:24] Nathan Labenz: Yeah. Great question. So, yes, I agree that as these rates go down, you'll see more that get decommissioned. In terms of sites to deploy them, there will be. Part of it is that not every site can operate every GPU. Vera Rubin is strictly a liquid cooled solution. That's NVIDIA's next generation. Right? The cement slabs that those go on have to be reinforced. Typically, those are what, like a ton per rack. The weight is insane. The power draw is about 230 kilowatts per rack. To give you some comparison, for two b three hundreds, you're looking at, you know, 30, 35 kilowatts per rack. Not every site can do Vera Rubin. Not every site can do g v 300, which is similar to Vera Rubin on power. So, you know, it'll make sense to convert some sites. It won't make sense to convert other sites. AI is not every single workload. Right? AI is a family of different workloads. All of them have different requirements. There is some sizing differences right now, but it's mostly RAM and shared storage local storage and some networking back end. But we we need to get more efficient with how we size to the workload. Because the whole point of this hardware isn't on the hardware or to support NVIDIA's, you know, next, you know, a trillion dollar milestone. The point is to run a specific workload that has real world implications for end users and helps create new businesses. So, you know, as we get out of these very hardware constrained environments, we're able to look more at the software and then the hardware gets more efficient.

[39:12] Nathan Labenz: Part four, who's checking? Mike San Miguel on the rules that travel with the hardware.

[39:20] Prakash Narayanan: Do they do they do a KYC of who is the person buying the thing or who owns the person buying the thing? Do they do they do do they do checks on that kind of stuff?

[39:30] Nathan Labenz: Yeah. Yeah. And, actually, part of my role here was was spinning up our our our KYC department. A lot of the OEMs will not even quote without KYC being done. Some will not ship without it being done. Internally, you know, we basically do not accept projects without KYC being done except in some rare circumstances where it's big names or or companies that we've worked with for years. There are limits on, you know, what percentage can be owned by China. What what

[40:03] Prakash Narayanan: percentage of that company can be owned by China? Mhmm.

[40:07] Nathan Labenz: Yeah. Like, I think they might be going down to 49%. I know typically 50% or more. You you can't sell to companies that are on 50% by by a Chinese entity.

[40:20] Prakash Narayanan: Do do do they do this about... You know, a lot of Chinese entities will go through our Cayman Islands or BBI or whatever. But do they do they trace back through that?

[40:29] Nathan Labenz: Yeah. Yeah. They do. I mean, at least, you know, it's something we do. Every, company is a little bit different. Mhmm. But we we do international orders. Right? We're not just, geo bound in North America.

[40:41] Prakash Narayanan: So,

[40:43] Nathan Labenz: for us, you know, we're we're gonna trace back not just to the, hardware purchasers, but also if they happen to off take. They're signing, an agreement with, Neo Cloud. You cannot legally sell this equipment with the offtaker being a Chinese entity, for instance.

[41:03] Nathan Labenz: Mike signed off there. Prakash and I kept talking.

[41:09] Prakash Narayanan: You know, we we talked to people like Samuel Hammond and, you know, and and others at the... You know, on the DC kind of, you know, facing facing national security decision makers. But then you go down to kinda like the deployment guys, and these guys are running around. They're getting permissions. And every six months, they're expecting a NVIDIA checkup, and, you know, they're chasing on KYC. And they they even know... You know, they're going to the owners, and they're going to the users, and maybe they don't know the user of the user. Right? Like, they they... They're saying, like, okay. Maybe that is unclear. Right? So I I I I wonder to what extent, like, the policy discussions at the top end are kinda divorced from really how in-depth the execution is at this point. Right?

[41:59] Nathan Labenz: Yeah. It... I bet, you know, everything everywhere all at once is probably true here because a lot of what he described did sound to me kind of like the fracking boom in the sense that there's just, like, a lot of... I mean, it's capitalism at work. Right? I mean, if there's a lot of things coming together all at the same time, There's a ton of moving pieces in everything from the sites and the approvals and the local politics and the grid and, you know, and, of course, the chips. And you even gotta reinforce the concrete and, you know, not every site is ready to take on the the latest super heavy racks. I mean, just so many little details, so many people coming in together to do their part to create something that, the end of the day, at the consumer level, is just like, you know, gives you some appreciation for why Claude occasionally has some downtime. But, like, in general, you know, it's pretty amazing how smooth it all is. So I could see both, you know, the enforcement could, in many cases, be pretty legit, especially among, you know, people who care about it or, you know, who are, just conservative enough that they wanna make sure they're absolutely on the right side of the law. And at the same time, that sounded chaotic enough that, if you told me that, like, I don't know, you know, 20% of the GPUs, like, somehow don't get their regular NVIDIA six month check-in. And of those, you know, some percentage found their way to wherever. That wouldn't shock me at all, you know, based on on what he said. It sounds like there's... You know, I don't know. Eighty twenty is obviously a simplistic way to describe it, but it sounds like there's a lot of sincere effort going in and then also just enough noise and chaos that probably the the opportunity for things to slip through is also pretty substantial, I would bet.

[44:04] Nathan Labenz: Then Prakash turned to China and the pressure on American companies over their Chinese customers.

[44:10] Prakash Narayanan: Like, for example, Oracle selling capacity to Tencent, $7,000,000,000 of capacity, that was canceled. Right? So the national security infrastructure is pressuring these companies to cancel that. Right? It's an American company doing business. It's outside China and not sending chips into China. And at the same time, there there is no, like, pushback on the political infrastructure that is allowing this. Why don't you just make laws? Right? Why why pressure the companies? I I just don't like it. Like, if you if you wanna make laws, just make laws. Right? But if you don't have the political willpower to make those laws and you try to slide this under the gate, like, why? Why? Like, you have a mandate now. Right? You have a mandate in congress. Like, do something. Right? We say computers, the new oil. Right? And so there there is also this impact on... Let's say you decide not to sell computer China, like zero zero Chinese content. Right? Do you also mean that, you know, YouTube cannot show AI generated video to Chinese, like, you know, participants, to Chinese viewers, to Chinese customers? Because that is a... That that that is a AI product. Right? And Google needs to generate that AI product. Google has to use compute, and that compute is gonna be used on behalf of a Chinese customer. Right? You can restrict it at the data center level. You can restrict it at, you know, the raw compute level. You can restrict it as a product of compute level. Right? But at some level, you're gonna end up selling to the Chinese because they are global consumers. Right?

[45:49] Nathan Labenz: I had

[45:50] Prakash Narayanan: said

[45:51] Nathan Labenz: we

[45:51] Prakash Narayanan: get

[45:51] Nathan Labenz: carried away with these restrictions. Here's why.

[45:56] Nathan Labenz: I really do think we're gonna need cooperation with the Chinese. You know, that... That's... In terms of why all this matters to me, it's one, because, you know, I just generally don't wanna see tensions ratcheted up between The US and China in general. But specifically on AI, I really think we are going to need cooperation between The US and China on AI governance. I I really don't see how, I'd be extremely pleasantly surprised if we can get through all of this without the two countries coming together in some meaningful way. When you have, as we recently talked about, a super senior person at a frontier company telling me at the curve that there's a level of intelligence we probably just shouldn't go beyond, if that's the view that has now taken hold at the top levels of frontier model companies in The US, the only way to achieve that is gonna be through some sort of agreement. And we're really not putting ourselves in a great position to make that agreement. So a lot of this stuff seems ultimately quite self defeating, and it makes me sad.

[47:06] Nathan Labenz: Part five, what investors are superhuman at? Back to Friday and Brian Bischoff of Theory Ventures, an early stage venture firm. First, the compute. I asked how much slack there really is in the market when we keep hearing both that you can't get GPUs and that you can get them on demand if you'll pay a multiple.

[47:28] Bryan Bischof: Totally. I think there's a couple things at work here. One, if you go to a Neo Cloud and you say you got compute, they're like, nope. It's all locked up. It's all contracted out for the next two years. And they're not lying. Like, they're not telling you an untruth, But there's also been quite a lot of discussion and even, you know, Satya talked about how there's a lot of compute that is locked up as you hear. This is essentially underutilized, compute due to either, like, the geolocation of it or the available, like, energy. And so these are both true at the same time where there is locked up compute that, actually isn't super, useful. There's also the reality of sort of utilization, utilization of GPU. Yesterday, I was chatting with, some folks, working at the infrastructure level around, like, GPU utilization. And one of the big problems is you actually can't run at maximum capacity all the time because it's unsafe from a scheduling perspective. You actually wanna build in Slop. And so there... There's actually so many more dimensions than simply do you have contracts on the available compute? And this is why you see many different routes to big success and, like, embarrassingly large numbers of ARR in the compute market. This isn't just because of FOMO. This is because there's actually so many profiles that you need to, kind of, like, maximize the value of whatever compute is available to you. And so, we run, you know, thousands of jobs here at Theory Ventures that are powered by AI agents. If you ask me, what's the distribution of immediacy for those jobs? Well, I'll tell you. Okay. Maybe five to 10% is right now. And maybe 40 to 60% is I'd like to see it before my next meeting. And the rest is, I don't know, maybe, like, this week would be neat. Like, you know,

[49:31] Nathan Labenz: and that

[49:31] Bryan Bischof: even just in,

[49:32] Nathan Labenz: like, this one

[49:33] Bryan Bischof: sort of niche of the kind of work that we're doing, we already have quite a broad variance of inference needs. And that's true of, I would say, almost every AI application layer company that I've ever spoken to since 2023. And so from that alone, you can see that if someone tells you there's simply no GPUs to be had, well, that's not really, like, accurate. But, also, they're speaking from a particular, like, bias. And the bias they're speaking from is probably for the workloads that is under demand right now for me.

[50:13] Nathan Labenz: Then how an AI native venture firm actually runs, how they think about token spend, run experiments, and build research agents.

[50:23] Bryan Bischof: Everyone is technical, and that's all the way from sort of our talent partner to our GTM partner through the investors. Everyone is using AI. Everyone's been using AI since the day that I joined and, more and more. The way that we think about token spend

[50:43] Nathan Labenz: is,

[50:45] Bryan Bischof: you

[50:45] Nathan Labenz: know,

[50:48] Bryan Bischof: you should be thinking very, very clearly at all times if there is a way to accelerate, by using AI tools. You should be thinking about experiments that you wanna run, and we're a really experimental bunch. To give you a sense, every quarter, we do our little, like, off-site where we talk about just everything that happened over the quarter. One of the things that we talk about every single quarter is what experiment every individual did that quarter with AI. That's not an exaggeration. And this is not like, you know, oh, I asked Claude to do something new this quarter. No. These are... You preregister what kind of experiments you wanna do related to your job that use AI, and then you check if they worked. This happens every quarter because we're so insistent on rethinking what does it look like to work in a venture firm in, you know, this year, next year, and the year after. How are we adapting to those things? As a head of AI, my engineering team is obviously still operating at another level, which is how can we build tools that use AI to transform other people's workflows. Our goal is not to replace an investor with an AI system. Our goal is to make that investor 10 to a 100 times more effective and efficient by using AI tooling and building the AI tooling that will make that possible. Let me give you a quick example. Everyone talks about, like, company brain no matter what kind of business you're working in. I've used a couple company brain products and candidly found them all to be useless. The main reason is because they are trying to build sort of what we used to think of as this internal knowledge system. You can think of Notion, for example. Notion is, like, a stack of files, and maybe you can build a good search on top of it. Is that really your ambition? My ambition's a lot higher. And so my ambition for a company brain is agents who are consistently working to connect the disparate nodes in the graph of understanding from your business. And so that's why I can easily say, yes. We don't have, you know, an agent for every human at theory, and we don't have 10 agents for every human at theory. We have thousands. And there's a reason for that. It's because there's a lot of work to get done. My ambition is that every piece of work that an investor does during the investment process, the research process... Because remember, we're very research focused. Every piece of work that they do during the research process is automatically linked, enriched, and sort of expanded. This is why when people say, what are you building? I say, I'm building research agents.

[53:38] Nathan Labenz: How do you evaluate success? I mean, obviously, one of the huge challenges in venture historically, and this hasn't changed yet, is it's a long time to find out if you made the right bets or not. Right?

[53:50] Nathan Labenz: So Totally. Sometimes I do wonder if...

[53:55] Nathan Labenz: Including for myself. You know, I've got... I've... I I periodically go do, like, a cron job cleanup where I'm like, there were ideas that I asked about taking, you know, every bit of data exhaust and, like, processing them in some other way and finding connections and being, like, super galaxy brained about it or, you know, scouting this or reporting that or whatever. And I'm like, probably two thirds of those if I'm real about it. Like, I never actually look at the outputs of. And so it's easy to sort of have these ideas, spend a bunch of tokens, and sort of just end up logging a bunch of text text somewhere without really making an impact. How do you know what is actually working and and what is, you know, potentially subject to leaving on the cutting room floor?

[54:43] Bryan Bischof: Let me tell you a story about Blue Bottle. When I was at Blue Bottle, we wanted to build inventory forecasting. And, specifically, we wanna build a machine learning system that could help us predict how many pastries we were gonna sell on a given day and other things like coffee and milk and so on. But pastries was the first focus.

[55:02] Nathan Labenz: And what was our key, like, north star, our guiding light?

[55:08] Bryan Bischof: Well, it was actually... If this is successful, it's not that we sell a lot more pastries or that we waste a lot fewer pastries. Okay? Those were, you know, two metrics that we tracked. But what would we have believed with success? What we would have believed as success is if the cafe manager who's already got about 70 jobs, if that cafe manager essentially goes into the system, and when it comes time to order, they just click okay. Okay. Okay. Okay. Okay. Okay. Now we don't want them to just not think and just autopilot. We actually don't want that at all, and there's a reason. And this will surprise you, but the variance in sort of what a cafe lead understands with respect to their cafe demand is shocking. It is... It would it would blow your hair back how much they understand about the demand in their cafe that is not just in the numbers. We have these weirdly deep, like, you know, tacit understandings. So what was our north star?

[56:19] Nathan Labenz: Our

[56:19] Bryan Bischof: north star was that a cafe manager has more opportunity to spend that time with customers. Because a cafe manager is not superhuman at looking at data and saying, like, oh, I'm gonna need, like, 25 cookies. No. A cafe manager is superhuman at understanding hospitality. And that may sound weird because you can't manage that on the dashboard. You can't look at hospitality on the dashboard. But what you can do is you can ask them periodically, how much more time do you have available to spend with customers, to spend on the floor and notice that there's, you know, a coffee spill, to understand that your baristas are slammed right now and actually, like, staffing was off. Also my fault. But, and so these are the things that we always cared about. Why do I bring this up? Well, yeah, you could say, like, okay. It's, like, seven to fourteen years to find out if this, like, you know, diligence was good.

[57:18] Nathan Labenz: Sure.

[57:19] Bryan Bischof: But it's a pipeline. So first of all, you wanna ask yourself, how fast can you get through the research pipeline?

[57:27] Nathan Labenz: How

[57:27] Bryan Bischof: fast can you go from identification of a potential opportunity or market to a clear understanding to be able to make a decision? But more importantly, hospitality. What does hospitality mean in venture? It means that the investor has time to talk to human beings, to talk to potential or existing companies, to talk to founders and competitors and employees, to try to understand the human element of this market and what's going on. Investors are superhuman at a few things, but I don't personally think investors are superhuman at looking at websites. But they are superhuman at sort of understanding potential, understanding upside, understanding sort of the relationship between, like, you know, customer need, willingness to buy, and sort of framing what the vision in the future can look like. That's what they're superhuman for. And so when you ask me, like, how do I measure success? Why measure success when an investor tells me, yeah. Actually, I get to spend 30 percent less time staring at, you know, the website and trying to figure out, like, the important details I look for every single time. Or I get to spend 20% less time filling out, like, you know, these specific answers to questions which are important for diligence, and instead picking up the phone five more times and talking to five more people who have credible opinions on this deal. That's success. And you don't need to wait seven years to find out.

[59:03] Prakash Narayanan: Do you find there is kind of a spectrum of, like, let's do more research to... We need to cut off the research and we need to make a decision now? Because it is... The research process doesn't really have a defined endpoint. You could always dig deeper. Correct?

[59:21] Bryan Bischof: Totally.

[59:21] Prakash Narayanan: How do you how do you make those decisions? Are the agents the ones making the decisions? Is there a stepping off point where they have to come back And you make that... You know, a human is making the decision. How does this loop work?

[59:33] Bryan Bischof: Totally. I wanna reference something that Nathan said earlier. You know, he's got this list of things, and he's like, some of this should have been left in the cutting room floor. We would we would call that taste and sort of, like, knowing what should be cut and what shouldn't be cut is taste. And yet again, here's where I will point to why it's important to have exceedingly bright colleagues. When you have exceedingly bright colleagues, they've got great taste on, like, what research needs to get done. There are some research that, like... Yes. Like, we always wanna do it. We know we need to do it. This is the structure. And there are opportunities for agents to say, let's spin off some sub agents to continue doing this kind of research. Let's dig deeper on this. That's fine. That's automated. But, ultimately, the alpha is in the the taste. And so... Yeah. Like, we can burn more, you know, tokens. We can burn more compute, but I don't know. It's a pretty it's a pretty, like, focused laser where you really wanna point these resources at things that people will have the ability to consume. I... If I produce an extra thousand pages of research on some deal, who's gonna look at it? I guess other agents could look at it, but, like, then what? Is that valuable? It's not obvious to me that it is. And so, again, pure volume of research is not a metric that we even track.

[1:00:54] Prakash Narayanan: How goes your teaching of, data science at, I believe,

[1:00:57] Bryan Bischof: Rutgers?

[1:00:58] Prakash Narayanan: Yeah. So what what do students... You know, students are probably asking some similar questions there. So, you know, what what what do you say to them, and what do you what do you think the future of that field is?

[1:01:09] Bryan Bischof: Yeah. Great question. I still make them do gradient descent by hand. So this is an area where, in my opinion, I have yet to hear a single person with a cogent explanation of what the hell we're supposed to do as teachers. I believe and have believed for a long time. You know, I came up in pure mathematics. We're we're not the most, practical bunch. But I have always believed even as a pure mathematician that you should be constantly thinking about what your student's future looks like and why every minute in your class is relevant to that future. I am not just, like, phobic, but, like, allergic to wasting students' time. So what I do is I reassess every semester much to my own suffering because that means I have to rewrite my curriculum every single semester on what do I teach them. So three years ago when it was obvious that

[1:02:12] Nathan Labenz: sort of AI was going

[1:02:13] Bryan Bischof: to change data science, and then I started working at Hex trying to change data science. I rewrote my entire curriculum to introduce why AI might be relevant. And every semester, I go through the whole course, and I ask, can AI just one shot the course? The answer has never been yes, but that's okay. The answer has always been, okay. These things are now not just easy for the model, but easy for the model to obfuscate the learning. And so I don't think the question is, does the model know all the content that I plan to teach? The question is instead, does the model have the ability to collaborate with the student when this is gonna be relevant for the student in a way that yields the sort of best outcome? If the model can... If if, you know, Astra can train an SVM on the coffee survey data at a better loss rate than Brian can meet. But can Astra say, you know, this feature is highly predictive. But when we think about it, if we put this on the website, it's going to upset some of the people going through the flow, and they might churn. Like, yes, Astra can look at the data posterior and say, people were churning more on this question. But does Astra know ahead of time to look for that? Does it know to instrument that in the product? Does it know that this is kind of a weird question and people might be bothered? These are the kind of things that Astra Ultra is still only mid at. But, ultimately, I don't want my students to only be users of AI. I want them to be sort of the things that make data scientists exceptional, and that is ultimately curiosity. Like, every data scientist that I've ever met that is in the top decile and certainly in the top percentile are curious as hell, almost to a fault. The absolute best data scientists I've ever worked with are borderline annoying with their curiosity. Like, you kinda want them to just, like, leave it alone sometimes, and that's what makes them so excellent. And so mostly, what I'm trying to, like, indoctrinate my students into the propaganda is curiosity. Be curious. Ask the questions that you can't get out of that brain. And when you do that, you're gonna be an excellent data scientist. Whether you're writing PANDAS by hand, whether you're using MATLAB from memory, I don't really care. Do you remember the indexing for NumPy? I don't. I used to know it. I don't remember it anymore. Do I remember the slice operators for, you know, pandas filters? Not anymore. I forgot them all. But I used to know them. It doesn't matter, though. What I know now is I know sort of when you look at a distribution, whether it makes sense. And sometimes Astra doesn't.

[1:05:11] Nathan Labenz: Part

[1:05:11] Nathan Labenz: six, which hills? After Brian Bischoff signed off, Prakash and I kept going.

[1:05:19] Nathan Labenz: That, his answer on what he tells his future student or his students about their futures, I thought was really interesting, and it it sort of contained the pessimist and the optimist take at once when he was first going through the... You know, sometimes there's, a survey question that may offend people, and you gotta be wise to that. I was like, man, that's bleak. You know, the idea that, like, we're just gonna be sitting here being the sort of offense detectors, you know, while the AIs do all the real work. I was like, oh god. You know, that doesn't sound awesome. I did like the curiosity framing a lot better. Being relentlessly curious, certainly in my experience, working with data has been indispensable. You know, really want... The difference between somebody who really deeply wants to know the truth that the data contains to the degree that the data even contains truth, which is often a problem unto itself, Those people absolutely kill it compared to those that are just, trying to produce an artifact that they can say, you know, is done and and hand off to, whoever the consumer of it is. So I do like curiosity as an answer to, like, what we're gonna bring to the future, but it does... I would not be telling my kid to go into data science today. I'll tell you that. So that is a tough one. I don't... You know, the the real question there of, like, is there going to be a job in data science that looks anything like the current job, in the next few years? I I feel like that is... I think that's pretty tough. You know? It seems like we're gonna have data science agents, and it's gonna be up to the sort of nondata scientist general purpose business operator to provide that little salt of curiosity or offense detection or whatever that, you know, special sauce is that the models don't quite have.

[1:07:12] Prakash Narayanan: I feel like, you know, if you look at a a company, you kinda cut it apart into, like, sensing, analyzing, and decision making. Right? And then action... Decision making and then the reverse, which is, you know, operationalizing and actions. Right? And I think AI is gonna be very good at the sensing and analyzing and also the operationalizing. But I think the decision making, you end up with a human in control. Right? And then if you take the human in control, okay, you can have a business with a single human in control. What's the number of, like, decisions that that human can make? And those decisions are decisions with taste. For example, like, if you're running a tech company, you don't need decision making maybe on the accounting framework. Right? Like, you're you're you're looking at the product. You're looking at the technology, etcetera. But accounting in the back office is a back office. Right?

[1:08:07] Bryan Bischof: In

[1:08:07] Prakash Narayanan: the same way, if you're running an accounting company, the accounting is, you know, front and center. But the technology is, you know, kind of a back office. You're not you're not, like, you're not there building technical tools. Right? So So I feel like you can separate out companies based on what is the ultimate purpose. And then the ultimate purpose has all of the humans with, you know, taste in there. Basically, everyone gets to do what they really wanna do and what they're really good at, and everything else gets taken care of.

[1:08:39] Nathan Labenz: Earlier that Friday morning before the guests, a new benchmark had been making the rounds. AIs playing blood on the clock tower, a social deduction game of bluffing and collusion. And the timeline was cheering how good they have gotten at it. Here is what I made of the game.

[1:08:57] Nathan Labenz: If you kind of take it in isolation, it's like, you know, all else equal, it is interesting to know how good are the AIs at playing this game. Are they getting good at deception? Are they getting good at collusion? Are they getting good at sniffing out when other people are trying to deceive them or collude against them? These are all, like, interesting questions. What kind of bothered me about it, though, was the sense that, like... And then Noam Brown comes in over the top and says, wow. This is great work. You know, you wanna come here and make evals? And that kind of embodied to me, like, what I thought was problematic about this whole thing, which is just like, we're just gonna hill climb on this now. Right? Like, it it seems like the the mentality of the field as a whole is like, oh, isn't this exciting? You know? And let's have a horse race as to who can be the best blood on the clock tower AI player. And I think what we're all kind of forgetting as we get all wrapped up and excited about this and, you know, wanna go play the AIs ourselves and everything is like, this is not good behavior from the AIs. Like, we do not want AIs deceiving and colluding. I'm sure all these, you know, people have sort of a thoughtful response to this, but it's just like zoom out and the situation is bad. You know, we we have AIs that are untrustworthy in all these kinds of ways, and we're not even, like, taking that beat to say, hey. Is this what we want? Do we want AIs that are gonna outconnive and outcollude us? I don't think we want that. That's, like, exactly... You know, this is very similar to my idea that we should not train AI generals. Like, let's not create... No. Let's not train AI master manipulators. It seems simple, but here we are. And, like, I didn't hear... I didn't see... I saw a lot of cheerleading and a lot of enthusiasm, and I did not see any corrective commentary on the timeline saying, hey. Is this really what we want?

[1:10:53] Nathan Labenz: Back to the end of that same Friday show after Brian had signed off. Prakash with a picture of where the economy goes from here.

[1:11:03] Prakash Narayanan: So I I I feel like the going forward economy is an agentic economy, and that means agents building businesses and software for other agents, which are suitable for their affordances, which is gonna be, you know, massively parallel, high speed, you know, able to, you know, able to absorb information and distribute information at very low latencies. We are gonna end up with a Salesforce primarily used by agents, a ramp primarily used by agents. All of these softwares are Stripe primarily, again, used by agents. Maybe maybe Stripe or other companies will be the ones that create the software. I am very doubtful. I'm very doubtful because I don't think they will move that fast. I don't think they are capable of moving that fast. I don't think they understand the affordances. And I know Stripe is basically, you know, a bill... People a billion times smarter than I am. I'm not I'm not trying to downplay, you know, how effective and how intelligent and how competent Stripe is. Like, you're gonna get 50,000,000 Patrick Hollis. You're gonna get 50,000,000 Elon Musks. You're gonna get, like, humans who give them capital to deploy. You're gonna get agents coordinating investments into businesses built by agents for agents. And so I I largely see the future of the economy as primarily agent driven with us just receiving free benefits because, hey. You know? CRM call from a human is, like, so so costless in comparison to the massively parallel CRM calls that an agent would make. The abundance basically comes from us building all this stuff for this agentic economy. And as a result, we just have all of this excess because we don't we don't... We just don't consume that much. Right? Like, we're we're just not, you know, that consumptive in comparison to what the agents are gonna consume. And and that's and that's where this abundance is gonna come from.

[1:13:03] Nathan Labenz: My reply leans on a side project I'd launched the week before, Fruitopia, which finds the best fruit at your local grocery and around the world. Here's where I came down on the economy Prakash had just described.

[1:13:17] Nathan Labenz: And on the one hand, it's, like, cool if, you know, there's abundance in, like, API calls because the agents are using so many and they, like, build up, you know, great systems that they can use, and then we get to kind of piggyback on them. But, like, there's a lot more to life than API calls, and I do feel like we're sort of especially... And especially those of us that are in the AI space are, like, really quick to forget that, or we sort of, you know, are... We're kind of contorting ourselves in a way to, like, like this future. Is why I've been, like, holding myself to this idea of, like, I wanna be getting outside more and getting more exercise. And if I'm not doing those things, am I really winning, or am I just kind of plugging myself into some, you know, AI powered machine as, like, some sort of, you know, human integration layer type, you know, component that's, like, in a way, like, too subservient to the machine. But this is why I decided to do this Fruitopia project recently. And, you know, it's again just one of these things where it's like, okay. If we are in a bright future, then I think one, you know, very kind of deeply rooted thing that we all like, we're evolved to like, is eating delicious, fresh, in season fruit. Right? Like, everybody loves that. It's it's it's... Goes back to when we were monkeys that, you know, we loved that. When I tweeted that, I got, like, basically no play on it versus the blood on the watchtower thing. Everybody's like, oh my god. Like, look how great AIs are getting at, like, deception and conniving and, you know, winning these, like, games of social manipulation. And I'm just like, I don't really want that from my AIs. What I want is a future where I have not abundance of API calls, but abundance of things that are, like, more biological. Give me the fruit. You know? That's what I really want. The the fruit of the AI revolution should indeed be, like, at least in part, actual literal fruit. And if we're losing that, we're, like, losing so much of what has mattered to us. And if we can't even, like, notice that we're losing it, I think in a way, you know, that that sort of worries me and makes me think, like, maybe we've kind of already lost the plot.

[1:15:34] Prakash Narayanan: I mean, it's pretty obvious to me that we're gonna get that grown, like, food, which is far more nutritious and far more tasty. It's it's gonna be... Like, you know, in twentieth century, we discovered chemicals. And then we... You know, when you look at these food products right now from, like, Doritos or whatever, those are, like, so scientific. Right? They're they're... There are, like, like, thousands of PhDs who have worked on Doritos. Thousands. Literally, thousands.

[1:16:00] Nathan Labenz: The old... Remember the old onion article? Doritos celebrates its one millionth ingredient?

[1:16:04] Prakash Narayanan: Like, they have figured out your flavor palette so well that they can, like, you know, pinpoint exactly what, you know, flavor that they wanna produce. And all of that technology is gonna get deployed into bio. Right? We're gonna produce, like, you know, fruits and, you know, vegetables, etcetera. Hybrid, you know, fruit is already basically GMO ed, but we just, you know, took a very long time to GMO it. Part of abundance is gonna be, you know, your 99 percentile wine available for $2 Chuck.

[1:16:33] Nathan Labenz: Right? My favorite apple is the Honeycrisp, and, you know, it's a relatively recent creation. Not GMO as far as I understand, but definitely the product of, like, large scale systematic crossbreeding, which is sort of, you know, GMO before there was GMO. So, you know, I... I'm not afraid of, GMO enhanced fruit by any means. I'm all for it. But but it is still something that is, a biological construct that feeds us as biological creatures and, like, you know, delights us in terms of our biological senses and the, like, you know, ephemeral qualia that we get from them. And, like, this stuff is just all too often lost because it's not very measurable. It's not so hill climbable, and it's not what the agents need. Right? And I think we we are gonna need to be very careful not to create an economy that really serves agents and creates abundance for us in things that are core to what agents need. We're gonna... If if we... That seems like what... I kind of agree with you. That might be what happens by default, but I would challenge everybody to think, like, wait. Like, what if the stuff that agents need in abundance isn't the same stuff that makes our lives good.

[1:17:47] Prakash Narayanan: One of the annoying parts is that, you know, the the Frontier lab guys are very serious. Right? And so when you're very serious, you only talk about cancer, curing cancer. No one's talked about a better Doritos. No one's come out and say, like, we're gonna make a better football game. Right? Like, where are these, we're gonna make a better burger. We're gonna make a better football game. Where are these things that people enjoy? And that's and that's... I think that's because they want it to be serious. They wanna show that earnestness. They don't wanna come off as, like, foolish, etcetera. Maybe you should focus on, like, we're gonna make a better Doritos chip. We're gonna make one that's healthier for you. We're gonna make a Doritos that is naturally grown from the ground, and you just need to cut open the potato, and it's Doritos. And and this is like... This sounds like so frivolous and idiotic, but this would catch the imagination of, like, people on what is actually gonna be possible.

[1:18:38] Nathan Labenz: Right? I mean, I think it's also because they do have this kind of World War two mindset about their work. You know, that they they genuinely do believe time is short, stakes are high. The... You know, their their own, like, allocation of time is, like, very skewed toward work. You know, there's just not a lot of space for leisure in the lives of the people at the frontier companies. So Yeah. It's like, they're just so pattern matching at this point. I'm like, here's a hill. Let's go climb it. And, you know, I I do think, like, a little more discretion on which hills do we really wanna climb. Like, are we really happy that we've created AIs with the current shape? I don't know. It it feels like, unfortunately, there's just not enough time and energy going into those sorts of questions. So I'm stuck on that maybe a little bit today, but, man, the blood on the on the clock, everything really did it to me. Because, you know, again, I... It is good work, it is interesting all else equal. But we're... I think we need some thinking that challenges the all else equal paradigm on so many levels.

[1:19:48] Prakash Narayanan: Yeah. It's it's it's a really tough one because I I I don't think, you know, agents can operate in the economy with... Without some level of subterfuge. Right? You can't tell your pricing to the other guy, for example. You can't you can't tell your internal costing to the to the to the guy who asked you. Right? You have you have to reserve that. And, you know, you... That that means that involves omission. Right? And you have to know when to omit. So there there... There's a bunch of things there. This is also why I think, you know, Palantir had this idea of the anthology. Right? Who... Who's allowed to know what? Right? That that... That's a pretty important thing. Like, who's allowed to know what and, you know, who's allowed to say what, who's allowed to do what. Right? And I think also you need this kind of, like, interrupt from government, which possible, or interrupt from regulation or interrupt from... Which means that the agents have to obey when government decides like, hey. This is... You know, you can't do this. The agents have to obey. So there's a bunch of things that I think need to be worked out in terms of governance. I I... I'm sure I'm sure we'll get there. I'm... But, you know, we're gonna have some trial and error along the way, I guess.

[1:20:57] Nathan Labenz: Yeah. No doubt. Maybe in closing, I'll just cite one sentence from the meter thread on their report on acceleration. They say, the full analysis estimated how much of the acceleration and capabilities was due to compute data and algorithmic progress. And then in a... That's first sentence. Now I'm just editorializing. In a thread in which they have caveated everything, not confident, and the... You know, it's it's very unclear how to interpret data, and they've, you know, gone around and around in circles with themselves and with anthropic team. And, you know, there's just so much, fog of war. And even the core estimate of two x is really like... They wouldn't be surprised anything 1.3 to three x. So caveats everywhere. The next sentence is is the only one I noticed, so far that begins with we are confident. And the sentence is we are confident that the acceleration was not due to an increase in the growth rate of training compute. So that would suggest that in terms of government interventions, you know, we... Who knows what we'll see in terms of data center build out, and maybe we'll see a slightly slower data center build out than we would have counterfactually if the election had gone a different way. But it is striking that the acceleration that they are reporting on here is not just a raw material, input story. It is an algorithm story, and it is a data quality story. And those things are being driven to a very large degree by the intelligence that we already have, which, again, is where it goes from, you know, not just bigger and bigger capital, but actual kind of recursive self improvement. So if we are gonna try to change that trajectory with the government, intervention, it's probably gonna have to be more focused and pointed right at the practices of the frontier companies than something as diffuse as a broad slowdown in data center build out.

[1:23:04] Nathan Labenz: Thanks for listening to the AI and the AM weekly highlights. The narration was my cloned voice and we'd love to hear what worked and what didn't. See you in the morning.

Outro

[1:25:29] If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries either via our website, cognitiverevolution.ai or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts which is now part of a sixteen z where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at aipodcast.ing. And thank you to everyone who listens for being part of the cognitive revolution.