Today, Zvi Mowshowitz returns to The Cognitive Revolution to discuss how recent AI developments like GPT-5 and IMO gold medals have led to modestly extended timelines despite being on-trend, while policy missteps around chip exports to China and alignment challenges from reinforcement learning have increased risk assessments, covering everything from live players in the AI race to safety funding priorities and virtuous actions for navigating our current moment.

Watch Episode Here

Read Episode Description

Today, Zvi Mowshowitz returns to The Cognitive Revolution to discuss how recent AI developments like GPT-5 and IMO gold medals have led to modestly extended timelines despite being on-trend, while policy missteps around chip exports to China and alignment challenges from reinforcement learning have increased risk assessments, covering everything from live players in the AI race to safety funding priorities and virtuous actions for navigating our current moment.

Check out our sponsors: Linear, Oracle Cloud Infrastructure, Shopify.

Shownotes below brought to you by Notion AI Meeting Notes - try one month for free at: https://notion.com/lp/nathan
- Timeline Adjustment: Zvi has lengthened his AGI timelines because there haven't been "very large jumps in capability" that would significantly shorten them. The chance of AGI arriving in 2025 has decreased dramatically.
- Recent AI Achievements Context: While achievements like the IMO Gold medal are impressive, they appear to be on-trend rather than revolutionary breakthroughs that would drastically change timelines.
- Evaluation Standards: There's a discussion about the challenges of trustworthy AI evaluations, with Zvi emphasizing that watchdogs must maintain standards of rigor and integrity "vastly above" what others are held to.
- Policy Priority: A short-term policy priority should be preventing advanced AI chips (B30As) from being sold to China, which requires raising awareness about the implications.
- Nvidia's Influence: Concern is expressed about Nvidia potentially having excessive influence over White House rhetoric and plans regarding AI.
- Career Impact: Working for Anthropic is considered a "net good idea" for those wanting to make a positive impact, though there may be other organizations that offer better opportunities for influence.

Sponsors:
Linear: Linear manages your entire product development life cycle with new AI capabilities that automate coordination, route bugs, and generate updates. Get six months of Linear business for free by visiting https://linear.app/TCR

Oracle Cloud Infrastructure: Oracle Cloud Infrastructure (OCI) is the next-generation cloud that delivers better performance, faster speeds, and significantly lower costs, including up to 50% less for compute, 70% for storage, and 80% for networking. Run any workload, from infrastructure to AI, in a high-availability environment and try OCI for free with zero commitment at https://oracle.com/cognitive

Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive


PRODUCED BY:
https://aipodcast.ing

CHAPTERS:
(00:00) About the Episode
(03:20) Summer Timeline Updates
(14:10) IMO Competition Analysis (Part 1)
(20:08) Sponsors: Linear | Oracle Cloud Infrastructure
(22:46) IMO Competition Analysis (Part 2)
(24:48) Model Withholding Strategies (Part 1)
(31:56) Sponsor: Shopify
(33:53) Model Withholding Strategies (Part 2)
(39:51) P(doom) Assessment Update
(51:25) Defense in Depth
(59:09) Constitutional AI Approach
(01:06:18) Uranium Enrichment Analogy
(01:16:23) Claude Model Differences
(01:30:03) Model Usage Patterns
(01:47:02) RL Bad Behaviors
(02:00:02) Interpretability and Neuralese
(02:08:06) Agent Development Prospects
(02:23:28) China Chip Policy
(02:42:09) AI Safety Funding
(03:06:23) Adversarial Model Evaluation
(03:12:47) Closing Action Items
(03:13:09) Outro

SOCIAL LINKS:
Website: https://www.cognitiverevolutio...
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://linkedin.com/in/nathan...
Youtube: https://youtube.com/@Cognitive...
Apple: https://podcasts.apple.com/de/...
Spotify: https://open.spotify.com/show/...