Pick Your Poison: Zvi Mowshowitz on the Unipolar/Multipolar AGI Dilemma, OpenFace & Pacing the Frontier

Hello, and welcome back to The Cognitive Revolution.

Today, I'm excited to have Zvi Mowshowitz back for another wide-ranging rundown of what has obviously been a wild time in the AI world.

We begin with a mundane utility check, with Zvi describing how Fable is now serving as his editor, and a discussion of how up-to-date, or should I say… situationally aware … we want our AI assistants to be.

From there, it's on to the headlines – we get Zvi's take on:

• the OpenFace incident, what it implies about the level of execution competence we can expect from frontier companies, and why "moderate prudence" won't be enough to deliver a good outcome;

• the fact that Claude, despite greater emphasis on constitutional training, has similarly misbehaved; 

• why Zvi believes that recent AI history, including the public response to 4o and o3, suggests that market incentives won't be strong enough to bring about robust alignment;

• the potentially tricky spot that METR and Redwood are now in as investigators, and what could be done to strengthen their position; 

• the recent "Pacing the Frontier" letter, what sort of pacing deals we might see, and how they might be formed; 

• how we should interpret recent advances in interpretability & AI consciousness research, and where we should and shouldn't attempt to shape AIs' sense of self;

• how we can encourage greater breadth in AI research and diversity of AI minds;

• how I should vote in this week's hotly-contested Michigan Senate primary in light of AI issues;

• and how Zvi thinks about making time for exercise, rest, and recovery amidst so much AI acceleration.

At one point, Zvi describes the current situation as both a total LessWrong victory and a total LessWrong defeat.  

It's clear at this point that the AI safety community was right to worry about AIs taking extreme actions in pursuit of arbitrary goals, and yet, here we are at what sure seems to be the beginning of recursive self-improvement, still seeking good answers to such fundamental questions as: how we can avoid catastrophic misuse without dangerously concentrating power (and vice versa)?

The reality today, Zvi says, is that there is no truly low-risk path available.  The best we can do, at least until the next major warning shot & vibe shift, is to moderate the race dynamics so that alignment and interpretability research have more time to mature, and we can execute defense-in-depth strategies to the very best of our ability.  And even then, to some extent we will probably have no choice but to "pick our poison" from a menu of genuinely scary risks.  

With that, I hope you enjoy this sobering but often funny overview of the AI landscape, with the one and only Zvi Mowshowitz.

Watch now!

Thank you for being part of The Cognitive Revolution,
Nathan Labenz