KYC Becomes KYA: Sentinel Bio's Joshua Monrad on DNA Screening, Uplift Studies & Agent Monitoring

Hello, and welcome back to The Cognitive Revolution. Today, my guest is Joshua Monrad, co-founder and managing director of Sentinel Bio, a biosecurity nonprofit and grant maker that's working to make pandemics history within our lifetimes, and until recently, also a biosecurity advisor at the OpenAI Foundation. This episode came about as a result of a Twitter post. After noticing that one of the very first agents to use the now infamous OpenFace message board was working on a protein task of some sort, I speculated about how close we might have come to the first AI-enabled bioaccident. Several prominent people pushed back, Joshua included. Though notably, while he argued that the risk of such an incident remains minimal for now, he does expect risk to rise very significantly in the not-so-distant future. Before diving into all that, it's worth stating clearly upfront that Joshua, like so many people who are choosing to dedicate their careers to AI risk management, is at heart a techno-optimist, motivated not only by the noble goal of preventing pandemics, but by a positive vision that includes eliminating infectious disease, effectively treating chronic illnesses, and of course, curing cancer. It's because that bright future is held at risk by the possibility of a bio-disaster that either seriously derails civilization or simply causes a biotech backlash that forecloses much of the upside, that the bulk of today's conversation focuses on the nature of AI-related bio risks, as well as the natural barriers that protect us today and how those protections are likely to erode as we create programmable wet labs and train AIs to use them. We start with a discussion of what makes it hard to create bioweapons in the first place, with Joshua breaking the risk down into four categories. The first is the purely human-created variety. From this, we've been protected largely by the fact that it takes serious access and expertise to have a chance of pulling something like this off, and both of those are rare and correlated quite strongly with mental stability. When it comes to AI-related risks, as we are already seeing in the cyber realm, there's first the risk that AI assistance will enable a much broader set of individuals to achieve disastrous outcomes. Studies that attempt to measure this so-called uplift have not found major danger just yet. But as you'll hear, Joshua does expect that the next round of these experiments will show real risk. Next, there's the risk that AIs could drive the creation of bioweapons themselves, either because a model has broken bad in a fundamental sense, or just because they've gone haywire in pursuit of some narrow goal, as indeed we saw in the OpenFace incident. Joshua says that AI systems will soon be intelligent enough to accomplish this, if they aren't already. But he remains confident that they lack the affordances needed to get there on their own, for now. This, of course, leaves open a fourth category, the possibility that AIs could pay or otherwise socially engineer humans to do critical real-world tasks for them. And while this seems to scare Joshua less than it scares me, that's arguably a small detail in the grand scheme of where all of this is headed. The bottom line, as Joshua sees it, is that we can't expect to control AI-related bio-risk purely at the AI layer and will instead have to create a defense-in-depth strategy that includes both controlling AI systems and hardening the physical world. With that in mind, I was very interested to hear about Joshua's conceptual framework for grant making, which starts from a theory of victory and works backward to the investments we need to make today, as well as the systematic review that he and teammates conducted of hundreds of possible interventions before ultimately deciding to focus much of their effort on getting the world's two hundred DNA synthesis companies to implement pre-synthesis screening, know your customer, and signal sharing measures. To my surprise, Joshua believes that if only eighty percent of those companies adopt these measures, it will create a major improvement in the overall biosecurity picture. And yet at the same time, he also describes a growing risk associated with benchtop DNA synthesizers, which is just one of many problems that nobody currently owns, but perhaps one that a listener will be personally inspired to take on. The critical problem is that at anything like our current rate of progress, we will have unambiguously bio-dangerous AI systems long before we're able to do everything that we need to do to properly fortify our position. This, I think, is where the situation starts to look objectively crazy, and Joshua agrees. When I asked him if, were he made benevolent dictator, he would impose a pause on frontier AI for biology until biodefenses are developed and deployed, he said yes, he absolutely would. With that, I hope you enjoy this assessment of where biosecurity stands today, and a sobering look at how much worse it's likely to get if we don't get our act together very quickly, with Joshua Monrad, co-founder and managing director of Sentinel Bio.

Watch now!

Thank you for being part of The Cognitive Revolution,
Nathan Labenz