AGI-Pilled Cyber Defense: Automating Digital Forensics w/ Asymmetric Security Founder Alexis Carlier

Full Transcript

(00:00) Nathan Labenz:

Hello, and welcome back to the Cognitive Revolution. Before getting started today, I want to take a moment to introduce our newest sponsor, Granola. You probably know Granola as a leading AI notetaker, but it's in fact much more than that. Because it works at the operating system level, it can capture all audio in and out of your computer, allowing it to take notes not just on meetings, but on every podcast you listen to, every video you watch, and if you choose, everything you say. Right now, to help new users make the most of the platform, Granola is featuring AI recipes from entrepreneurial thought leaders, including several past guests of this show. There's a Replit recipe that converts discussion notes to a Replit app build brief, a Bento recipe that creates content production plans, and a Dan Shipper recipe that looks across multiple sessions to build an unspoken company culture handbook. My own recipe, which you can try now on Granola, is a blind spot finder. It looks back at recent conversations and attempts to identify things that I am totally missing. This has already proven useful in the context of contingency planning for my son's cancer treatment, though I am very happy to report that he is still doing extremely well. And over time, I expect it will become invaluable for suggesting AI topic areas that I've neglected and really ought to explore. For today, my guest is Alexis Carlier, founder and CEO of Asymmetric Security, which just recently came out of stealth. In response to the launch announcement, Logan Graham, who leads the red team at Anthropic, described Alexis as one of the most AGI-pilled founders in the space. So naturally, I had to find out what that means for the future of cybersecurity. As you'll hear, Alexis's motivation is increasingly familiar, but nevertheless profound. If we assume that AGI is coming and that it represents a near infinite supply of intelligent labor, the question becomes, how should we redesign our cyber defenses from the ground up? His answer is to move from a paradigm of reactive emergency triage to one of proactive continuous digital forensics. We begin the conversation by describing the current threat landscape, distinguishing between the spray and pray tactics of financially motivated criminals, the more sophisticated ransomware attacks of cybercrime gangs, and the patient high-stakes IP theft operations conducted by nation states like China. Alexis also shares fascinating details on the North Korean remote worker phenomenon where state-backed actors infiltrate western tech companies not just to steal secrets, but also to earn salaries that actually fund the regime. From there, we turn to how Asymmetric is building AI agents capable of performing the deep investigative work that was previously only available from a very limited number of expensive human experts. We discuss the jagged frontier of current model capabilities in the security domain and why, though off-the-shelf models can already achieve 90% accuracy on many investigative tasks, Asymmetric is going to market with help from insurance companies with a services-first business model focused on business email compromises, both to ensure that they deliver consistently for customers and to build the proprietary dataset needed to close that final gap in performance and reliability. Importantly, Alexis also makes the argument that while most AI technology is inherently dual use, the fact that people who specialize in these investigations don't tend to become particularly outstanding hackers suggests that digital forensics could be a domain that allows us to differentially accelerate defensive AI capabilities. And arguably, it's this strategy of intentionally shaping the AI capability frontier that is Alexis's most important contribution, and I would love to see this approach ported to other domains. The opportunity to build specialized datasets, evaluation methods, and training environments that can help harden society's defenses seems both increasingly tractable and urgent. So if you're thinking this way and building, whether in cybersecurity, biosecurity, mental health, or anything else where a DX strategy could make a big difference, please reach out and tell me about it. With that, I hope you enjoy this overview of the cybersecurity landscape and preview of the future of automated digital forensics with Alexis Carlier of Asymmetric Security. Alexis Carlier, founder at Asymmetric Security, welcome to the Cognitive Revolution.

(04:26) Alexis Carlier:

Thanks for having me.

(04:27) Nathan Labenz:

So yeah, I'm excited about this conversation. An interesting little endorsement of you and your point of view that I saw online on Twitter from Logan Graham at Anthropic, who leads the red team efforts there and really kind of is at the intersection of national security concerns and what can models do and has been deep down the rabbit hole of how weird the world might get in the not-too-distant future. He said about you in response to the announcement of the new company that few people are more AGI-pilled than you. So I wanted to start off by just getting a little bit of your worldview. You can tell maybe a little bit of your background too and how you came to be one of the most AGI-pilled people around. But how did that happen? And even more so, what does it mean to be so AGI-pilled?

(05:13) Alexis Carlier:

Yeah, I saw this. I suspect what he's getting at is I've taken throughout my career professionally done a lot of stuff that's been already premised on the idea that AI and AGI is going to be a big deal. So a few years ago, I was on the early team of the Center for the Governance of AI, which is a big AI governance think tank, especially prominent in the UK and in the US and elsewhere. And this was like in the pre-ChatGPT, 2020, 2021 days when AI people were talking about it, but it wasn't really this big thing it is today. And back then, we were trying to think about what would happen as we saw that it was going to be a bigger and bigger deal. And there were just very few people doing that at the time. And today, sort of continuing that with Asymmetric Security, where we have quite a distinctive thing we're doing in the space of cyber defense, and I think because we're taking AGI very seriously. One frame on what we're doing is we're assuming something like AGI will exist in the world, and then be like, okay, what makes most sense from a cyber defense perspective assuming that? And that takes you to quite a different place than lots of other security companies. And if we're wrong, the company is still valuable. It's way less valuable than it would otherwise be. And so it's these big costly bets on, yeah, this is a thing that's happening.

(06:37) Nathan Labenz:

And what does AGI mean to you? Because we've had multiple rounds now. Like Tyler Cowen at the o3 launch said, okay, I'm calling it. This is AGI. Friend of the show, Dean Ball, called it with Claude Code with Opus. And I have to say, it's certainly getting there. Right? Where do you think we are on the is AGI here or not? If not, what is missing in your perspective that you think is kind of going to tip us into a stranger future?

(07:08) Alexis Carlier:

Yeah. At least how I think about AGI, it's not here yet. I have in mind more the drop-in remote worker style version of things where you're really fully substituting for remote work that humans do, including for long-horizon types of tasks and can really fully sub in for humans. At the moment, we have this weird jagged frontier of capabilities where the models are geniuses at some things and then just cannot string together tasks over long enough horizons. And I think that sort of explains why we have this weird paradox of, on some dimensions, clearly they're superhuman, but then they clearly are not having that much economic output just yet. And so I think for me, we will see it in the economic output and GDP statistics and so forth when we really have this, like, you're actually subbing in for human work.

(07:54) Nathan Labenz:

I find the GDP question so confusing because so many of the most impressive moments that I've experienced with AI have been GDP destroying because it's when it's effectively substituted for something that I would have otherwise had to go out and hire done that I'm like, this is where I really feel the AGI. And so I have kind of a strange confusion, I guess, around more output. Sure, I would expect more services rendered. Right? More legal services, more medical second opinions. Obviously, lots more code, but potentially at a lot smaller prices than previously. So that's kind of a weird -- I don't know if you have a point of view on that, but I still kind of find myself just like, I don't know, the frontier model developers' revenue is exploding, but we don't quite see it in the GDP statistics. But I'm not even sure what I would expect to see. Like, in some ways, I might expect to see GDP shrink.

(08:53) Alexis Carlier:

That's interesting. Yeah. I suspect probably the more relevant thing to be tracking actually is just output. And there, I think very clearly, we should expect a lot more than we're seeing today.

(09:02) Nathan Labenz:

Yeah. It obviously becomes a little tricky. How do you measure output? Because dollars are the standard way, and it can be tricky. I guess just because I've been so deep down this rabbit hole, like so many, I don't know where your Claude Code or OpenCode or bot journey is at the moment, but I have been, like many, trying to figure out how I can create leverage for myself. I've done over time tons of task automation type things where I break down a task and make a little eval set and build a workflow. But that doesn't feel like AGI. That feels like task automation. Now I'm kind of like, oh, we're getting definitely a little closer with this Claude Code setup. Like, it can figure things out in really remarkable diversity of different situations. The one thing I have noted, and I'd be interested in your reflections on this, is -- and it feels like it's probably going to be pretty easily fixed -- but the one thing I've noticed recently is that Opus 4.5 in Claude Code has a strong tendency to try to write code with heuristics for things that I really just want it to spend tokens on to understand with its own fluid intelligence. So a random task was backfilling transcripts for the podcast onto the website because we didn't do that in the early days of the podcast. And I was like, well, it'd be really nice to go back and have transcripts for all those episodes. But so many times in the backfilling process, I see it write a little Python script with guesses about how things are going to be and then that doesn't quite work. There's too many edge cases, whatever. And I'm just like all-capping it: use your fluid intelligence. Just read the file. If you just read it, you'll know what to do. Don't write these kind of roundabout Python scripts. I definitely think that speaks to jaggedness and it also speaks to, in my mind, how subtle and honestly minor some of the major weaknesses still are. Like, it just has this strong impulse to try to code up heuristic-based guesses about how things should work when it's like, if you just read it, you're definitely smart enough, Claude, to know what to do. But it just has some wrong impulses in a few different ways. I mean, maybe we should save your response to that and contextualize it in what you're actually building. But if you have any immediate reactions, I'd be interested to hear them because I do -- I mean, Logan commenting that you're one of those AGI-pilled people out there, I think that is a strong statement, knowing who he spends his time with. So I am interested in your take on how you understand jaggedness and how soon it might be resolved.

(11:44) Alexis Carlier:

Yeah. I think my take here is roughly that RL is a big part of what's going on. With the pre-training paradigm, you were getting a lot of generality kind of for free because you're reading all the internet, and the models were just improving in this very predictable way. And I think a lot of what's happening now, and I think Karpathy and others have said this as much publicly, is because you're sort of relying on verifiable rewards, the areas where they're getting good are really kind of predictably things like coding and math. And so I actually suspect it'll be, until we have some big other paradigm, I don't think they're going to get that great at writing like poetry, for example, for quite a while. That's generally the high level.

(12:28) Nathan Labenz:

We'll put a pin in that. We'll come back to some of those related questions as we get deeper into the problems you're solving and the way you're solving them. I'd love to take a little survey of cybersecurity broadly and how AI is beginning to impact that and how you expect it to impact it more and more over time. Again, this is an area that really confuses me, so this is a great learning opportunity. On the one hand, I kind of work from the assumption that everything is hacked all the time and I've gotten so used to it that I would confess to not my key -- I've privileged passwords or passwords I don't reuse that are for key accounts like my Gmail and so on. And then I just have throwaway passwords that I use for long-tail stuff. I'm like, ah, if anybody compromises any of these services, I could probably live with that. And I've just gotten to the point where I click ignore when Chrome pops up that one of my passwords has been found in a breach. So that's the level of security I'm operating with. And I would have to assume that it puts me in pretty good company as being certainly exposed in some ways. And yet, life mostly goes on and not only does critical infrastructure continue to run, but even I, with my terrible practices, mostly don't have any problems. So I'm a little bit confused about the landscape today, including who are the attackers? What do they want? Who are they attacking? How are they attacking as kind of a baseline? And then, how is AI starting to change all that? I'd love to just get kind of a survey rundown from you.

(14:01) Alexis Carlier:

Sure. Yeah. I think broadly, the majority of attacks are by relatively unsophisticated amateurs or criminals. Maybe this is roughly 80%, something like that. And often, these are financially motivated. And then on the other end, you have these very sophisticated attacks from nation states. Like a small minority from a volume perspective, but often they're going after really critical IP or things like this. And so some of these low-sophistication groups will do things like spray and pray attacks, sending out phishing emails to as many targets as possible. There'll be script kiddies, so using malicious programs that others, more sophisticated folks, have come up with, but just sending them out en masse and hoping it works out. That's sort of one end. If you were being targeted yourself, that's probably what you would be seeing, probably because you don't have a ton of super sensitive IP or stuff that really sophisticated folks are going to come try to steal. And probably you're just like, you're not going to click the phishing email and you're fine. And then at the other end is the nation states, and here it's a very different ballgame. A few important actors here. You have China, a big one. Obviously, the CCP. A big focus over there is IP theft from R&D-heavy industries in the West. AI is almost definitely a huge focus now. Russia is another big player. They're interestingly a bit more focused on political disruption. So famously, there was the 2016 US election disruption. And another big one here is actually North Korea, which has very different motivations. Actually largely about funding the regime. And a big thing they do here is they have this North Korean remote worker program where they send literal North Korean operators to just get tech jobs as remote workers in foreign companies just to fund the regime. That's a pretty unusual one, but that happens a lot.

(15:45) Nathan Labenz:

I've heard a little bit about that, but I didn't realize it was like a --

(15:50) Alexis Carlier:

Yeah. It's a thing. We actually just had -- we just brought on someone from CrowdStrike who works some of these cases where the companies in question realized that something was up with their employee, and they were both sending back money and sending back various bits of IP.

(16:06) Nathan Labenz:

So the play is to get the job, get access to either information or some sort of financial tooling that they can then siphon money off. Exactly. Get the Brex card. Get the Ramp card ASAP. Yeah. That's fascinating. Okay. So we got China. We got North Korea. Russia, obviously, a player in this as well. Is there a middle? There's also kind of -- like, who is it when a hospital -- this recently happened to a medical system locally here. I went to see my primary care physician and he was like, oh, it's been a nightmare. Our stuff is totally inaccessible and there's some cyber gang somewhere that is demanding ransom or whatever, and they don't want to pay it. And so I'm like taking notes on paper. Who's that?

(16:57) Alexis Carlier:

Yeah. That's one of the most common types of attackers is ransomware. And these are often organized criminals who are moderately sophisticated and act in this coordinated way. In particular, what they tend to do is they find ways of getting access to systems, encrypt it all, and then just put up a notice on the screen being like, hey, if you want your data back, send us a bunch of money. This was a huge pop-up during COVID, actually. And this, economically, is one of the most damaging things. Relatively recently, Jaguar had a big ransomware case. And if I recall correctly, they needed a $2 billion loan from the UK government after this happened to avoid going under, and their operations were down for a month, two months, something like that. So very significant economically. Interestingly, the type of attack they're doing means they operate very differently than nation states, where the motivation of the ransomware folks is to be discovered as soon as possible once they've encrypted the data. The nation state, the more sophisticated guys on the other end, are just trying to stay stealthy for as long as possible. And so you end up with these very different modes of operation.

(18:07) Nathan Labenz:

Yeah. Okay. That's quite interesting. When you talk about a $2 billion loan, that makes me wonder how much of the -- in business, there's always this question of how much consumer surplus are you creating? How much of the value that you're creating are you, as a business, able to capture? How much of the damage, I guess, that the cybercriminals are causing do they manage to capture for themselves? Like, if the damage to Jaguar was X billion dollars, that doesn't mean they paid X billion dollars to the hackers. Right? Is it a 10 to 1 ratio, a 100 to 1 ratio? Do you have any sense for how much they're actually able to get from the damage they do?

(18:51) Alexis Carlier:

Yeah. I don't know the specific numbers, but it's incredibly negative sum. I think order of 10 to 1 sounds right. It's just incredibly economically damaging. And I've heard some economists claim that if you sort out this scourge of cybercrime, it's actually a huge effect on growth because it's just so value destroying.

(19:06) Nathan Labenz:

Hey. We'll continue our interview in a moment after a word from our sponsors. Are you interested in a career in AI policy research? If so, you should know that GovAI is hiring. 10 years ago, a small group of researchers made a bet that AI was going to change the world. That bet became GovAI, which is now one of the world's leading organizations studying how to manage the transition to advanced AI systems. GovAI advises governments and companies on how to address tough AI policy questions and produces groundbreaking AI research. GovAI is now hiring its next cohort of researchers to tackle hard problems that will define AI's role in society. The research scholar position is a one-year appointment for talented, ambitious individuals looking to transition into the field. And they're also hiring for research fellows, experienced researchers doing high-impact AI policy work. Past scholars and fellows have defined new research directions, published in leading media outlets and journals, done government secondments, gone on to work in leading AI labs, government agencies, and research groups, and even launched new organizations. Applications close on February 15, so hurry to governance.ai/opportunities. That's governance.ai/opportunities, or see the link in our show notes. Want to accelerate software development by 500%? Meet Blitzy, the only autonomous code generation platform with infinite code context. Purpose-built for large complex enterprise-scale codebases. While other AI coding tools provide snippets of code and struggle with context, Blitzy ingests millions of lines of code and orchestrates thousands of agents that reason for hours to map every line-level dependency. With a complete contextual understanding of your codebase, Blitzy is ready to be deployed at the beginning of every sprint, creating a bespoke agent plan and then autonomously generating enterprise-grade premium quality code grounded in a deep understanding of your existing codebase, services, and standards. Blitzy's orchestration layer of cooperative agents thinks for hours to days, autonomously planning, building, improving, and validating code. It executes spec and test driven development done at the speed of compute. The platform completes more than 80% of the work autonomously, typically weeks to months of work, while providing a clear action plan for the remaining human development. Used for both large-scale feature additions and modernization work, Blitzy is the secret weapon for Fortune 500 companies globally, unlocking 5x engineering velocity and delivering months of engineering work in a matter of days. You can hear directly about Blitzy from other Fortune 500 CTOs on the Modern CTO or CIO Classified podcasts or meet directly with the Blitzy team by visiting blitzy.com. That's blitzy.com. Schedule a meeting with their AI solutions consultants to discuss enabling an AI-native SDLC in your organization today.

(22:17) Nathan Labenz:

In terms of the baseline situation, of the attacks that are happening -- and maybe you could, I don't know if it makes more sense to segment this by volume, probably not, maybe more by damages to the degree that we can -- how many of these attacks are of the social engineering variety versus finding purely technical exploits where somebody left, I don't know, a port open or whatever the kind of common things are? And how many of them are like these real cinematic type of things where people are really figuring out exploits that nobody knows about? Obviously, that's got to be relatively rare. But how should we understand how much damage each of those kinds of things is doing?

(22:58) Alexis Carlier:

Yeah. I think a bit hard to say in terms of damages. A bit more clear on volume. So the majority is social engineering or phishing. Hard to say exactly, but say roughly 70 to 80%. The reason for this is kind of simple. Attackers choose the path of least resistance, and there's no reason to burn a valuable zero-day if you can just get away sending a convincing email. And so that's what most people do to start with. And in terms of the technical risk, most actually comes from known issues. So it's vulnerabilities that people are aware of and just most systems have not been patched. A large fraction of what happens is just people haven't done the basics. They haven't patched what's known. It's very solvable stuff. Zero-days, on the other hand, are a lot more rare and tend to be exploited by the most sophisticated actors, like nation states. But you hear a lot more about them than you would expect based on what you're actually seeing in the wild in terms of what attacks are going on all the time.

(23:53) Nathan Labenz:

So what works in terms of defense? Turn on your two-factor auth. How much does that get you? Update your operating system. How much does that get you? If I do those things, am I good? Or how much risk remains if I do kind of the basic common sense things?

(24:13) Alexis Carlier:

Yeah. It depends a lot on who -- like, good against who. For most people who are not the targets of nation-state attacks or aren't big corporations who are going to get ransomware, doing the basics like MFA, regularly updating your software, checking your supply chain of vendors, checking their security, having some sort of automated monitoring systems, doing occasional compromise assessments, so proactively looking into a system to see if it's been compromised. That will get you pretty far against lots of these amateurs and some organized crime as well. Frankly, on the other end, it's just extremely difficult to defend against nation states. It requires -- like, if you really want to do this in a very determined way, if you've got some IP that's super important, you need things like extremely strong limitations on software and hardware providers. You probably just couldn't use most SaaS, for example. You need to be super careful about who your employees are and vet them in a way that's just super outside the norm and plausibly illegal in some contexts. You need to be way, way more stringent on proactive security, like searching for zero-days in your system, doing compromise assessments all the time. It's just very, very difficult to do. And most organizations are just not there, which is why most nation states just have a lot of ongoing intelligence operations that are stealing IP all the time.

(25:32) Nathan Labenz:

I've been advised -- I didn't make it this past summer, but I want to go to China and participate in some form of inter-civilizational AI safety dialogue and hopefully mutual understanding building. And when I was thinking about going this past summer, people advised me, okay, you can't take your devices. You have to get burner phones. The second you get back, you've got to throw that phone away or you could maybe sell it on or something. But absolutely, don't take your devices. Don't log in to anything while you're there. Take all these extraordinary measures. Do you think that's necessary if I am just a regular American kind of AI yapper taking a trip to China?

(26:16) Alexis Carlier:

Certainly, if you didn't do it, you would probably get people reading over your stuff.

(26:21) Nathan Labenz:

And by that, do you mean they demand my phone at the airport and take a look at it? Or that they have a way of getting into an Apple device and gaining access?

(26:32) Alexis Carlier:

The latter. Yeah. You should assume if you do that that people are monitoring your communications in an ongoing way. And maybe that's fine. For normal people, maybe that's often fine. I suppose most people would not love the idea of the CCP reading all your stuff. But yeah, I think people are generally right that if you go to China, you should expect stuff like that to happen, especially someone like you with a public profile and so on.

(26:51) Nathan Labenz:

And that would continue -- like, just to understand the architecture of the surveillance, it would be something they would be putting on the device such that when I take the device home, that comes with me, not something that because I'm on the local network there, they would have access to only while I'm there. Is that the correct understanding?

(27:09) Alexis Carlier:

Yeah, you should assume they will have some way of getting a persistence mechanism. This might be a physical thing or it just might be they've got access to your login credentials or something like this, but you should assume they'll have persistence.

(27:23) Nathan Labenz:

So that's a little bit like, how should I sort of reconcile the idea that I'm not that important for one thing with the idea that there aren't that many zero-days to be burned? Presumably, Apple -- I think I should trust in terms of they've got a big team of people that are working on this all the time, right, and identifying these things and closing them down. So how is it that they have so many of these mechanisms that they would use one? I can't be that high on their power rankings of targets. Right? So are there just a lot more zero-days than I have been led to believe, or how do I resolve that seeming tension where I just don't feel like I should be worth using these things? Because presumably, that gives Apple some ability to figure out what's going on and close it down each time they use it. Right?

(28:16) Alexis Carlier:

Yeah. Apple's security is generally pretty good. I think broadly what's going on here is just it's extremely difficult to get visibility on what nation states actually have access to from an offensive security perspective. And the amount of effort in terms of people going into this is estimated, I think, in the hundreds of thousands for the CCP. So there's just a lot of man-hours going into finding zero-days and finding ways to break into systems. And often, there are many cases where access that nation states have had wasn't discovered publicly until decades later. And so we should just expect that there's a lot of forms of access that very sophisticated organizations like these have and are using all the time. And yeah, it is the case that for most normal people, you won't be high on the priority list. And so they're not likely to spend these zero-days if they expect to be discovered. But certainly, it's within the realm of capability. And in many cases, they probably can just get away with stuff without anyone knowing for a very long time.

(29:18) Nathan Labenz:

So how do these things tend to come to light? This starts to get a little bit into what you're doing. So I don't know if it makes sense to talk about how people discover things first or maybe just talk about how the landscape is changing in light of AI. Because I think pretty much everything we've talked about so far has been baseline, like, you could have said all the same things pre-GPT-4. Right? So yeah, maybe what's changed since GPT-4 class models have come on the scene? How has this landscape started to shift, if at all, in a meaningful way? Again, it doesn't feel like it's changed my life yet, but everybody's telling me it's going to.

(29:59) Alexis Carlier:

Yeah. I think for the most part, it hasn't actually had a huge effect just yet. Obviously, you could automate phishing emails and things like this, but there hasn't been this really big uplift from an offensive perspective yet. I think that's about to change. It sounds like on most measurements that the labs and others are doing, the models are already on the precipice of these offensive security capabilities. And broadly, I think the way to think about the effect here is like bringing down the threshold of sophistication necessary for any given kind of attack. So attacks that are pretty moderately sophisticated, like ransomware -- assume that a lot more people will be able to do these all of a sudden. That's not coming online yet, but will be coming on soon. And from a defensive perspective, I think it also hasn't been super transformative yet. And again, I think probably about to change, and that's part of what we're building at Asymmetric.

(30:51) Nathan Labenz:

So how do we measure these capabilities? It's one of these weird things. I mean, I guess this is becoming increasingly common, right, where it's like across the board, when the tasks were easy and anybody could tell if the AI was doing a good or bad job, things were pretty easy. We're now in so many domains in a world where, because the performance is so good, very few people can even really critique it. I'm staring down the barrel of an interview next week with the founders of Harmonic, who created this Aristotle model system that got the IMO gold alongside, obviously, OpenAI and DeepMind. And it's like, damn, I would score zero on that test. So how do I even understand what's going on? I basically feel the same way in cybersecurity, and it feels like there's a very limited number of data points we can get to measure how good these things are when -- I know there are some things like, well, there's been some zero-days since the training cutoff, so we can see if they can do those. But my sense is those are pretty small number. So how do we even get a handle on what the model capabilities are in a way that we can trust or base our plans on?

(32:13) Alexis Carlier:

Yeah. I think this is super tricky in cyber for a reason that you were sort of gesturing at. One big thing going on is unlike, say, software engineering where you have a ton of code on the internet, most cybersecurity stuff is just not public. In the context of incident response, for example, when a company gets hacked, they're not going to share the logs from the email account getting compromised publicly. And so it's very tricky to actually get a sense of how exactly the models are performing in these contexts because you've got nothing to benchmark on. And I've talked to some of the folks at the frontier labs. This is a big bottleneck right now on trying to evaluate because you just have nothing to benchmark on. You're not seeing all the attacks that are happening out there. So this is a big problem. And secondly, the models are getting sufficiently good that you do need a bunch of subject matter experts who themselves are very good at this. And there just aren't that many such people. CrowdStrike, one of the top incident response companies in the world, they have a team of roughly 60 people. It's not huge -- who can do this really deep forensic defense cybersecurity investigations. So you're just really constrained on talent and really constrained on data. And it does make it very hard to know what's going on by default, I think. I think there are ways to solve this, but by default, things are tricky.

(33:27) Nathan Labenz:

Hey. We'll continue our interview in a moment after a word from our sponsors. Your IT team wastes half their day on repetitive tickets. And the more your business grows, the more requests pile up. Password resets, access requests, onboarding, all pulling them away from meaningful work. With Servo, you can cut help desk tickets by more than 50%. While legacy players are bolting AI onto decades-old systems, Servo was built for AI agents from the ground up. Your IT team describes what they need in plain English and Servo AI generates production-ready automations instantly. Here's the transformation. A manager onboards a new hire. The old process takes hours. Pinging Slack, emailing IT, waiting on approvals. New hires sit around for days. With Servo, the manager asks to onboard someone in Slack and the AI provisions access to everything automatically in seconds with the necessary approvals. IT never touches it. Many companies automate over 50% of tickets immediately after setup and Servo guarantees 50% help desk automation by week 4 of your free pilot. As someone who does AI consulting for a number of different companies, I've seen firsthand how painful manual provisioning can be. It often takes a week or more before I can start actual work. If only the companies I work with were using Servo, I'd be productive from day 1. Servo powers the fastest growing companies in the world like Perplexity, Verkada, Merkor, and Clay. So get your team out of the help desk and back to the work they enjoy. Book your free pilot at servo.com/cognitive. That's servo.com/cognitive.

(35:09) Nathan Labenz:

The worst thing about automation is how often it breaks. You build a structured workflow, carefully map every field from step to step, and it works in testing. But when real data hits or something unexpected happens, the whole thing fails. What started as a time saver is now a fire you have to put out. Tasklet is different. It's an AI agent that runs 24/7. Just describe what you want in plain English. Send a daily briefing, triage support emails, or update your CRM. And whatever it is, Tasklet figures out how to make it happen. Tasklet connects to more than 3,000 business tools out of the box, plus any API or MCP server. It can even use a computer to handle anything that can't be done programmatically. Unlike ChatGPT, Tasklet actually does the work for you. And unlike traditional automation software, it just works. No flowcharts, no tedious setup, no knowledge silos where only one person understands how it works. Listen to my full interview with Tasklet founder and CEO Andrew Lee. Try Tasklet for free at tasklet.ai, and use code cogrev to get 50% off your first month of any paid plan. That's code cogrev at tasklet.ai.

(36:21) Nathan Labenz:

So I appreciate you taking a long time here to just give me the kind of baseline lay of the land. How are you going to make this better for us so we don't have to worry about this? And maybe you could start with a little bit of what does it look like today when a company starts to get the inkling that they have been pwned? How does that come to light? What do they do today? And obviously, you have a different vision for how that can work in the future, and I think being more thorough, proactive, playing to AI strengths are a big part of that. But take us through the kind of before and after of how things are and how you hope you can change them to be.

(36:58) Alexis Carlier:

Yeah. So the problem we're trying to solve basically is make it much, much easier and much more accurate to detect when you've been breached and have that happen as quickly as possible. Today, roughly speaking, you have these monitoring systems, detection systems, that are based on static rules, basically software that hardcodes, oh, this thing is maybe suspicious. And this kind of works a little bit. It does do something. But you end up in a situation where you get a lot of false positives, a lot of alerts going all the time. And the reason for this basically is that a lot of activity that could be suspicious is also just could be normal. So if you're logging in from different places or devices, yeah, that is maybe weird, but also maybe you're just traveling to Tibet this week. That happens. Maybe you're just up late in the middle of the night, and that looks suspicious, but it also could be normal. Maybe you're downloading files in bulk. Again, sometimes you just download files in bulk. And without the ability to reason over the forensic evidence in a lot more detail, you just have no way of distinguishing between these things. Also, by the way, on the other hand, there's this genuinely suspicious behavior that looks normal. So if you were trying to exfiltrate data, you might just, over the course of months, exfiltrate very small amounts, which are just not very noticeable. And then typical detection systems at the moment won't pick up on it at all. And again, if you had something like a human who suspected there was something going on here and had the time and energy to reason over all this evidence very deeply, you'd probably be able to figure it out. But that's not how the stuff works right now. And so generally, status quo, if you have some alert go on in, say, an enterprise, the security operation center that handles all this stuff will do triage. So an initial, basically rapid assessment to figure out when an alert fires fast and quickly. Is this something worth prioritizing? Is this a real alert? Is it a false positive? Can it be ignored? And then in the minority of cases where they're like, yeah, this is actually a suspicious thing, let's pull it out and do digital forensics, which is this methodology of doing very deep security investigations, where you're trying to figure out in a lot of depth what exactly happened, looking at every evidence source you need to figure out all the details. Intuitively, it's just like a detective, a forensic detective, and you're reasoning over all the evidence you need in this sort of judgment-based way. So that's the status quo. I think what will happen -- you can think in principle, what would be ideal here? We know that digital forensics is the best way of doing this if you had sufficient time and sufficient energy and sufficient labor to do this. The reason you can't right now is because you're just bottlenecked on people. There are just very few people with this expertise. They're very expensive, and they work slowly. And it would just be impossibly expensive to do this. And so at the moment, it's this thing that's mostly used reactively after a breach. That, I think, will not stay the case going forward. One more general intuition here. I think a lot of people, when they think about what to do with AI, like you're building AI agents, what should you focus on, have this frame of, oh, this is a human workflow that's really important in the world. And what if I could just automate that work with AI? And that can work. Obviously, you're automating legal work. You can do stuff like this. But I think a better question is, what would you do if you had 100x the intelligent labor at your disposal, something like that? And I think the answer is that for many kinds of work, you can perform work that was previously too slow, too expensive. You can perform it at scale now because of the change in economics. And that's the bet we're making in the context of defensive security. This domain that is currently super accurate and super costly, only really applied reactively. You could just imagine, eventually at the limit, getting this done to the cost of compute, and you just have these sort of close to continuous proactive assessments of systems in a very deep way. And that's what we think will happen eventually. The question is how quickly can we get there, and can we get there soon enough that we can deal with a lot of the security risks that are going to come on with AGI and offensive security changing and that kind of thing.

(41:13) Nathan Labenz:

Yeah. I like the frame of, okay, you could take -- usually, for guidance, I usually say you can probably save 90% time and money on some task that's being done by humans if you automate it with AI. But yeah, I think you're totally right to emphasize that the other side of that coin is what can you scale that was just previously impossible to scale?

(41:42) Alexis Carlier:

And this is an interesting one.

(41:44) Nathan Labenz:

Can you describe a little bit more what the humans do? So let's say I'm breached -- maybe set the scene in terms of who your ideal customer profile is and how it comes to their attention that they've got a problem and then who they go to and how long it takes these people to engage. And then if I just were to sit there and watch what they do, what are they doing? Are they just grepping through logs kind of using the intuition they've built up over time? Or what is that sort of story? And then, again, we'll switch to the future state story next.

(42:21) Alexis Carlier:

Yeah. So to make this concrete, we're currently focused on what's called business email compromise. More intuitively, it's just email-based cyber attacks. And broadly, what this is -- the common modality here is an attacker somehow got access to an email account. Maybe they've sent out a phishing email that someone's clicked on. Maybe they've got access to your credentials some other way, and they're trying to use that access to trick people into doing stuff like sending them information or sending them funds. So this is often financially motivated, but can also be quite different. Like when the DNC was hacked by Russia in 2016, that was an email-based attack. And so at Asymmetric, we come in and we respond to these attacks as do other cybersecurity folks. And what this generally looks like is, okay, you come in and you pull down all the email logs from an environment. The enterprise has been hacked. They might be using Microsoft. They might be using Google. And you look through the logs. And first of all, you're trying to figure out, how did they get access? When did they first get access? Did they then escalate their access in various ways? Did they get access to another account, something like that? And very concretely, it's looking through email logs and trying to be like, okay, this is a weird location to be logging in from. And maybe the user was in two places at the same time. That's impossible. So this suggests something is suspicious here. So you figure out how they got in. Then you figure, okay, what did they do? You're again looking through the email logs. Did they send emails? Did they read them? Did they delete them? Often, people will set up automatic inbox rules, so automatically forwarding emails or automatically deleting inbound emails. Are they looking at files in the Google Drive? So you're painting this very comprehensive picture of what happened and also whether the attacker is still in the network. And then finally, you're looking at, okay, how did this happen? Was it a phishing email? And at this point, you also sometimes look at the email bodies themselves. And so you have a pretty clear sequence of things you're trying to do, but you're doing this in this sort of very flexible human way of, like, okay, this looks kind of suspicious. This looks funny. Let me pivot off this bit of evidence. Let me think what comes next. And so it's been hard up until very recently to actually automate this in a meaningful way.

(44:33) Nathan Labenz:

How much does access to -- like, if you're running your own enterprise email server versus you're a Google customer and your employees have Gmail, how much does that change how this goes down? If you're a Gmail customer, you obviously, or I would presume, you can't just get that level of access. Right? So do they have a team that you work with to try to resolve that sort of stuff? Or how does it work when you've got a SaaS provider like Gmail?

(45:06) Alexis Carlier:

So most tenants, like a user tenant, will have a bunch of logs that you just use. So actually, they do have all the logs you need. You just get admin privilege to those accounts, and you can go from there. The provider does actually matter a lot. So for example, there are far fewer Google compromises of emails than Microsoft, at least an order of magnitude. So big differences based on how they've configured the environment.

(45:29) Nathan Labenz:

Do we know what causes that? If it's that big of a difference, why has Microsoft not managed to close that gap?

(45:36) Alexis Carlier:

Yeah. I can't recall the specific reason, but it is this very tractable thing they could be doing that they just have not been doing for reasons that are kind of baffling to me. I guess it just doesn't really affect them in this big way. But yeah, I know some folks who have considered working in this space and building various bits of tech to help deal with these attacks, and they were like, no, probably we shouldn't do this because Microsoft's going to solve it one day. But yeah, up until now, it's not been the case.

(46:05) Nathan Labenz:

Yeah. That's funny. Okay. So how do we get AI to do this? And where are we right now in terms of how good is Claude Code? Where does it fall short? And what are you building to make sure it actually works?

(46:21) Alexis Carlier:

Yeah. So I can tell you a bit about what we're building in this context. We are a full-stack AI digital forensics and incident response company where the mission is to accelerate AI cyber defense, in particular by trying to automate this field of digital forensics as quickly as possible. And we're doing this broadly by having these human-AI teams doing these kinds of investigations. And so we're currently focused on these email investigations. And concretely, what this looks like is we go in and do the kind of things I was just talking to you about, but on this AI platform we've built that ingests all the logs. Broadly, we have an agent that will do a first-pass analysis over these logs. And then the human investigators will click through and basically try to do something like a QC most of the time to just check the quality of the reasoning. And in a minority of cases, they do need to meaningfully change what's been done. And so based on what we're seeing in these initial cases -- and for context, these email investigations are on the much simpler end of investigations -- the models are pretty good already. Even without doing anything fancy, without training on specific data, without anything like this, you can get maybe something like 90% accuracy, something of that order of magnitude out of the box. That is super helpful for speeding up the process of this investigation. It's totally insufficient for actually automating this work. This is a context in which you need very high accuracy for various reasons, which means you do completely need the humans, and it's also an area where the nines of reliability just matter a lot. So for the foreseeable future, and certainly as we get to more complicated kinds of investigations, it does feel to me like this would be an area where there's this long tail of needing humans to be around to really push up the nines.

(48:04) Nathan Labenz:

Okay. That's really an interesting kind of production possibility frontier that I'd like to understand better. But just as a little context, I think that this is going to play out, I suspect, in very different ways in different domains. And I'm always kind of like, well, geez, if AI can do 90% of the work, how much does that leave for humans? Is it 10% or is it still like, can they finish it in 10% of the time that they would have had to spend, or do they still have to spend half the time that they would have had to spend even though in some sense 90% is done? You can imagine that last 10% taking longer. And then of course, there's also the question of how many nines do you really need or can you even measure in a given context. And then there's the question on top of that of how much latent demand is there for different kinds of services depending on how much the overall price can come down based on how much can be automated and how much more productive the humans can be and whether or not they can hit key reliability thresholds. So my usual kind of somewhat tongue-in-cheek way of saying this is like, on the one end, we have dentistry where I do not want any more dentistry no matter how cheap it gets. I want the minimum, ideally zero, and it's never something I want, and making it cheaper doesn't really entice me. On the far end, you might put something like massages where I would potentially get a massage every day if it was close to free. I have limited -- I mean, I guess one of the big bets you have is that there is a ton more demand for this than is currently served. How would you think about the thresholds that matter in terms of the ratio of human productivity that would then enable, subject to certain reliability thresholds, the vast explosion of this market that you're obviously trying to unlock?

(50:11) Alexis Carlier:

Yeah. So I think a couple of ways to think about it. From the perspective of just applied reactively in the context of incident response, that I think is not of the form that demand will continuously grow here. Or rather, it's not of the form there is a bunch of latent demand. I do think demand will grow because there'll be a bunch of AI-enabled cyber attacks, but it's not of the form people, if they could have more of this, they would want more. It's just like, after you've been hacked, you're bleeding, you need to deal with it. That's basically the demand. On the other hand, if you get to this paradigm where you're using the sort of investigative reasoning but applying it proactively, then I think it sort of ends up substituting for the current approaches to doing detection in cybersecurity, which are not at all based on this digital forensics type approach. And it's a much, much bigger market on the order of a few hundred billion, something like this, maybe $500 billion. And so I think as a bottom line here, at minimum, you're substituting for this existing huge market of detection. Then there's an additional question here that I think is sort of an open question of if it becomes really cheap to get high levels of security, much better than what you currently have in detection, what is the demand for that? That, I think, is more uncertain and depends probably a lot on the type of actor. There'd be some organizations where it's just, you're getting what you need from security, and it's a meet-the-minimum-bar thing. Then there'd be other organizations, say like AI labs who are trying to protect really sensitive IP or national security agencies and governments who have this much higher demand for very high levels of security. And I think those are the areas we should expect this additional demand above and beyond just substituting for the current approaches to detection.

(51:53) Nathan Labenz:

So how do we get there?

(51:57) Alexis Carlier:

It's a great question. So this actually relates to something you brought up before around the difficulty of measurement in this space. And I mentioned these two bottlenecks around actually having access to data or context on what's actually happening, what these incidents, what these cyber attacks look like, and then having the people that can actually assess performance. At the moment, the status quo is that it's very difficult, and we've talked to many of the folks at the frontier companies. It's very difficult to make the models better in these situations because you don't have these two things. It's hard to build verifiable rewards for reinforcement learning without having a lot more color on what's actually going on on the ground in cyber and without having people who can assess these things. And so I think the key to really accelerating progress in this space is solving both of those two things. And I think broadly, the approach we are taking to solve this is actually having this whole cybersecurity team doing these investigations all the time. So we've just hired folks from CrowdStrike, from Palo Alto Networks, from most of the biggest cybersecurity companies whose day job is really just to do these investigations with the AI tooling. And as they're doing this, they're implicitly evaluating the model performance based on the use of the tooling. And then secondly, I think the tricky thing is you need to really just be seeing a lot of the different types of cyber attacks that are happening. And off the back of that, building evaluations that are really high fidelity, really realistic, building environments that are really realistic and that you can use to train on. And I think this just doesn't happen by default. And by default, the model providers, I think, will lag on these dimensions. None of the big cyber companies actually are pushing on this either. They don't have a huge immediate incentive to do this. So unfortunately, I think the default is just this stuff lags behind in a way that doesn't apply as much to offensive security, by the way, because from an offensive perspective, you can just try to hack something. You can just do that all the time. You don't need any sensitive data. So there is this asymmetry -- exactly. There's this unfortunate situation where a lot of bottlenecks apply to improving the defensive side, a lot less apply to the offensive side. And I think what the space needs is basically a lot more companies trying to solve this data verifiability evaluation problem. We're doing one attempt, but I would love there to be many more.

(54:22) Nathan Labenz:

So how do you position yourself in the market to -- because I can sort of see if you become the go-to company that everybody knows is going to do the best job with this, then as we see in --

(54:36) Alexis Carlier:

I mean, this is kind --

(54:37) Nathan Labenz:

-- of a general phenomenon in the AI space. Right? Like, the more of the share of the business you win, then you have more access to data and you get kind of a positive flywheel going. So I can see how that dynamic could work once the flywheel starts to turn. But it does strike me that it's probably a pretty hard market to enter. So how are you thinking about entering it as a young company that obviously doesn't have the track record? The sort of the old adage of nobody got fired for going with IBM. Right? I assume there's probably something similar in cybersecurity where it's like, well, nobody got fired for going with CrowdStrike. How do you overcome the just relative unknown factor and win business so that you can start to accumulate this data and get that flywheel turning?

(55:27) Alexis Carlier:

Yeah. The flywheel is, I think, actually especially interesting in cyber. And CrowdStrike is an interesting example where about 15 years ago whenever they got started, they actually also started as an incident response service provider initially before they productized a couple of years later. And so for the first two years, they were just doing this kind of services. The reason for that was they could solve this problem of understanding what cyber attackers are doing all the time and collect this sort of data. And secondly, build these relationships with the enterprises who they were serving. In cybersecurity, trust is really important for distribution because it's hard to assess how all the tools are working. For most, like, if you're in a bit of HR software, you can just tell, does the thing pay my employees on time? In cybersecurity, you're sort of taking it on trust. Like, yeah, is this thing stopping attacks? Am I not attacked, or do I just not know about it? So trust is a big thing. And so they figured out that there was this really strong flywheel both from data and distribution from doing services. And this has remained the case since then. I think what's different now is that quite recently, it's become possible to get AI-enabled services to work pretty well and get these much better margins than you would otherwise. And I think this is what's creating this new opportunity in the space. So we are initially focusing on these email-based attacks where we have built out this platform that basically takes the time required to respond to these attacks down from, depends on the firm, but on the order of two days to a week for these investigations, to a few hours. So a very meaningful difference here. And this means that we can just do these things much quicker and much more cheaply, and that is proving sufficiently compelling to get a bunch of folks to trust us. And from that initial trust, it'll be much easier to get a lot of different types of cases here. So I think the opportunity is coming, again, available because of this technological shift where in this context, incumbents are just not adopting as quickly as startups.

(57:32) Nathan Labenz:

And is that just the purely sociological phenomenon that we see in many places of incentives aren't quite there, who wants to take a risk, maybe they're billing by the hour or whatever in the first place?

(57:46) Alexis Carlier:

There's some of all of that. Billing by the hour is part of it. Another piece that's more subtle is on the face of it, it doesn't look necessarily like this massive opportunity. It's a pretty big market. If I recall correctly, it's like a $40 billion market of this sort of services. But from the perspective of, say, CrowdStrike, who is close to a $200 billion company, I think, and the majority of their revenues are coming from their detection products, which are not these sort of digital forensics types of services, it's relatively small fish. And so it only becomes this very compelling, comparatively compelling thing to this other big pie if you have this view that, yeah, okay, we're actually going to get to the point where we're meaningfully automating all this field. And then this will change actually not just how these forensic investigations are working, but how detection is working. And that requires you to really take AGI seriously and to really be like, yeah, okay, this work that at the moment is just entirely humans is, in pretty short order, going to be automated in a meaningful way and will change how all these other bits of cybersecurity work. And most of the players in the space just don't have that belief. In general, and I've spoken to some of the folks who I know who work at the intersection of AI and security, the security people generally don't -- they're, by nature, skeptical people, and they don't really believe in AI in this big way. And so I think part of the opportunity is just like, do you really believe in AGI and have thought through the implications of that? I think most of the big players haven't.

(59:18) Nathan Labenz:

Yeah. Fascinating. Okay. Can you tell a little bit more about the jaggedness of models? And I guess I'm also interested in -- of course, we've got the age-old debate of proprietary API models, which I would presume are the best at these tasks off the shelf. But then you've also got, if you're using open weights, you have the ability, of course, to fine-tune or modify however you might like. What's the mix? What's the model mix look like and how much do you think that this is going to be -- how much value is in the harness versus how much is in the training data that you can create? And how much of -- I guess there's also another question of training data can be monetized or can be commercialized in multiple ways. One is you could sell it back to the model providers. And I think that's, from what I hear, increasingly a very good business. Or you could try to make your own models that outcompete them, so you have that asset. How would you describe what's working today and what is your strategy to not get steamrolled by just better and better frontier models kind of winning everything?

(1:00:34) Alexis Carlier:

Yeah. On the jaggedness, I think there is actually this sort of underrated opportunity here to defensively and intentionally accelerate capabilities in various ways. So one intuition pump here: in the context of alignment, for a long time, AI alignment people have talked about the automated AI researcher being one important part of the way we're going to solve alignment. There hasn't really been similar analogous things in other fields. And I think part of the reason was just with pre-training and this very strong generalization, it was hard to see how you could accelerate in this intentional way rather than just across the board. That, I think, with the jaggedness we're seeing, is no longer the case. And the implication here, I think, is that actually you can go out and pick various areas that you think it might be important to harden the world, like defensive cybersecurity or biodefense or bits of AI safety, for example. And you could very intentionally curate the datasets, the environments, the evals that you need to sort of pull out the jagged frontier in this specific direction. And people don't seem to talk about this very much. I think this is a mistake. It seems like there's a really huge opportunity. I mean, for many of the same reasons that AI is going to be so transformative in the first place. If you're a subject matter expert in one of these areas, and instead of doing bits of work yourself, you can just put this expertise into the AI systems and then scale it immensely more. That's an incredible thing you could be doing. And yeah, it strikes me that very few people are doing this. A few reasons. I think in some contexts, in the context of cybersecurity, for example, people often have this idea that actually it's just inherently dual use. That's, I think, partly true, but also overstated. I saw Sam Altman tweet this the other day. There are some areas like pentesting and vulnerability discovery that are, in fact, dual use. You can patch holes or you can exploit them. But cyber is a diverse field, and this is not true of all areas in cybersecurity. And so digital forensics, for example, I think, is very much not like this, where fundamentally what you're doing is trying to detect where stuff has broken into a system. You're asking, is there evidence of attackers here rather than can I break into this? And so that's just a defensive application. And I think there are other areas of cybersecurity too that have this feature. And so generally, I think it'd be good if there was much more of this kind of stuff out there. On your other question around the models we're using, how much is the scaffold, all that kind of stuff. Generally, the way I think about this, at least from how we as a company should be going about this: first, we should just see what we can get the low-hanging fruit out of the box. Pick the low-hanging fruit, test the different models, see how they go, build minimal scaffold, see where the performance is, make sure we have sophisticated evals that are working well here, and that we can really tell what's working. And then only then consider doing some more fancy stuff, like training our own models or things like this. At the moment, we are at the eval stage of that. And on these simple incidents that we're currently working on, just models out of the box with some scaffold is actually getting pretty far. Again, we're not so far down the nines of reliability, but it's actually unclear how much more effort it's worth from us to get them good at these specific email-based attacks. Going forward in these other kinds of incidents that are way more open-ended, long-horizon, I'd be pretty shocked if the models were as good or anywhere near as good. And I think that's an area where all this differentiated access to data that we use as blueprints for evals and the reasoning traces of analysts using the platform -- I think that will become this very differentiated and valuable asset.

(1:04:19) Nathan Labenz:

And do you have an intuition at this point as to whether you license that data back to model developers or keep it for yourself to try to make your own specialist model?

(1:04:33) Alexis Carlier:

Yeah. Both are possible. And you could also imagine doing this sort of iteratively where you have some stuff that you've done in the past that you're then sharing. I think the benefits of sharing, at least from a mission perspective, are pretty significant in that if you're getting the foundation models better at these tasks, you then get this immediate insane distribution where you're deployed across huge parts of the world. You're naturally deployed across AI labs, across governments, some of the most important areas from a security perspective. And also, yeah, they will be big contracts. And relative to other data providers, just the fact we're seeing all the cybersecurity incidents all the time does make it quite differentiated. On the other hand, is it giving away some sort of proprietary thing? Possibly. But yeah, I think it's sort of an open question, but there are certainly big benefits to sharing. Not the data per se, I would say, because that's delicate, but sharing the sort of the evaluations drawing on that.

(1:05:30) Nathan Labenz:

Going back to that question of generalization, I mean, I do think the jagged frontier is obviously a major factor affecting all of us that are trying to use AI to do stuff on a daily basis. And the question of how well RL generalizes seems like a pretty big question for what the next couple of years are going to look like. What would you say is the case for your specialists? Are they -- I mean, I definitely take the point that it's very different to grep through logs versus try to break in in the first place. But I then would also kind of guess that the people that you have doing the incident response would probably be very good at breaking in. Right? Like, they would know all the tricks of how to make a phishing email compelling or whatever. It would seem that if you had trained as a human at least on the response, you would be maybe not the most elite on the attack side, but I would expect significant generalization. So how do you see that playing out at the human level? And is there not reason to expect that a similar kind of generalization could happen for models?

(1:06:48) Alexis Carlier:

Yeah. So at the human level, actually, surprisingly, this is not the case, where the digital forensics field is actually a pretty distinct thing within security, and the people who do it have a very different community to the offensive white-hat hackers, things like this. And so at least at the human level, there's not a ton of generalization, interestingly. And I suspect there is just actually relatively limited overlap in terms of the skill sets here. And I think this would also apply to models. Again, of course, at some point, we're going to get continual learning, and the models will just get good across the board very quickly. But until we're in that regime, I think there is this opportunity.

(1:07:29) Nathan Labenz:

Okay. Cool. That's quite interesting. Do you have other things in mind that you think are like that? We can get out of our domain pretty quick here, but is there a biosecurity analogy? Is there something that we could be differentially accelerating in biosecurity hardening that wouldn't lead to the same kind of advances in the offensive skill set, or is this perhaps something that's relatively idiosyncratic just to the way the cyber world happens to be structured?

(1:08:05) Alexis Carlier:

Yeah. I will caveat that I don't know bio very well, but you could imagine, obviously you need robotics for this, but just producing a bunch of masks and having those ready to go. Stockpiling masks. Stockpiling vaccines. Certainly, there seem like there are at least some -- like the mask example. I don't know to the extent to which the biological research is dual use. And then, what are other areas? I would love, by the way, for someone to just go through and catalog a bunch of important domains and try to figure this out. I could imagine in the context of AI safety, maybe there are specific things here. I'm not sure. But it seems like an important thing for folks to go figure out.

(1:08:45) Nathan Labenz:

So I recently did an episode with a couple pioneers in the formal methods space. And obviously, there's a general agreement that cybersecurity could become a big problem in the AI future. And then there's several flavors of ways that we might respond to this. The formal methods angle -- I mean, it doesn't really necessarily address email-based hacks, but you could imagine a lot of things could get a lot better if the rate of vulnerabilities is just dramatically reduced based on the vision that I understand that they have, which is using formal methods as a reward signal to train coding models such that it's not just that you're taking output from today's models and trying to verify them, but you're also really closing that loop and getting to the point where models should be writing superhumanly secure code by default in a super majority of cases. That seems very plausible that we will get there because it seems like the viability of the formal methods -- I mean, that's obviously its core strength. Right? So that flywheel should be pretty easy to get turning. How do you think about where you want to be in the cybersecurity -- if it's a defense in depth type of thing, right, where you can harden your outermost defenses and then you can have better forensics when things do get through. In the extreme limit, if the outer defenses are totally impenetrable, then the forensics becomes less valuable or, in the extreme limit, unnecessary. How do you think about that? Is that just never going to happen in your mind?

(1:10:44) Alexis Carlier:

Yeah. I think more hardening would be great. People should totally do that. But I'd be very surprised if it would be sufficient. A few reasons. One is just for very high levels of defense against, say, nation states, it's just extremely difficult to defend. And it's not just like having no code vulnerabilities is sufficient. There are other ways you can get in. And the strong assumption should be stuff is going to get in. Additionally, regarding these other forms of ways of getting in, generally to be very, very secure, you need to do these things that trade off with your productivity as an organization. So say you're an AI lab and you have this way of interacting with the model weights that restricts the output. That's kind of annoying, but it does protect against weight exfiltration and things like this. And so in general, you do have this trade-off with productivity that is hard to get around in many areas of security. And so that's the thing that's generally going to stay there. A nice feature of automated forensics or detection is that it doesn't really have this feature. You can just sort of run it in the background, and it does increase your defensiveness. And so generally, I think of these things as substitutes. It would be great if we could get to this full, in theory, full hardening. I don't think that's at all realistic. And so in general, the more of each you can get, the better.

(1:12:07) Nathan Labenz:

So wait. Substitutes or complements? I guess maybe both.

(1:12:12) Alexis Carlier:

They substitute in the sense that if you have better detection, you need less hardening for a given level of security. But also having both together makes you more secure.

(1:12:22) Nathan Labenz:

Is there -- I've not actually done an episode with these guys yet, but there's a company called Jericho Security.

(1:12:28) Nathan Labenz:

I just --

(1:12:29) Nathan Labenz:

-- talked to them offline a while back. But I have their website up. Next-generational security training trusted by the US government. Smarter cybersecurity training starts here. Jericho Security is designed to protect you against today's most advanced email, SMS, and deepfake threats. And basically, what they do is automate the phishing attacks, more or less the spear phishing attacks, to see who in your organization is clicking on these links and make sure everybody's on their toes. And I don't know that there's -- when I spoke to them, there wasn't a deep forensics component. When you click on the link, they take you to a sort of "we got you" page that's like, hey, wise up, dude. You just clicked on a bad link and here's how you should have known, and how you can know next time. But I could imagine that in terms of environment creation or something like that, there could be some interesting collaborations between a company that is systematically testing for and finding the soft spots in the human defenses or lack thereof and what you guys are doing under the hood. Are there other -- how do you go to market? I mean, do you have partnerships or people that bring you in? Are there other alliances that you have that allow you to get into the room? Because obviously, people need to -- this is crisis stuff. Right? So they need to either find you real quick when they're searching or they kind of need to have known you or they need somebody to give a trusted referral. What does that look like?

(1:14:06) Alexis Carlier:

Yeah. The go-to-market is actually kind of interesting. It's via, for the most part, insurance companies actually, cybersecurity insurers. So they will insure a bunch of folks who get hacked. And when they get hacked, they have a pre-approved panel of incident response vendors who they will call in to deal with the hacks. And so actually, a lot of the motion is going and meeting the insurance carriers and talking to them and building trust there. And then off the back of that, you end up building the specific relationship with the enterprise. But yeah, a lot of the distribution is through the insurance carriers. It's quite unusual.

(1:14:40) Nathan Labenz:

The AI underwriting. You probably know the guys from the AI underwriting company then, I suppose. Yeah. Okay. Interesting. Insurance strikes again. Okay. Well, let's zoom out a little bit here in closing and talk about what's the big picture look like. You've got this kind of initial service that's your wedge into enterprises. How do you see that line of service expanding? Kind of paint a picture of, as this matures and you get really good at it, what can we enjoy in terms of additional security? Does it impact my day-to-day life at all, or is it just something that runs in the background and is sort of a guardian cybersecurity angel watching over me? And what's the timeline -- like, can I invest before you sell to CrowdStrike when they finally wake up and realize AGI is a thing?

(1:15:34) Alexis Carlier:

Yeah. The way I see this broadly, imagine we have AI systems capable of fully automated digital forensics, and much sooner than we otherwise would because we've done all this work of bootstrapping with humans and building these evaluations and so on. And the implication, I think, is that this completely changes how defensive cybersecurity operations work. Instead of missing this huge number of breaches because there's no one investigating deeply, you actually just have this proactively on in systems, and so you're catching a lot more than you would otherwise be. And the implication here is there's a huge reduction in the number of breaches that go undetected. And then, hopefully, this is just deployed across all the most important parts of the world. So it's deployed across AI labs. It's deployed across the AI supply chain. It's deployed in Western governments, safeguarding things like autonomous weapon systems. And when you look back at the overall effect of AI as it diffused and the effect it had on cybersecurity, you say, actually, look, it seemed like this was overwhelmingly beneficial for defenders. That is, I think, possible. It requires a lot of hard work, but it's pretty clear what you would do to get there.

(1:16:40) Nathan Labenz:

Anything else we haven't touched on that you want to make sure people are thinking about as they try to absorb your extremely AGI-pilled worldview?

(1:16:53) Alexis Carlier:

I think the big thing for me is just, I think we are one important attempt of doing this, pulling out the jagged frontier in a way that takes seriously what the world will look like when we get these really powerful AI capabilities. But I see very little of that, and I don't see any reason why that's the case. And generally, I don't think it requires any huge leaps of imagination or things like that. It seems just very tractable, very important to do, and I would love for there to be many more such projects across cyber defense, across biosecurity, across the whole space.

(1:17:26) Nathan Labenz:

Yeah. Time is of the essence, it seems. We're due for the automated AI researcher intern edition in just a -- coming sooner, I hear. Yeah. Wild times. Cool. We'll definitely continue to follow this with interest. And maybe with your motivation, I'll even get around to changing some of those passwords. Sounds great. Alexis Carlier, founder at Asymmetric Security, thank you for being part of the Cognitive Revolution. Thanks, Nathan.

(1:18:00) Nathan Labenz:

If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts which is now part of a16z where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at aipodcast.ing. And thank you to everyone who listens for being part of the Cognitive Revolution.