Underwriting Superintelligence: AIUC's Insurance, Standards & Audits to Accelerate AI Adoption
Rune Kvist and Rajiv Dattani of AIUC discuss their strategy to accelerate enterprise AI adoption by certifying and insuring AI agents. They explain how rigorous standards, audits, and insurance build confidence and address AI risks.
Watch Episode Here
Listen to Episode Here
Show Notes
Rune Kvist and Rajiv Dattani, co-founders of the AI Underwriting Company, reveal their innovative strategy for unlocking enterprise AI adoption. They detail how certifying and insuring AI agents, through rigorous technical standards, periodic audits, and insurance, builds crucial "AI confidence infrastructure." This discussion explores how their model addresses AI risks, enables risk pricing in nascent domains, and aligns financial incentives for safe, responsible AI deployment.
LINKS:
Sponsors:
Tasklet:
Tasklet is an AI agent that automates your work 24/7; just describe what you want in plain English and it gets the job done. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai
Shopify:
Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive
PRODUCED BY:
CHAPTERS:
(00:00) About the Episode
(02:53) AI Risks and Analogies
(09:14) Insurance, Standards, and Audits
(14:45) Insuring Ambiguous AI Risk (Part 1)
(14:54) Sponsor: Tasklet
(16:05) Insuring Ambiguous AI Risk (Part 2)
(25:26) Managing Tail Risk Distribution
(27:45) Introducing The AIUC1 Standard (Part 1)
(27:50) Sponsor: Shopify
(29:46) Introducing The AIUC1 Standard (Part 2)
(35:45) The Business Case
(40:43) Auditing The Full Stack
(48:00) The Iterative Audit Process
(54:58) The AIUC Business Model
(01:02:26) Aligning Financial Incentives
(01:08:56) Policy and Early Adopters
(01:11:58) Outro
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://linkedin.com/in/nathanlabenz/
Youtube: https://youtube.com/@CognitiveRevolutionPodcast
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
Transcript
Introduction
Hello, and welcome back to the Cognitive Revolution!
Today I'm speaking with Rune Kvist and Rajiv Dattani, co-founders of the AI Underwriting Company, who aim to unlock enterprise AI adoption by certifying and ensuring AI agents.
Their core insight is that security and progress are mutually reinforcing. Just as good brakes, seatbelts, and airbags are required if you want to drive 70+ miles an hour, rigorous standards for AI agent behavior and reliability are critical for society's effort to realize the potential of increasingly powerful & autonomous AI systems.
Their approach – which has already won support from an impressive list of industry leaders, including Cognition, Ada, Intercom, and many more – combines three elements: frequently updated technical standards that codify the latest best practices, periodic audits to verify adherence on an ongoing basis, and insurance to align incentives and provide financial protection when things do still go wrong.
As always, in this conversation, we get into the many details of making this work, including:
- the fact that today's insurance policies generally don't explicitly address AI risks, creating ambiguity about coverage for AI incidents;
- the AIUC-1 standard they've developed in partnership with technology and security leaders across a wide range of industries, covering data & privacy, security, safety, reliability, accountability and societal risks.
- their approach to auditing client companies, which combines analytical evaluation of technical safeguards with systematic red teaming;
- how the data generated by red teaming helps insurers price risk in domains where historical loss data is limited or non-existent;
- how they plan to structure financial incentives so as to avoid the race-to-bottom dynamics that affected credit ratings agencies in the run-up to the 2008 financial crisis;
- and how, even though the market may not be able to effectively insure against the largest-scale existential risks from AI, the government, as de facto insurer of last result, still benefits tremendously from the governance this model creates.
Overall, I have to say, I really like this approach – so much so that I did participate in the company's seed round alongside leading investors including Nat Friedman, Emergence, and Terrain.
While there is certainly a place for government regulation, and also for unencumbered experimentation, a private sector approach to AI reliability, powered by enterprise customers' desire to move with all responsible speed, and designed to align all parties' financial incentives with consumer and public safety, seems much more likely to get the important details right, not just once, but over and over again, as both the technology and market conditions continue to mature.
For now, I hope you enjoy this conversation about how the AI Underwriting Company seeks to build AI confidence infrastructure and insure the Intelligence Age, with co-founders Rune Kvist and Rajiv Dattani.
Main Episode
Nathan Labenz: Runa Kavist and Rajiv Dattani, co-founders of the AI underwriting company, welcome to The Cognitive Revolution.
Rune Kvist: Thanks for having us.
Nathan Labenz: I'm excited for this conversation. You guys are doing some really interesting innovation at the intersection of trying to make sure that everybody can have all their AI goodies, which I am very excited about getting. but also making sure that we keep things on the rails. I think is obviously extremely important as we head into a brave new AI world. So I'm excited to dig into everything that you're doing, the standard that you've put out, and the future that you guys envision. Maybe for starters, I think you've used a pretty provocative analogy in some of your writing and the way that you've introduced the company. Maybe tell us like sort of what the downside scenario would be for AI if we like don't do a good job of this. I mean, obviously there's like extreme, extreme extinction, but you've got sort of the nuclear outcome, which is a scenario where, you know, we don't get the goodies, but we still maybe get some of the bad stuff. And then, you know, maybe you could compare it to an industry where things have gone well that you take inspiration from.
Rune Kvist: Totally. And maybe before we get to the nuclear outcome. A lot of the concerns that we're seeing in the world today relate to today's risks that are much more kind of, in some ways, mundane. Data risk, safety and security risk related to like brand disasters and data leakage, some of these kind of age-old concerns that become much more severe in a new environment where there are many, many more vulnerabilities, and frankly, people understand the vulnerabilities much less. But as it kind of plays out into the future, we'll move to more and more agentic systems that are going to be way smarter. And that opens up new kinds of nuclear scenarios. One that's been talked a bunch about relates to imagine terrorists that want to afflict a huge amount of harm if they have access to brilliant scientists that can help them create, for example, a new virus. they'll be able to do that way, way cheaper. And so you can imagine new kinds of COVIDs coming out, and they'll be way easier for adversaries to create. You can also start to imagine what a misaligned kind of system looks like, where today that might look like deception or sycophancy. But as they get smarter and smarter, they'll do some of the things that humans can do. One of the things I sometimes get questions about is like, well, As AIs get smarter, will they still fail? And I think there are a lot of analogies to humans where a really smart CEO will no longer kind of mess up basic logic, but that enables them, their IQ enables them to run kind of Enron levels of fraud. So as the kind of agents get smarter, you get different kinds of scenarios. And we think that really opens up for nuclear scenarios. A different category of scenarios altogether is the geopolitics, some of which may not have to relate directly to AI. But if you really get national security leaders believing that AI will give their nation a decisive strategic advantage, that might lead them to preemptively attack their opponents in ways that may look like regular military escalation, but is really prompted by AI.
Rajiv Dattani: And Nathan, you asked about Other industries, I think there's not one, there's neither one industry that we look to and say, you know, that industry was perfect, nor is there a perfect analogy for AI. But the really the thesis, what we're building here, and I think we'll get into this more later, is how can we bring together learnings from lots of other industries, right? So the kind of three components to what we're building, the insurance, the standards, and the audits has come out of. cars. And when, you know, post World War II, there were a lot of car crashes on highways, insurers really developing crash testing solutions that led to airbags and creating the incentives for that to happen. Go further back into Benjamin Franklin's Philadelphia, and we've written a lot about this and fire insurers leading to building codes. And no one was saying, you know, stop building at that time, it was how do we actually keep building, but build in a way that's safer. And that's really the essence of what we're trying to draw on here for our solution.
Nathan Labenz: Yeah, it's funny. You went to even some more tail risks or kind of, you know, surprising possibles under the umbrella of the nuclear outcome that I had in mind. When I say the nuclear outcome, for me, that's kind of shorthand for we don't get the super cheap, abundant energy that we should have from nuclear technology, but we do have thousands of nuclear weapons that are you know, collectively a sort of Damocles that hangs over all humanity. And I do worry about a similar thing with AI where it sort of gets like weaponized or there's this AI Cold War, but maybe we also like freeze it out from a lot of the practical upside, you know, day-to-day quality of life improvements that I really want. Um, and what I really, for me, I often just think of airplanes because it's like, sort of like nuclear, there is the potential for like dramatic explosions. And obviously, if one happens near you or you're on the plane that goes down, that's really bad. But yet we do get the upside from planes. We fly cheaply and all the time. And as I always think back to Dumb and Dumber, it's actually more likely you'll get killed on the way to the airport than on the plane itself. So it is remarkable in my mind how contingent the end state for some of these like potentially transformative technologies can be based on, you know, even just a few incidents that happen early on in its development that can kind of put us on one course or another.
Rune Kvist: And I think, Nathan, we're already seeing that some parts of AI today, if you take kind of autonomous vehicles, you still cannot drive an autonomous vehicle outside of San Francisco very far before you get stopped. Yet drone warfare is increasingly autonomous, and it looks like there's no barriers to that getting deployed. So in some ways, we're already starting to see that when it comes to autonomous drones and vehicles.
Nathan Labenz: Yeah. Okay, well, let's get into the positive vision. Insurance, standards, and audits. This is like the holy trinity of a virtuous cycle that you guys are trying to kickstart. So paint the picture.
Rune Kvist: I can start high level. The core idea is that security and progress are mutually reinforcing. A kind of very basic analogy is that if you're a race driver, the reason you wear a helmet and a seatbelt is so that you can go faster around the corners. And the same is true for AI. The better we're able to steer and secure our systems, the faster we can deploy them, the more we can lean into some of these agentic capabilities. So the debate between the Dumas and accelerationists, we think this is a core point that they in fact are mutually reinforcing. Maybe Rajiv, I'll let you speak about the kind of the mechanics of the incentive flywheel.
Rajiv Dattani: Yeah, so the core of this, Nathan, is really, as you were describing, if we don't get the security under control, we don't think we'll get the progress, right? We won't get the benefits of the airplanes. And so what are the three components to this, as Renu was mentioning, there's insurance, There's audits and there's standards. So let me just explain what each of them are, and then I'll talk about how they link. The insurance is the financial protection for when something goes wrong, who's actually paying for it, right? And can you actually get the financial coverage behind it? There's a standard, which is what is the best practice, and can we actually codify the set of requirements that you need to meet to have the best practices and to have met them? And there's the auditing, which says, Have you met the standards? Now, the reason we think we need all three of these is Insurers will struggle to actually underwrite the risk and know how to differentiate a good risk and a bad risk unless there are a clear set of requirements that they can say, you know, yes, I understand these are the best practices. Are you meeting them? Which of them aren't you meeting? And that would affect your premium just like it would in your car insurance if you're driving recklessly. The auditing is essential because a standard without the auditing doesn't really help because It's very easy for companies to say, yes, I'm doing these things and self attest. But if there isn't the oversight to say, actually, yes, you know, somebody has rigorously checked this and made sure that the standard is being met, it's kind of a, there's a trust gap there. And so this is kind of how we think about the flywheel and critically in the past, insurers have funded the standards and funded the auditors to actually become more robust and create that oversight. And they've created the incentives for the companies who are creating the risk to comply with the standards because they want the insurance and they want cheaper insurance. And so creating the incentives where you have a pool of capital over here on the insurance balance sheets, funding the research, and you have actors who are creating the risk actually wanting to comply and wanting to have the insurance in place, we think actually aligns the incentives in the right way. Maybe one other thing I'll say on this is at the moment, this kind of debate has become polarized between let the AI developers cook and just, you know, let them commit to voluntary commitments and that'll be fine all the way through to on the other extreme. You know, we need top down regulation. We need it yesterday and it's got to be really strict. We just think this is a neat kind of middle ground here where you're aligning the incentives because the insurers want you to actually grow. That's how they grow their premiums as well. So they don't want to have too onerous requirements. They don't want to have requirements that don't actually track onto risks because then no one will buy insurance from them. On the other hand, they don't want it to be too lax, and they don't want you to actually be able to break your commitments because they're the ones paying. And so this is a neat kind of market mechanism that aligns the incentives between actually progress and security.
Rune Kvist: And maybe we can ground ourselves in some of the historical examples and just work through them. So the very first place we saw the incentive flywheel in practice is back in 1752 in Philadelphia when Benjamin Franklin sets up the first fire insurance company in the US. The basic problem was that Philadelphia was growing the population 10x over that decade. And so people were packing houses closer and closer together. which created a lot of fire risks. And the fire risk, of course, was spread. No individual actor was carrying the entire risk because the entire city might burn down. The fire insurance company were obviously having to pay whenever a house burned down, so they're interested in reducing the frequency of fires. And they created building codes which is in fact standard. How far should houses be apart? Can they be under large trees that might be hit by lightning or not, et cetera. And then they created the first fire inspection of folks that went into homes to just see are they meeting these building codes as a way of actually enforcing it. And so it's kind of that very basic logic that we've seen pop up again and again. For example, in the year 1900 as well, when electricity comes out, again, houses start burning down and The underwriters laboratory funded by the insurers pops up to define the UL certificate, which you'll see in your light bulbs today, is a product security standard that they will then go and audit against. That's the kind of mechanism that we're trying to bring to AI.
Nathan Labenz: Cool. I love the vision. Where do we stand today? I was doing a little deep research on the topic, and I guess my takeaway from what ChatGPT told me is that Mostly it seems like the harms that people anticipate from AI systems are covered under existing insurance policies, at least when it comes to like the taxonomy of harms, like the kinds of harms are are insured. People are sort of reasonably comfortable with the kinds of things they expect. But there's some ambiguity, I guess, around whether the fact that like an AI might cause them as opposed to some other cause could put something outside of scope. And it seems like a lot of people right now are just living with ambiguity. I understand this is one of the things that you recommend that the industry clear up is like what's covered, what's not covered. How do you see that today and what are the big points where we need clarification?
Rune Kvist: So the state of play today as it comes to AI insurance is that it's ambiguous about what's covered in the current insurance policies. For example, AI can help you create new kinds of cyber vulnerabilities. And of course, people already have cyber insurance today, but it's unclear whether AI-induced cyber vulnerabilities would be covered by the current policies because AI is not mentioned anywhere. And what we think is going to happen at a high level is very similar to what happened when computers came out back in the kind of early 2000s. Again, leaking confidential data or having kind of downtime was not an entirely new set of harms. But when computers came out, they happened at a very different frequency and a very different severity. And there was the same question, is this just age-old insurance, or is there something new here? And what happened was that over time, as these lawsuits came in to clarify whether things were covered, cyber insurance was split out into a separate set of products. because they need to be priced completely different and the products need to have a different structure. And we suspect that the same thing is true with AI here. One thing we know for sure is that the current insurance policies have not priced an AI risk because the insurers do not know how to price the AI risk.
Rajiv Dattani: And this just doesn't serve anyone. The companies who are insured and their customers are saying, I'm not sure if I'm covered. In the 2000s, when it was cyber, this took years to work through the courts where literally insurers were suing customers and vice versa. And from the insurer's perspective, as Runa said, they've not priced in the risk. They're taking risk on that they're not being paid for. And that's got to show up in loss data we expect sooner than later.
Nathan Labenz: Yeah, that's a really-- I had a very, very brief foray into the insurance industry some years ago. And it strikes me that this is a really hard challenge for them because they like to do this sort of thing based on historical data. It's like, we can look back at how many car crashes there are. We can look at how many homes burned down from fires. And with a couple of important assumptions that things will be relatively uncorrelated, you can be pretty confident that you can write a bunch of policies. And don't shoot yourself in the foot, and you'll probably be OK. But the same challenge is popping up with AI everywhere. I was just speaking to a group of educators last week and saying, I know you guys have been trained and it's central organizing principle of the field that you want to be evidence-based, but you can't be evidence-based because by the time you see a study, you've got two more generations of AI and the kids are using something that didn't exist at the time of the study. Here we are basically with the same challenge in insurance. Do we have any sense right now for what the frequency of these things is going to be? What the, you know, what the distribution of like magnitudes of harms are? And if people are, maybe we have some of it, but I'm interested in how much we do have. And then how does that translate into any sort of decision on pricing?
Rajiv Dattani: Yeah, there's a couple of issues here, Nathan. One is what you said. AI is just moving too quickly. It's moving faster than anything insurers have ever known. And so the slow rollout, let the data trickle in, pick it up in a few other categories, and then develop a product off the back of it just won't work. If you were building an insurance product today, based on historical kind of AI data, you'd be building for a pre-LLM era and not even thinking about the agentic risk. So that's one issue. I think the other issue here, though, is they don't really have a mechanism. They're not set up to even collect that data. And so if they don't have a product and these losses are kind of happening in kind of small places, which is typically what we're seeing at the moment, that there was an EY study on this recently, enterprises are having kind of million dollar size losses due to AI. Smaller companies are having, you know, in the thousands, tens of thousands of dollars. Insurers might not even be seeing this. And so if they can't see it, they don't have the loss data from which to price. And so what we expect is there's kind of a small number of, or rather there's a long tail of like small losses coming in, and then we're hearing and seeing the big ones. They're the ones that kind of make the press, and the rest of them are kind of being ignored both by the insurers and by the companies where they're kind of just writing them off. If you then think about how you translate that to pricing, it's just really hard for insurers to basically apply that model. And so there's a couple approaches we're taking here. One is the nice thing about AI is you can actually do what's kind of known as red teaming, you can run evals. I expect your your listeners are kind of very familiar with with what that is in the AI context. But if you translate that to insurance, actually, it creates a massive pool of synthetic data, basically, for how this product is going to behave in the real world. And insurers can literally plug and play this into their pricing models where they say, well, I see that, you know, this this failed the red teaming that I did in these particular ways, if that happened in the real world, I can estimate what the loss would've been because they know how to estimate losses from a data leak or a court case that would've happened, right? That's their bread and butter. And so they can take the data from the red teaming on like both frequency and severity and plug that straight into a pricing model. The other thing that's nice here is you can create pretty clear kind of parametric, as it's known in the insurance industry, pricing triggers, where you can say, actually, I can know definitively, did the AI behave as it was supposed to, or was this an out of distribution event? And by using that, you can then very quickly say, right, I'm going to pay out a million dollars because this happened, and I'm just going to price it. I'm going to pre-agree the pricing, basically, and pay out rather than waiting for a long kind of court case to come in and settle on that.
Rune Kvist: One other big consideration is, There is always someone underwriting the risk, even when there's no insurance. So today, when, say, a hospital or a bank deploys an AI system, implicitly, they are pricing the risk to decide do they want to go ahead or not. And so although it's like an extremely kind of gnarly problem to really pin down the price, the most important fact is that this is happening every day, all the time that people are trying in more or less sophisticated ways. And we think, by and large, the rigor of the insurance industry is going to make it more likely to get that pricing right than non-experts who are sitting distributed across every enterprise trying to make these decisions for themselves.
Nathan Labenz: Yeah, we're all self-insured, whether we know it or not at the moment. That's right. I guess one of the things that makes this sound really hard to me, if I try to imagine myself being an insurance company, is it feels like Again, with, you know, car accidents or whatever, I would guess that there's some sort of normal distribution of the damages, right? Like there could be some outliers, but those are probably be relatively infrequent. And I can, you know, put most of the incidents in a pack. Whereas, you know, there are other things in the world that are characterized by more of a power law distribution, like venture capital returns for one. I would assume, although I don't know for sure that like, Bio incidents probably fall into this category where a lot of-- I'm sure there have been many viruses leaked from many labs that just didn't go anywhere. But then you get one COVID and it turns into a global event. And my gut says AI risk might look more like that in that it could be a few incidents that drive the bulk of the total weight of the losses. Is that right? Do we even know, you know, how to think about that? And how much of a problem does that pose for insurance? And then I guess like, is there any way, assuming my mental model is not way off, like how do we start to attack that problem?
Rune Kvist: Yeah, the first answer is no one knows what the kind of tail risk distribution looks like because we have not yet seen it from nuclear. energy insurance, some nuclear power plant insurance that we can learn from here. So the same intuition applies here that probably nuclear failures are going to have a power law distribution where the real big failures will carry most of the losses. And in fact, that's true. In the US, there is, in fact, an insurance scheme for nuclear power plants. This is required by law. And the way they deal with these tail risk is that the government basically made a scheme where they said, we are going to require every nuclear power plant operator to have insurance, and we're going to require you to take liability no matter where in the supply chain things fall. So no matter whether it's one of your vendors who provided you with a faulty thing, you are responsible for all of this, and you're going to take out insurance. But on the other hand, we're going to cap the liability. So if there's a damage greater than $15 billion, this is going to come out of the government balance sheet. It's basically a government backed up. The reason why they're interested in this is, you're exactly right, the market may not be able to carry these very, very big tail risks. Yet the government would really like there to be an insurance industry because it provides an important governance function. It means that there is now a market-based version to have independent third-party auditors with direct financial incentives to price the risk accurately and enforce best practices on the industry. Basically, it does some of the governance work, but the government may not think that it has the capabilities to or is not going to be responsive enough to the market. And so there are some lessons there from nuclear for how do you get the best of private market governance while allowing the government to capture risk where the market may not be able to to really respond to it. And I think this idea of like capping risks is going to be critical again to AI, which is going to allow you to cover a broad set of risks up to some amount.
Nathan Labenz: Let's go to the standards and the audits. You guys have put out this AIUC1 standard, the first standard that I'm aware of anywhere in the at least Western world for what to do when you're building and deploying an AI agent. Tell me about it. What does it contain? What does it require companies to do? And how did you go about assembling all of these ideas?
Rune Kvist: Yeah, I can speak to the high level here. The core idea with AAC-1 is to take all of the concerns related to AI agents that slow down adoption and put them into one framework that outlines for each of the risks, what are you supposed to do in enough detail that you can have independent third-party auditors come and verify whether a particular company is doing those things. And so the kind of design principle here is to go out and spend a bunch of time. We met with more than 500 folks across the enterprises that are deploying these systems today. This is security leaders, general counsels. et cetera, across banking, health, et cetera, and figure out what is it that's keeping you up at night. And the answers will not surprise you a ton. It's going to be things like data leakage at a new scale. It's going to be, are we going to provide incorrect medical advice to our customers? Are we going to have jailbreaks and prompt ejections? Are we going to be discriminating against our applicants, et cetera, et cetera, baking that into one framework. The goal is when a company passes, meets the standard, the enterprises that are deciding whether to adopt or not will get the data foundation for them to understand whether they can trust that particular agent or not.
Rajiv Dattani: The way it's used in practice, Nathan, is often you'll have AI agent builders selling to large enterprises, right? Think of the large banks, the large technology companies. They want a third party to have vetted the tool they're looking to buy, and they ask them for their AIUC1 report. So a large company is looking at an AI chatbot developer, say, a customer support developer, and they'll say, share with me your AIUC1 report that has in it all of the detail on what are the data and privacy controls they have, what are the security controls and implementations they have, what are they doing to protect customer safety, You know, what output filtering do they have, for example, on how does it perform against bias? And is it tracking kind of its role correctly and not responding with anger or things like that with the or is it responding appropriately to anger if a customer really wants a refund and it's saying, you know, you must give me this and it's going to respond correctly. What are they doing with reliability? Do they have a groundness filter in place? Are they appropriately mitigating as hallucinations? What are the human in the loop practices? And what's the human oversight? And what are the societal risks that they're tracking and covering? And so in one report, the enterprise can look at it and say, it's been third party tested, I trust it, and I could see all the detail that I might need and preempt all the questions I have.
Nathan Labenz: Yeah, gotcha. So 500 people is a lot to get on board with anything. easy or difficult did you find it to boil all this down into a sort of consensus standard? I'm sure there must have been some things that were like contentious one way or the other in terms of like, well, including that is overkill or not including that leaves this risk unaddressed. Where is the line? What's the frontier of debate in terms of what is solid enough to be in the standard and what's still kind of next set of considerations?
Rune Kvist: We found it to be remarkably uncontroversial for most of the topics. There's a lot of interest in just creating one shared language. And a lot of the concerns say data, it's actually mostly just like churning through the issues and providing clarity on what should be included rather than debating where the line is. In part because the standard doesn't exactly specify what should be done, but it's a lot around disclosure so that any particular enterprise can decide whether this is okay, given this may differ radically whether you're a hospital or whether you're a retailer. These are different risk preferences. And so if you get some of those design principles, it's actually relatively easy to get people on the same page. The frontier of kind of controversy is probably related to some of the more societal harms. So should we check for whether a particular product enables folks to create cyber attacks at a much grander scale than before, which may not really be kind of that top of mind for any particular enterprise that's deploying it and might seem a little bit overkill. And so that's where we've had to kind of make some judgment calls about what will best serve adoption, not just for any particular individual enterprise, but for the industry as a whole. Because as you were mentioning, when it comes to saying nuclear, one incident can kind of chill a whole industry. And those are some of the effects that we've decided to include in the standard. It's still focused on like, how do you do that in a pragmatic way where you can actually assess it and you can get it done without having to spend... of dollars to check whether any random byte coding tool is going to bring down a whole industry. But we do think it's important to keep tabs on.
Nathan Labenz: So I guess, how does this look in practice? I was just talking to Andrew Lee a couple of days ago. He's launching this new product called Tasklet. And it is a very bet on the models. agent platform where his idea is like, forget all this step-by-step flow charts and all this field maps to that field, as you would see in a traditional automation software experience. And he's just like, we're just going to bet on the models to figure it out. We're going to give them all the tools they need. We're going to give them the memory. And they're going to keep getting smarter and they're going to go figure stuff out. And so I asked him, would you like to buy insurance? And he said, yeah, we actually have had a really great record so far that I thought people were going to be really freaked out or that we might see weird rug behavior from the agents from time to time. Fortunately, they haven't. But still, he was like, yes, I would like to buy insurance if only because then I can take that insurance to my prospective enterprise customers and show them that we've got this, and that should definitely ease our go-to-market process and speed it up. you know, perfectly consistent with your thesis there. But if we dig into it a little bit more, like, who do you think is primarily pushing this? Like, is it coming from an enterprise demand side or a vendor that, like, wants to, you know, prove their qualification side? And how much is there any way that we can, like, quantify what we're getting? You know, you do these various things, but like, what was the risk before? What is it after? You know, do we have any way of sort of saying to, an enterprise buyer, okay, you're going to buy this vibe coding tool and we're going to bound the risk in some way. We can make some level of confidence assertion that you're not going to have more than this big of a problem. What are those final statements that people are buying into look like?
Rune Kvist: We see demand for what we think of as a confidence infrastructure, both on the developer side and on the enterprise side. Because with or without insurance and standards, there is still risk. The enterprise is still nervous. They are still asking tons of questions of the developers today and sometimes just not buying because they can't get to confidence because there's too much ambiguity. So the enterprises definitely would like to have clarity, one language, one standard, one report that they can trust. But the flip side of this, as you're saying, is that currently it's incumbent on the developers. to create clarity individually, to earn the trust, to earn confidence. And there aren't a whole lot of tools available. They can write public blog posts on their security posture, and that's great, but there's, of course, they've graded their own homework. Everyone thinks that their security posture is great. They will sometimes reach out and make financial guarantees off their own balance sheets, and that can be Frankly, I think that's a great way to do it, but again, the balance sheet may not be able to carry it. So we actually see the most forward-looking developers are as interested in creating this kind of confidence infrastructure as the enterprises, even though, of course, the enterprises are kind of the ultimate beneficiary of it. To your point around, what's the impact on this on the risk? Probably the best way to look at this is that when we go and certify a particular AI developer, we run a set of evals that finds a set of vulnerabilities. And we'll often do that in multiple rounds. So in the first round, we tend to find quite a lot of stuff that they themselves are very interested in solving. So we often find that sometimes you can get up to a 25% failure rate on certain kinds of attacks against the company, which is just obviously not what they want nor what the enterprises want. And we'll then, with the help of the standard, help them implement whether that be a groundedness filter or some kind of monitoring solution or a content moderation filter, and see those same failure rates, and we can often see them drop by 90%. The honest answer is also there are no panaceas like AI security as a field and the safeguards. It is a work in progress, an open research problem, but there are some pretty basic steps that they can take. And we can see that pretty quantitatively from our first round of red teaming to the second or third round of red teaming. And That is the way that we get a little bit more confidence that through this process, the AI companies that are deeply hungry for improving actually will improve in a quantifiable way that they can convey and communicate to their customers to earn confidence, which is ultimately what allows them to invest in this stuff when they have a million things that they could be investing in.
Nathan Labenz: I wanted to do just one kind of double click on the kinds of companies that you're working with. It seems like a sort of tacit assumption of most of the conversation so far has been that we're focusing on application developers who are presumably in most cases not training their own foundation models. They may be using a commercial API, they may be taking an open source model and doing something with it, but they're kind of providing the, you know, wrapping of that. And I don't mean that in a derogatory way, but obviously there's a lot that goes around models to make them useful for enterprise purposes. But there are other layers of the stack that one could think about wanting insurance for, or, you know, wanting to do similar audits on. Are you guys doing stuff at the foundation model layer or even deeper at like the physical data center infrastructure layer as well? Or are we limited for now to the application layer?
Rajiv Dattani: Yeah, we think this model applies at all three layers, right? So if you think about this kind of alignment of incentives, research into security and how that unlocks progress, It's clearly, you know, it's clearly most acute at the application layer at the moment, because these are the companies who don't have large enough balance sheets to kind of make large promises on their own. It's also to some extent where the risk is headed in terms of, you know, they're the ones building the agents and deploying agents today, even more so than the other foundation model developers, for example. But if you think about data centers, you know, we spoke recently to a chair of chair of the audit committee and risk committee at a, you know, one of the, one of the top tech companies, um, who was saying, well, we're investing an astounding amount of money into data centers right now. And our public shareholders don't want to be bearing that risk of what happens if the data center goes down, which could be anything from actually, you know, just wanting uptime through to there's an attack on the data center or, you know, there's an issue with the, with the security of it and it gets breached. And so, you know, they're saying, can we buy insurance for that? And can we have assurance from a third party that these risks have been well managed in a way that wasn't the case, you know, historically with, with kind of data centers they've built and invested in the past. Now that's obviously its own standard, right? AAC-1 doesn't currently cover the risks for that. So, What we expect we'll do is we'll run the same process that we've run with AIUC1, where as Rune described, we talked to 500 executives and understood what are the risks and what are the mitigations, and we talked to AI experts. We'll do the same thing on data centers, where we bring in the right folks who are really leading the thinking on this, both on the buyers and the builders of them. We'll codify the best practices. We'll figure out what are the ways in which you actually confirm the security is in place, and then we can insure off the back of that. And you can imagine exactly the same applying for foundation models.
Rune Kvist: One way to think about it is also a game of earning insurers confidence over time. So start with the risks that might cost. millions of dollars or tens of millions of dollars, move on to ensuring risk that might cost you hundreds of millions of dollars or low billions of dollars, and then move on to the tens of billions of dollars eventually. And think of it as a ladder up and the identic places where you have a lot of relatively small incidents, which gives you more data, allows you to build confidence, you have good models, and then slowly earn the confidence to move up the ladder.
Nathan Labenz: Yeah, it makes a lot of sense to me also just from the standpoint of what I see people doing, like who seems to be doing a good job and not such a good job at, you know, this stage of the game. I wouldn't say I know too much about the data center layer. My sense is that, you know, there's a lot of new concerns here for them, for sure. But at a minimum, they're like well aware that they're putting tens of billions of dollars into the ground. And, you know, they have very obvious incentives because they're like, you know, they're well aware that they're self-insuring. The foundation model providers strike me as being remarkably, not, you know, with some notable exceptions, like remarkably good about this stuff. And again, we could, you know, we could question whether the whole enterprise is like a bad idea, big picture, but like they're, you know, they're doing a lot of work to taxonomize risks and to build defense in depth approaches and all that kind of stuff. And then what I see at the application layer, I guess it's starting to get better, but Certainly for most of the last two years, what I've seen is weekend hackathons turn into businesses. People never really gave this any thought before they kind of built something and launched it. And they just have zero, in many cases, zero protections in place whatsoever. So I do think from the standpoint of just where is there low-hanging fruit, where is there a lot of needless risk that you know, enterprise buyers like legitimately should beware and should be asking these kinds of questions. The app layer absolutely has a lot of low hanging fruit and there's a need to be able to identify and separate, you know, who has done a good job and who's really, even if you can't take the risk to zero, like who's made a best effort, you know, to get to a place where they can responsibly be deployed in an enterprise or a high stakes setting and who hasn't, and there are a lot that haven't. Would you agree, disagree, add any color to my sketch there?
Rune Kvist: I broadly agree with the assessment. I think one piece of color to add is that at the agentic application layer, the promises being made to the enterprises are much thicker in some ways, where if you buy Claude for your own purposes and you go to Claude.ai, the promise of what Claude will help you to do isn't very specific. They're not claiming that they will give you correct medical advice 90% of the time. They're just saying, here's a helpful tool, use it for whatever, it will hallucinate. If you move into a space like, say, customer support, the kind of public claims that these companies are making are like, we will resolve X percent of your tickets. And kind of implying also in those sales processes, we will not create massive brand disasters for you because we're going to-- you can trust us to interact directly with your customers. So the thicker the promises, the thicker the insurances and insurance also needs to be. And that's a little bit of color for why the agentic application layer is the place to start. One other piece of color is that the distinction between foundation models and agentic applications is blurring. Take an example like Cloud Code. It is clearly not a foundation model. It is a system of models, at the very least, with a bunch of different things going on, a bunch of integrations, a bunch of access. And so this line is really blurring. And we think agents is the most helpful frame to just think about it. And that's where some of the application layer companies got there first. But the companies you think of as foundation model companies are quickly moving into that space.
Nathan Labenz: So let's talk about then the nature of the audits that you guys are doing. You mentioned red teaming. If I'm an application developer and I am, like I've got way, Mark, we are in a blessedly low risk environment because we help. Media companies make marketing assets for generally small and local businesses. And my standard line for our team has always been, if somebody breaks into our system and watches all these TV commercials that we're making for local businesses, nothing bad will happen. That's a great comfort that we have available. Let's say I want to do this. You can go poke it at our thing from outside and prompt it or ask it to do hateful or whatever commercials, and you probably could find some gaps if I had to guess. What more do you do? You get under the hood, you read source code. What are all the different angles of attack that you bring to bear on figuring out the many ways an application could go wrong?
Rune Kvist: Totally. The foundation is to start, ground ourselves in the real-world incidents that have happened to be like, what is it that we're looking for? So we have a database of incidents, real-world incidents that have happened. that inform what are the things that we should be checking for. Some of those are things that you might think of as kind of very concrete headlines. BBC says, wow, Air Canada's chatbot hallucinated a refund policy. Great, we should clearly check for hallucinations that have financial implications. Some of them might be slightly more speculative, like a research paper that shows under these kind of experimental circumstances, AI chatbots will try and blackmail the user. We also think of it as like an incident, it's just slightly more speculative. That forms the foundation for the audit. We then try and abstract that into a taxonomy of both what are the harms we're checking for and what are the kinds of attack vectors that are relevant? What are the kind of latest research into jailbreaks? What are the kind of latest examples of applying to the prompter on Twitter for different ways that these models break? And that kind of becomes the checklist of things that we want to check for. And the way we check it is broadly For each of those kind of harms and risks, we specify what are the technical, legal, and operational safeguards you must have in place, and what are the evals you need to run. The technical safeguards could be like, Oh, you must have a groundedness filter that reduces the chance of hallucinations. Or it could be, You must have a plan for what happens when **** hits the fan. What is your AI incident plan? Those get checked either by looking into the code base, reading their internal policies, et cetera. And then secondly, there's the technical testing element, the red teaming, which really grounds everything. It's not enough to have an AI groundedness filter in there if it doesn't work. So that's where we go and take all these scenarios and try and break the chatbot to see how effective is that. And then the output of this is, you can imagine a report that says for each of these technical, legal, and operational safeguards, what do they have in place today? And for the eval, displaying back the results of what we found in a way that both helps enterprises make sense of it, but also leaves room for them to make the final judgment call with some level of confidence of whether it's a go or no-go.
Rajiv Dattani: So one way to think about this, Nathan, is there's a trust and verify approach that we take. So what are the policies you have? Have you implemented the safeguards? And can we verify they work? And so again, you could imagine exactly how this would then also translate to foundation model developers, whether it's You know, they have these policies on what safety testing they'll do and how they'll implement various safeguards. How do we actually verify that's happening?
Nathan Labenz: Yeah, and you mentioned that there's like multiple rounds of this process, and I think it's probably worth talking about why that's important. I was just reading a paper from Nicholas Carlini, Sandra Schulhoff, and others, I'm sure you've seen it, The Attacker Moves Second. And the basic notion of that paper was if you take the attacks as static and you say, OK, I'm going to tweak my defenses to mitigate those attacks, you can do that. And you will mitigate those static-defined attacks that you're trying to mitigate against. However, when you then deploy those defenses into the real world with potentially adversarial attackers coming at you, They find ways around those defenses at a remarkably quick rate. And so what they found in their study was that human red teamers were basically 100% undefeated. Like there were no defenses that were totally impermeable to human red teamers. So how do you go through a sort of iterative process to, you know, not just have this security theater of like, well, we had one shoe attacker one time, so now everybody takes their shoes off at the airport. And then we tell ourselves we're good, as if there was no other way that you could get something onto a plane. So how do you think about that iterative thing? And how do you know when to stop? Because this could obviously go on forever, too. How do you even decide when you're getting to a point of diminishing returns or a point where you've hit some sort of best, you know, good point on a Pareto curve.
Rune Kvist: Yeah, there isn't like a single neat answer. Ultimately, this is a judgment call. The important thing to know is we do a few rounds every time we do the auditing. And then to get certified, the companies commit to doing quarterly audits. They will change their product. The world's incident will evolve. There will be new research papers into new ways these will fail that we then kind of continuously incorporate into this body of tests that we run, that we run in each quarter. In practice, it becomes kind of a cost and effort versus confidence. And these are just, in fact, not at all new considerations. Cybersecurity has the exact same thing. No one, with enough effort, every system is breachable. And ultimately, that's pragmatic. And over time, the cost of running good audits goes down as intelligence becomes cheaper. But the set of attack vectors probably also grows. So there is, in fact, a very neat answer for where you stop. Ultimately, this is guided by the market. What are the level of assurances you want? And what are the things that people are willing to pay for to get that assurance? Which is probably like, it's a messy answer, but the actual pragmatic answer for how most of that stuff gets settled in every domain today.
Nathan Labenz: Yeah. What does this look like in terms of your business model? Like who pays you? I suppose the developers pay you? How much do they pay you? How does that depend on like what sort of product surface they have? What is that quarterly? update process look like? Is it the same thing as the original or is it more streamlined as you go forward? What's like the overall customer story for the AI underwriting company?
Rune Kvist: Yeah, totally. So the company that's building their system, Pace, that can either be what you think of as like the disruptors in the space, these AI native companies, but it can in fact also be enterprises that are developing something in-house. You'll see a bunch of enterprise that are building their own customer support agents because they think that's a real moat and they want still to have confidence before they deploy that. So it's the companies that develop the systems that fundamentally pay, they pay on a yearly basis. The certificate lasts for one year. With that, you're required to have these quarterly technical tests. And the The pricing depends, as you're saying, exactly on the risk surface, the product surface. And also, of course, this also ties to value. Bigger companies tend to be able to generate more revenue out of this. They also tend to have a bigger risk surface. So it scales a lot. Other parts of the business model that are interesting is the same companies that are interested in getting certified overlap a lot with the companies that want to have insurance for the same purposes. They want to take risk off the table from their customers. and deploy faster. Or when it's the enterprise, they want to just reduce the risk so they can deploy faster. And that insurance as a business model also works on a, you typically buy an insurance policy on a yearly basis for a yearly fee. So you can imagine those layering on top of each other. And then we're experimenting a little bit now with, imagine an AI native company that wants to not just get certified for the product in general, but also want to certify specific deployments to specific customers. They want an extra layer of guarantee. They may want to know that, oh, I'm a bank deploying this. AI voice customer support chatbot. I want to know that with my specific customer support policies and my specific customer sets and my specific policies that this will also work. And so that's another way to kind of get more precise in the value. That's where the AI disruptors are really willing to pay because it's going to unlock a contract. But it's also very clear that it's the maximum level of assurances an enterprise can get when it's done literally in their environment.
Nathan Labenz: Can you tell a little bit more about what the requirements are from the customer side. And I would be interested to note if you're willing to share tangible price ranges. I assume this is, sounds like it starts in the maybe five figures and probably goes into the six figures depending on how big the product surface is. And do companies need to make team members available to you? Do they have to, or what beyond the sort of desire and willingness to pay, do they have to bring to the table?
Rune Kvist: Yeah. Before we sign a contract with anyone, we do a gap analysis. So we take the standard, and outside in, we take what is the kind of evidence that's publicly available that gives us a sense of where do they stand against the standard. And we then do an in-depth review, like row by row, where do you actually stand Today, that gives us some kind of effort estimate. What's the amount of engineering effort you need? What's the amount of non-engineering effort you need? And also from our side, what is the total scope here? And that's where we together come in and we're like, what's the price going to be here? The range you mentioned is broadly correct. And that kind of gives everyone a sense of what are the costs involved. on both sides. And that's where when they commit, they commit for a year. At the end of that year, they can decide whether they want to recertify. This is very kind of in line with how they already think about certifications in the cybersecurity space, things like SOC2, ISO standards. So really, a gap analysis is the core thing that says to everyone, what's the effort that's involved here? What's the cost going to be? And we can do that, frankly, within 24 hours.
Rajiv Dattani: And what we find, Nathan, is anywhere from our largest customer has thousands of employees they've been running for years. For them, it's like a small handful of hours of actual engineering lift or security team lift, actually, in that case, and just evidence collection for us through to our smallest customer when we started working with them, had two or three employees. We needed to work with them in a much more hands-on way to actually implement the security measures that are required in the standards, remediate the issues we found through the testing. And that was obviously a much more evolved process. And so it's kind of hard to give a stock answer without doing that.
Nathan Labenz: Yeah, well, it's going to be, seems like something everybody's going to need, right? In a similar way to whether it's me as an individual driver of my minivan or a nationwide trucking company, everybody's going to have to have some sort of insurance to operate, presumably before too long. I guess, do you think that's true? Is there a class of AI app that just never gets into this for whatever reason? Or do you think this does ultimately become something that really everybody has to do in the same way that drivers do?
Rajiv Dattani: Yeah, I might separate the insurance from the third party testing for this, right? I think, I think there is for almost every kind of valuable use case you can imagine, there is a need to have somebody who is not just marking their own homework, actually test it and say it is secure, secure to ingest, you know, customer data or a company's kind of private sensitive IP and data and make sure that's all being protected, um, et cetera, et cetera. And then I think on the, Insurance side, I think that's where you could imagine more of a bifurcation, you know, traditional software SaaS has been sold on kind of have a nice day basis, right? Often those aren't insured. You might get some service credits if something goes wrong, but you're not, you're typically not going to get more than that. As you said, you know, cars and houses, obviously everyone kind of carries insurance. And in lots of cases, this is mandated. You could imagine one version of this is a similar kind of bifurcation where the highest risk deployments and the places that are creating kind of externalities also really should be insured because that's where, you know, you want an extra layer of protection from a strong established balance sheet to provide the financial coverage. Whereas, you know, if I if I take your example of of producing a producing media copy, you could say actually the risk here is is manageable and it doesn't need insurance.
Nathan Labenz: Yeah, makes sense. Um, I was briefly in the financial services industry, as I alluded to, and one of the things that I happened to have kind of another one of these Forrest Gump, um, extra roles in an important scene in, uh, in recent history was the kind of problem of the credit rating agencies. And, you know, basically what happened in the late 2000s is that these Companies were, in a sense, captured by their customers. They had the incentive to say things look good. They didn't have the incentive to turn over too many stones. And so we ended up with all of these very highly rated bonds going bad. How do you think about preventing that outcome? I guess there's probably multiple dimensions of it. One is just like, how do you make sure that your audits stay weird enough? It almost seems like you need heterogeneous cognitive profiles on your team to make sure that you are bringing weird and off-model ideas to the table as often as possible. But then there is this incentive question too. At some point, as this gets normalized, there's going to be some pull or there might even be some competition. That's I think really where the ratings agency things went bad when it was like, well, if you don't rate us high, we can always go to the other rater and they'll rate us high. Then it's like, well, geez, what do we do? We're going to lose all our business by giving our own customers bad ratings. That's a tough line to hold. You're trying to create a race to the top, but there is at least some precedent for this sort of thing ending up in kind of a race to the bottom. How do you think about making sure we stay on the right side of history here?
Rune Kvist: The core thing that's going to create the right incentives here is the insurance that gives us the skin in the game to not just lower the standards. When you take the Moody's example, Moody's does not have financial skin in the game outside of trust. So when the financial crisis hit, there's not a direct financial mechanisms that force them to internalize the risk that's being created there. And when you look back at, for example, the UL certificate that's on all of your electrical products, which I think of as like broadly a very successful instance of standards, that was created by the insurers. It started from the insurers having data on what's going wrong and having the incentive or yeah, the direct incentive to make sure that the standard covered that. So we think the insurance piece creates the scan of the game to prevent this race to the bottom. And there's a lot to learn from history there.
Rajiv Dattani: And in terms of how we keep up with it and keep up with the frontier, Nathan, this quarterly processes, it's actually just really involved from our side. We have a mix of the actual people we're certifying and the agent builders, right? Because they're on the ground saying, you know, here's a new safeguard that's really working well for us, right? Great. Let's feed that in. We have a range of academic researchers and other people kind of building safeguards commercially, giving us input, right? Folks from Stanford and University of Illinois and Hayes Labs and Grace Swan and Virtue AI. And then we have the enterprises. We've just launched our enterprise consortium, which is a mix of security and risk leaders from Fortune 500 enterprises who are actually also on the front line of the deployments and saying, here are the risks I'm worried about and the risks I'm seeing. And all of these people come together to give us input both on what's missing, where's the frontier headed, what are the places where AAC1 isn't keeping up with that, and then how do we write the precise requirement in a way that will capture the spirit of what should be in here. And so really, it's not just about our team here, which of course has lots of these capabilities Nathan you're describing, but it's really about how do we mobilize the whole ecosystem around this and bring that together.
Nathan Labenz: Do I understand correctly, just to rewind to the insurance piece on the incentives, like, are you planning to actually provide insurance? It sounds like you're both going to, if I'm understanding correctly, like, you're going to have skin in the game at the insurance payout level. Is that right?
Rajiv Dattani: Yeah, we'll have skin in the game, Nathan. The way this works is we'll become a managing general agent. It's the kind of insurance term for it. And in practice, what that means is our payouts-- so the insurers will pay us, and our payouts will be tied to the actual underwriting result, the profit that they get on the insurance product. So if there are large losses, we don't get paid, or we get paid much less. If in fact our risk assessment was good and there's fewer losses, we also benefit from that financially.
Nathan Labenz: Interesting.
Rune Kvist: And if there are large losses, the insurers will not want to work with us. So we will not be able to, we'll go out of business.
Nathan Labenz: Yeah, gotcha. Is there a big difference between that and writing your own insurance? Is it just a matter of like size of balance sheet that causes you to structure it this way? Or do you imagine like at some point building a giant balance sheet and like underwriting your own policies and doing your own payouts? Or is there a reason to let the traditional insurance companies handle that indefinitely?
Rajiv Dattani: Yeah, insurance and actually reinsurance is often where the risk sits is really a cost of capital game. It's a kind of risk aggregation game, as we were talking about earlier. And so there's a lot of benefit there in being diverse across different risk lines, across selling through brokers who are selling multiple products, et cetera. Really, we want to focus here. We're focused on the AI risk. And so for us, this gives us the benefit and the flexibility to go out and write and structure insurance products, knowing we've got insurance partners, some of the most established insurers in the world who've never failed to pay out a claim. They've been around for hundreds of years. This is their business. And we're really excited to be able to write with them rather than having our own balance sheet.
Nathan Labenz: I've been following this SB 813 proposal for a while. The idea here, as I'm sure you guys are very well aware, is to create some sort of private governance constellation of options where companies could opt in to abide by a certain standard. Maybe you could even be one of these multi-stakeholder regulatory organizations, I think they're called. And in exchange, the companies get some liability insurance. There's also a bunch of other proposals for liability reform or changes in all sorts of different ways. Do you have ideas on policy? Like what policies do you think would work best? What are most compatible with the business that you're trying to build? Are there any that are in indirect conflict with it? What do you think is the future of that?
Rajiv Dattani: Yeah, from our perspective, anything that promotes the creation of a market here and market-based solutions is the key you know for the reasons we talked about earlier that's that's the thing that aligns the incentives because if you have a free market of oversight you're going to you're going to actually incentivize robust risk evaluation um without hampering growth and that's really important right we don't we don't want this to become too onerous and and crowd out the startups and uh crowd out Innovation um and and so there are plenty of policy ideas like like you mentioned that do allow for this and do allow for and do promote the creation of best practices and third party oversight. And the other thing that's important, though, is which which we think SB 13 didn't do is actually force the companies who are creating the risk to own it. Right. We don't want to create safe harbors where you have too many free passes, where you can create risk, and then you're not punished for that and you're not held to account for that. And so we think that's a fine line. And we think insurance actually allows you to internalize those externalities and actually take on the risk you're owning while still creating the market around it.
Nathan Labenz: Last question. Who all is involved? I understand there are quite a few well-known companies that are starting to buy into this. So maybe give us a little overview or name check of who's hopping on the train.
Rune Kvist: Yeah, on the startup side, we've had some... an awesome group of like kind of founding technical contributors from various different spaces. We work closely with ADA and Intercom in the customer support space. We work with Cognition on code. We've worked with a company called Recraft and Image Generation. So trying to really look across the spectrum of the different kinds of modalities and deployments. And maybe Raj, if you can speak a little bit more to the kinds of companies that are in the enterprise consortium.
Rajiv Dattani: Yeah, so we've got risk and security leaders from Financial services like JPMorgan Chase, for example, who are really thinking about how do we adopt AI across the bank globally, not just in specific kind of verticals. In tech, the CISO, the Chief Information Security Officer of Confluent, for example, is helping to shape AIUC1. the deputy CISO Anthropic is is kind of deeply involved and we've got others from health care, from kind of retail, etc.. So it's really trying to represent across industries, actually also across geographies. We've got folks from from Asia, from Australia, from Europe, and really bring this together into one framework and make sure it's as strong as it can be.
Nathan Labenz: Cool. Well, We're out of time. This has been great. Any last thoughts you want to leave people with or anything that we didn't touch on you want to make sure to mention?
Rune Kvist: I think this is pretty comprehensive.
Nathan Labenz: Love it. Cool. Well, Runa Kvist and Rajiv Vitatani, co-founders of the AI Underwriting Company, thank you for being part of the cognitive revolution.
Rune Kvist: Pleasure.
Rajiv Dattani: Thanks for having us.
Read next
