No Code Is Code: Zapier CEO Wade Foster on Headless Tools, Zapier MCP & Automation Bench

Watch Episode Here

Listen to Episode Here

Show Notes

Working title — no approved title yet. Second appearance; first visit was September 2024.

When Wade Foster last sat down with Nathan, Zapier was a no-code automation company learning to call out to language models. Two years later it has built what Nathan calls "everything": workflows that invoke AI at any step, Agents, Chatbots, an MCP server, an SDK, and AI Guardrails. Asked what all that building taught him, Wade's answer is about shape rather than features. What the company didn't know in 2024 was where AI work would actually happen — inside existing products, inside new ones, or inside the labs' own apps. What happened instead is that most people picked a single "daily driver" — Cursor, Claude Code, ChatGPT — and do most of their work there. That makes tools like Zapier MCP, which bring a person's context and data into whatever harness they already use, "the way knowledge work is moving." Wade sees a split in the market between companies still trying to force users onto their own agent-building platform and the headless camp — he names Salesforce — that says bring it wherever you want. "I definitely think that latter is the winning strategy." His own daily driver is Cursor, with Zapier's internal harness (a virtual-file-system context layer with automations and deployable apps) pulled in over MCP.

Nathan brings in Tasklet founder Andrew Lee's thesis from his recent episode — that everyone is building the same horizontal "mecha suit" for models, and only a few will win it. Wade agrees with the premise that nobody wants to be beholden to one model provider, and grounds it in data: Zapier's AutomationBench scores frontier models on roughly 600 realistic knowledge-work tasks across marketing, sales, HR, and operations (the public task set and paper are open). GPT-6 Astra, released the week before this recording, is the new state of the art on the official leaderboard at about 40% of tasks completed correctly, by Wade's account — while Gemini 3.7 "does pretty good, but does it at a fraction of the cost." That cost-versus-capability curve is why any modern organization wants a layer that lets it swap models per task. The sample task Wade reads — close the Meridian Core deal, mark it won, route the win notice per the routing policy, confirm accounts against the hierarchy spreadsheet, convert currencies, check for open escalations — is the kind of thing that fires many times a day at any company that is closing deals, not a one-off. And the benchmark is far from saturated.

The most durable idea in the conversation is one Wade restated from 2024, now with more evidence behind it: most of what people use agents for should be deterministic code. "The vast majority of what people are using an agent for, in fact, they probably should be using old-fashioned deterministic code. You really only want the AI to reason over the things that you need it to reason for." Zapier MCP is built around that division — the model builds the workflow, writes code for the parts that should be cheap and reliable, and reserves reasoning ("the ghost in the machine") for the steps that genuinely need it. AutomationBench V2 will measure exactly the lift Nathan asks about: how much higher the scores go, and how much lower the cost, when a model gets Zapier as a tool. Building itself has flipped. "The idea of building in no-code, it feels antiquated to me. It's like the new code — no-code is code." Humans still benefit from the visual workflow as verification and documentation, but the agent does the editing, and increasingly the troubleshooting when a hardened workflow fails. Inside Zapier's own support org, the auto-email troubleshooting program spins up five independent agents; when four of five agree on the diagnosis, it's usually right. Human reviewers give a thumbs-up or a thumbs-down with a reason, and the reasons feed back into the system. Nathan probes for a clever retrieval stack underneath and Wade declines the bait: "A lot of this is not particularly fancy at the end of the day. It's just — take the example, did you like it, did you not like it, and just rinse, wash, and repeat." Fifteen years of automation data, plus hooks into everything, is the moat — if it can be wielded into a product the models alone can't match.

On competition, Wade is unbothered by the labs and worried about something else. He recalls Paul Graham's advice when Zapier went through Y Combinator: the question was always "what would happen if Google built this?", and PG's answer was that you are rarely competing with Larry and Sergey — you are competing with "a potential mid-level product director who's trying to get a promo..." Today, "OpenAI, Anthropic — they're big tech now... they can't build everything. They just can't." The harder problem is the "sea of sameness", and the fact that the average user is "candidly not doing much" — a bit of ChatGPT or Gemini, Copilot at work. The real competition, in his view, is non-adoption, and the opportunity is educating a market he now estimates is "probably a thousand times bigger than what we sort of set out to build." Nathan admits he has been wrong about the pace of economic change; Wade, calling in from Missouri, agrees that "life is kind of the same" for his friends and family, who use ChatGPT for meal plans and vacations rather than anything Astra-level. What actually moves people, he says, is specificity: an agent that watches what you do. His own weekly automation collects event streams across Gmail, Slack, and his browser through Zapier MCP and proposes tools to build; ideas only have to be "50% good enough" to kick off the brainstorm. He expects Zapier to productize it, and when Nathan asks why logs rather than screen recording, the answer is pragmatic — "we're good at APIs" — with Nathan noting that AIs are simply better at reading logs than a human advisor would be.

Pricing gets a blunt verdict: "seat-based pricing is dead or dying." Commodity products drift toward usage-based pricing — the Sam Altman intelligence-as-a-utility framing — while enterprise products reach for outcomes, like customer-support tools billing per resolved ticket. Most products stop a step or two short of a clean outcome and end up "selling work of some portion, at the end of the day." Asked whether model companies' consumer-plan token subsidies amount to price discrimination against the ecosystem, Wade leans free-market, but points to Microsoft's bundling era as the case where advantage was used "to box out better products," and says the rules are for countries to set; his job is to find an edge inside them.

The second half turns inward. Zapier put its chief people officer — Brandon Sammut, now Chief People & AI Transformation Officer — in charge of AI transformation, which Nathan notes cuts against Wade's 2024 line that AI should be everyone's job. Wade still believes that; what changed is the bottleneck. Within about a year of ChatGPT, nearly 100% of Zapier's employees were using AI daily, so adoption stopped being the problem. Turning individual fluency into production-grade workflows across the company looked like people work — rewriting job descriptions, rethinking compensation, re-skilling teams — and Brandon's team was already best at it. "It could have just as easily been a CMO, or a CPO, or any number of roles." Zapier's default-to-transparency value got a similar upgrade: Slack's public/private readout showed work drifting into DMs, agents were plainly more effective with full Slack context, and a friendly executive-team competition to put more in public channels did the rest. HR incidents and active security vulnerabilities stay private. "Most people far overestimate the number of things where that is required, and definitely underestimate the power of what happens when both humans and agents have access to the full context of what a company is working on."

Security is where Nathan expects Wade to be scared — Zapier holds more credentials for more services on behalf of more users than almost anyone. Wade's answer is that the stewardship has been the job for fifteen years; what's new is Mythos-caliber models that can "patiently just loop over issue after issue after issue" and find vulnerabilities. "Every software project has vulnerabilities. It's just, has someone found them yet?" The same models make patching easier, so smart companies are wielding them for offense and defense. He calls the Hugging Face incident "straight out of a sci-fi book" and expects a one-time re-acclimation before a new steady state. He confirms Zapier wants early access to frontier models, and Nathan argues that membership in programs like Project Glasswing could become a real differentiator for anyone asking to hold your credentials. On spend, Nathan cites Wade's post about engineers running $30,000 a month in tokens. Those people are outliers, Wade says, and there are no individual budgets yet — but Zapier built dashboards so people can see what a powerful model costs versus a cheaper one on the same workflow, and his first move when someone's spend spikes is to ask what they're doing. Some are "insanely productive"; others "don't need to be using Fable for this, or Astra for this." Across an almost-800-person company that is an education problem, and he expects token budgets tied to demonstrated fluency to become normal.

Nathan raises AI-assisted writing carefully — "not a gotcha, because I'm in the same boat" — confessing that his own Pangram scores sometimes read as AI before noting that some of Zapier's output does too. Wade's answer is a set of norms rather than a ban: the problem is not AI, it's low-quality communication, and AI lets a low-judgment person overwhelm colleagues with high volumes of it. So own what you send ("you probably should be putting more time into authoring the thing than the reader is reading it"), understand it well enough to answer questions, make asks explicit, verify details — an agent with Slack access can pull a related-but-wrong project from three months ago, and "AI summaries take an AI summary of a summary" — and scrub the tells: "it's not this, it's that," the em dashes, "the honest truth," "the load-bearing point". "Using AI is not the problem. It's low quality — that's the fight, at the end of the day." The closing question, on team composition, gets a nuanced answer: classic EPD is largely gone, management layers have flattened, everyone is now their own mini data analyst — and yet "the company feels quite a bit similar". The direction is a software factory and a support factory in which humans design the loop and each model release chips away at the remaining human-in-the-loop steps. Wade's plug: "go install Zapier MCP."

Topics covered

Chapter timestamps intentionally omitted — ad placement shifts them after the edit. Derive from the published cut.

• What building Agents, Chatbots, MCP, SDK, and Guardrails taught Zapier: people pick a daily driver; headless wins
• Wade's setup: Cursor plus Zapier's internal harness over MCP
• AutomationBench: ~600 tasks, GPT-6 Astra as SOTA at ~40%, Gemini 3.7 on cost; why nobody wants one model
• Deterministic code beats agents for most work; AutomationBench V2 will measure the tool lift
• "No-code is code": the agent builds, visualizes for humans, and troubleshoots
• The five-agent support jury and the thumbs-up/thumbs-down-with-a-reason loop
• Defensibility: fifteen years of automation data
• Competition: PG's "what would happen if Google built this?", labs as big tech, the sea of sameness, non-adoption as the real rival
• Why the world hasn't changed more — "life is kind of the same" in Missouri
• Watch-what-I-do automation from event streams; APIs vs. screen recording
• Pricing: seat-based is dying; usage vs. outcome; token subsidies and the Microsoft bundling analogy
• Why the chief people officer owns AI transformation; ~100% AI fluency; match the leader to the bottleneck
• Public-by-default Slack: transparency value, agents need context, what stays private
• Security: credential stewardship, Mythos-caliber models, the Hugging Face incident, early access as differentiation
• $30K/month in tokens: outliers, self-policing dashboards, budgets tied to fluency
• AI writing norms: own it, understand it, make asks explicit, verify, scrub the slop
• Team composition: EPD gone, flatter management, everyone a data analyst, the factory model
• Closing plug: install Zapier MCP

Resources

Zapier

• Zapier MCP — install docs
• Zapier Agents · Chatbots · AI Guardrails · SDK & developer docs
• AutomationBench leaderboard · launch post · public task set · paper
• Zapier's values — "default to transparency"
• Zapier's AI fluency rubric
• Brandon Sammut, Chief People & AI Transformation Officer
• Wade's post on top builders hitting $30,000/month in tokens and the Forbes piece it responds to
• Zapier at Y Combinator

Models & harnesses

• GPT-6 Astra · Gemini 3.7 Flash · Claude Fable · Claude Mythos
• Cursor · Claude Code · Claude Cowork · ChatGPT Work · Microsoft 365 Copilot · NotebookLM

Competitors & peers

• Tasklet — Three Kinds of Software Survive, with Andrew Lee (CogRev, May 2026)
• Composio — CogRev episode with CTO Karan Vaidya (March 2026)
• Zero.xyz · Salesforce Headless 360

Security

• Project Glasswing (Anthropic)
• The Hugging Face incident: Wikipedia overview · Hugging Face disclosure · OpenAI's account

Other

• Pangram — AI-text detector
• Sam Altman, "The Gentle Singularity" — "intelligence too cheap to meter" (Wade paraphrases the utility framing)
• United States v. Microsoft — background for the bundling comparison
• Wade's first appearance: Zapier's AI Revolution: From No-Code Pioneer to LLM Knowledge Worker (September 2024)

Mentioned without a citable source

• Paul Graham's "what would happen if Google built this?" YC-era advice — Wade's recollection
• Zapier's internal daily-driver harness and the five-agent support jury — described on tape, no public write-up

Quotes worth pulling

"The idea of building in no-code, it feels antiquated to me. It's like the new code — no-code is code."

"The vast majority of what people are using an agent for, in fact, they probably should be using old-fashioned deterministic code. You really only want the AI to reason over the things that you need it to reason for."

"The reality is, OpenAI, Anthropic — they're big tech now. These are not small, tiny startups. Yes, they're obviously capable of building incredible things... but they can't build everything. They just can't."

"You're not going toe-to-toe with Larry and Sergey. You're not going toe-to-toe with Zuck. Oftentimes, the things that entrepreneurs are trying to build, you're going toe-to-toe with a potential mid-level product director who's trying to get a promo..."

"The market for automation is like orders of magnitude bigger than I ever thought it was when we started the company, 15 years ago. It's probably a thousand times bigger than what we sort of set out to build."

"I would ascribe to, I guess, the point of view that for the most part, seat-based pricing is dead or dying."

"Most people far overestimate the number of things where that is required, and definitely underestimate the power of what happens when both humans and agents have access to the full context of what a company is working on."

"Every software project has vulnerabilities. It's just, has someone found them yet? The models just make it a lot easier to find those things. But the good news is they also make it easier to patch them."

"I have no problem with people using AI for communication at Zapier. What I really have a problem with is low-quality communications."

"If you wrote it, you probably should be putting more time into authoring the thing than the reader is reading it."

"Using AI is not the problem. It's low quality — that's the fight, at the end of the day."

Corrections & clarifications (to confirm against the final cut)

• "I was on 18 months ago" — the first episode published September 26, 2024; the gap is closer to two years.
• AutomationBench numbers — Wade's "about 40%" for GPT-6 Astra refers to the official leaderboard, scored on a held-out private set. The public GitHub set ranks models differently (its README lists Claude Opus 5 at 50.3% on top). Say which set before quoting a number.
• Headcount — Wade says "an almost 800-person organization"; third-party trackers show higher. Use his figure, attributed to him.
• The Pangram exchange — Nathan framed it as "not a gotcha" and volunteered his own scores first; preserve that framing.

Sponsors: TBD — not yet confirmed. Do not infer from the transcript.

Sponsors:

Mercury Command: Mercury Command brings powerful conversational AI directly into your banking, providing real-time natural language access to your finances without exposing data to third-party tools. Learn more and apply online in minutes at https://mercury.com

Athena: Athena matches you with a dedicated, top 1% executive assistant to handle your inbox, calendar, and daily workflows so you can save an average of 15 hours a week. Get matched with your EA today at https://athena.com/cognitive

OutSystems: OutSystems is the leading agentic systems platform that enables enterprises to engineer, orchestrate, and govern AI applications on a single unified platform. Learn more and see how it works at https://outsystems.com/tcr

Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr

CHAPTERS:

(00:00) About the Episode

(03:33) Sponsor: Mercury Command

(05:21) Daily drivers and MCP

(10:20) Benchmarking automation models

(18:01) Replacing no-code with code (Part 1)

(19:31) Sponsors: Athena | OutSystems

(23:02) Replacing no-code with code (Part 2)

(30:34) Navigating market competition (Part 1)

(34:01) Sponsor: Claude

(35:36) Navigating market competition (Part 2)

(39:30) Passive workflow discovery

(46:09) Internal AI transformation

(55:46) Managing tokens and slop

(01:03:37) Episode Outro

(01:05:59) Outro

PRODUCED BY:

https://aipodcast.ing

SOCIAL LINKS:

Website: https://www.cognitiverevolution.ai

Twitter (Podcast): https://x.com/cogrev_podcast

Twitter (Nathan): https://x.com/labenz

LinkedIn: https://linkedin.com/in/nathanlabenz/

Youtube: https://youtube.com/@CognitiveRevolutionPodcast

Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431

Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Transcript

This transcript is automatically generated; we strive for accuracy, but errors in wording or speaker identification may occur. Please verify key details when needed.

Introduction

[00:00] Hello, and welcome back to the Cognitive Revolution!

Today, I'm excited to welcome Wade Foster, co-founder and CEO of Zapier, back to the show. When I last spoke to Wade, in September of 2024, some 400,000 customers had used Zapier to delegate more than 100 million tasks to AI, and YC president Garry Tan was calling Zapier “the AI-powered knowledge worker of the future.” Since then, models have of course become dramatically more capable, and Zapier has built out a full AI portfolio, including Agents, Chatbots, an MCP server, an SDK, and an AI Guardrails product, and yet… White collar work, and the world as a whole, have changed much less than I would have expected. With that in mind, I wanted to hear not only about what Zapier has built, and how it's continued to evolve as a company, but what Wade and team have learned about how businesses across the economy understand and use AI tools. At a high level, Wade believes that most people are settling into using a single "daily driver" – whether that's Claude Code, ChatGPT for Work, Grokbot, or, in Wade's case, Cursor – and that platforms like Zapier will need to adapt by making their tools available and effective in those environments. Practically, he observes that models still struggle with many business tasks, as illustrated by Astra setting a new high of just 40% success on Zapier's Automation Bench, which consists of roughly 600 knowledge-work tasks across marketing, sales, HR, and operations – and he argues that many tasks that people are delegating to AIs would be better done with deterministic code, and that, for a while longer at least, there is tremendous ROI to time invested in structuring and validating workflows. We go on discuss what Zapier is doing to help people recognize exactly what AI might be able do for them, starting with his own weekly automation, which Wade says they will soon productize for customers, that reviews his activity across Gmail, Slack, the browser, Cursor, and more – and then proposes specific tools and workflows he should build. We also discuss: - how Zapier is implementing recursive self-improvement loops internally, and how much value they find in running multiple different AI models on the same problem; - why Wade chose to put Zapier's Chief People Officer in charge of AI transformation, but wouldn't necessarily recommend that strategy to other companies; - how Zapier is moving toward "public-by-default" communications to make more and more context available to AIs; - why they still don't limit individuals' use of AI, but have created dashboards to help employees better understand and manage their own usage; - how Zapier, which holds a huge number of high-value user credentials, is thinking about security in the context of rapidly rising cybersecurity risks; - and finally, how Wade thinks about co-authorship between humans and AIs, with the upshot being that he believes individuals should use AI to help improve their writing and work generally, and shouldn't be afraid of being Pangram'd, but also that it's critical that people be prepared to explain and stand behind the work they ship. With that, I hope you enjoy this very grounded and highly practical conversation about making AI automation work for people outside the AI bubble, with Wade Foster, co-founder and CEO of Zapier.

Sponsor

[03:33] Mercury Command: Mercury Command brings powerful conversational AI directly into your banking, providing real-time natural language access to your finances without exposing data to third-party tools. Learn more and apply online in minutes at https://mercury.com

Main Episode

[05:21] Nathan Labenz: Wade Foster, CEO of Zapier. Welcome back to the Cognitive Revolution.

[05:27] Wade Foster: Yeah, thanks for having me again, Nathan.

[05:29] Nathan Labenz: Boy, what a difference a not super long period of time makes. I, you know, I can't believe it. Every time I have a returning guest, it's an opportunity to look back and look at what the state of AI was, models, outlooks, you know, what the, what the possibilities were, what the capabilities were at the time. And suffice it to say, obviously, a lot has changed. We only have one hour today, so I'm gonna try to discipline myself not to talk too much and give you most of the airtime. I would love to start off with just, first of all, an observation that like many ambitious software companies, you guys at Zapier have been really prolific, and I would say you've kind of built everything in the sense that you now have, in addition to workflows, which of course can call out to AIs, you've got agents, you've got chatbots, you've got an MCP, you've got an SDK, you've got a guardrails product, and probably more, you know, that I didn't even notify, identify or mention. What have you learned by building all that stuff? Like, what has taken off and resonated maybe better than you thought? What has been slower than you thought, and what's the sort of synthesis view that you have informed by these different product efforts and their relative successes?

[06:37] Wade Foster: Yeah. Well, I mean, shoot, I was on 18 months ago, and if I think back to that time, what I don't think we knew yet is what the shape of these AI experiences that were gonna take off. Were we gonna see, you know, AI infuse into, like, all the products that we already knew and love? Were we gonna see these new products shape, take up? Were we gonna see new stuff out of the labs that was, that, that was gonna be where AI was gonna take off? And what, what would seem to have happened, at least when we look at our own usage, is that most folks seem to have adopted their own daily AI driver tool. Maybe this is Cloud Code, maybe this is Cursor, maybe this is ChatGPT, you name it, but it's, that's kinda where people do most of their work. And by and large, the way they... What that means is tools like Zapier MCP or any MCP server, tools that bring your context, bring your data into that person's daily driver, that feels like the way knowledge work is moving. The split in the market where there's some folks that are still trying to be like, "Ah, we're gonna force people to build agents on our own platform," or, "We're gonna bring them over here, and that's kinda where things are gonna get done." And then you see, like, the Salesforces of the world that are like, "We're headless. You can bring it wherever you want," et cetera. And I definitely think that latter is the winning strategy. It just feels like that's what us as consumers of these tools want, and that's where all the growth is. We kinda want our daily driver, and we wanna bring that context in, and we wanna be able to manage it all in one place. Doesn't mean that there's not gonna be tools that call out to all these third parties. I still think there's plenty of room for applications and tools to exist, but you kinda have to integrate with that person's core daily driver if you really wanna be a part of their day-to-day workflow. And that feels like a pretty big new learning for me in the last 18 months.

[08:30] Nathan Labenz: Yeah. That's interesting. Do you... So does that mean then, like, what's your daily driver? It sounds like you're kind of positioning Zapier not as being a daily driver, but more as being, like, an uber tool for whichever daily driver you choose to use.

[08:46] Wade Foster: Yeah. So I mostly use Cursor every single day. We have a internal tool that a lot of our employees use as a sort of daily harness, but that also has an MCP associated with it. And so I have that pulled into Cursor, so it's using the data from that, which has a virtual file system that acts as a context layer. It has automations that sit on top of it. It has apps that you can deploy, all the kinda things that you might expect from, like, a modern AI capable tool. I just happen to use a lot of that stuff inside Cursor.

[09:17] Nathan Labenz: Very interesting. So okay. I, I had a conversation with Andrew Lee from Tasklet. He advanced the, I thought, provocative idea that in his mind, only three kinds of software companies survive in the big picture, and that kind of everyone is building the same thing, which sometimes gets described as, like, the mecha suit for models. So he described himself as kind of trying to earn a place in a, you know, eventual winner's circle of this, like, horizontal layer that sits on top of models, enables them greatly by pro- like providing all these different tools and access points and guardrails and, you know, whatever else the, the case may be. And he thinks that there's, like, not a huge number of companies that win in that case, but that that layer, even if there's not too many companies ultimately in it, is super valuable because nobody wants to be beholden to just one model. They don't wanna be overly locked into a single model provider. How would you compare and contrast your worldview against that summary?

[10:21] Wade Foster: Well, I definitely agree with that last statement. I think it is becoming more and more obvious that these... You don't wanna be beholden to one model or one company's suite of models. We see this with our own Zapier's Automation Bench. We have a benchmark that measures all these models on automation tasks, and last week, Astra came out, GPT-6. It's the new state of the art on that model. It performs about 40% of the tasks accurately, which is the highest that there is. Now, it's more expensive than, say, something like Gemini 3.7, which is, does pretty good but does it at a fraction of the cost. And you kinda have this, this curve that sort of exists where you're trying to figure out w- how much am I willing to pay for incremental, you know, performance on certain tasks? And as a result, I think any modern organization wants the ability to make those trade-offs, to say, "Oh, these tasks, it's good at a hun- you know, I, I can sort of pay this rate and get 100% of these types of tasks completed, but for this type of task, I need to maybe move to a state-of-the-art model to do well on it," so on and so forth. And so I do think that- Companies are gonna look for the, I don't know, you called it the mecha harness or whatever, the sort of tool that allows them to swap in and out for different workflows of their choice. Which I... To that end, I certainly believe that there's a lot of innovation yet to be had on the application layer. But how those applications are used I think looks very different than the last decade. The last decade was SaaS, and there's, like, all this sort of explosion of SaaS, but now it feels like there's this almost like explosion of headless tools that is happening. And of course, you have this new thing which is that your harness itself can build some of those tools. And so the, the sort of build option is, uh, more readily available than it was in the past. Now what's not as obvious to me is yes, it can build it, but should you have it build it? Because now you're accepting a certain amount of maintenance, a certain amount of reliability, a certain amount of uptime that, you know, may not actually be, like, the best thing for you in a given circumstances. And so I actually think there's still a lot of innovation to happen at that application layer that we just haven't seen yet. I think everyone's kind of trying to figure out what that looks like. I think the, the model companies have a little bit of a head start, but they're gonna struggle because they can't sell tokens from each other or from open source or all that sort of thing. And so th- it does feel like there needs to be a third party that helps you wrangle all the capabilities that are out there.

[12:53] Nathan Labenz: Obviously Zapier came from a history of, like, very structured workflows because if you didn't fully encode what the workflow was supposed to do, there was no ghost in the machine back when you started to figure it out on the fly. Now I'd be curious to hear a little bit about, like, how would you describe the tasks of Automation Bench and, like, where are the models good, where are they not good? And then maybe you can describe how usage of Zapier is changing qualitatively. Like, how often are people still doing box by box, you know, defining of workflows? How often are they prompting an AI which then turns their, you know, kind of thoughts into a structured workflow? How often is it happening through an MCP where it's the model deciding, you know, to even use Zapier given a range of options? And maybe there's even more there that I'm not intuiting.

[13:46] Wade Foster: You bet. Yeah, what Automation Bench measures is tasks that are kind of like the following. So here's an example from our site. We just closed the Meridian Core platform deal. Uh, market it as won and route it to the win notice in the right team per our routing policy. Confirm the accounts here from the account hierarchy spreadsheet. Convert the currencies if needed, and check for any open support escalations. So we have probably 600 tasks that are of that variety that describe like a normal knowledge work workflow-y task across a very variety of disciplines, marketing, sales, HR, operations, you name it, et cetera. So that's kind of what it's trying to measure. And as you can see, the models are getting better at it, but this is by no means a saturated benchmark yet. And what we are doing at Zapier is we are making sure that when you install Zapier alongside of your agent, that you're actually getting better output on these benchmarks than you would if you were just using the models alone. And the reason you do that is you're teaching the model... What folks are doing is they're in their daily h- driver and they're saying, "Hey, I want you to go build a workflow." So they're calling Zapier MCP and it's gonna say, "Hey, I'm gonna go build out that workflow, and in some cases I'm gonna write code to actually complete that task so that it is deterministic," which means lower cost, better reliability, et cetera. And then I'm gonna invoke an AI or build an agent for the parts that really require reasoning. And when we look across, like, even our own a- like, in customers' usage of agentic products, the vast majority of what people are using an agent for, 80% in fact, they probably should be using actually old fashioned deterministic code. You really only want the AI to reason over the things that you need it to reason for. And so I still think there is a huge amount of work that is happening inside of these production workflows inside of a company that you shouldn't actually try and be delegating it to an AI. And we'll see how long that lasts. Obviously AIs are getting better and better. But I struggle to think of a world where there are just certain jobs that, like, deterministic code is still gonna be more reliable and cheap. And there are certain jobs that it cannot do, and for those you need the ghost in the machine, you need the AI that can tackle those tools. And so I think the, the right thing is you're trying to teach the agent how to go do that on its own. And so when you talk to it, it goes and builds those things in an optimized way instead of saying, "Hey, I'm just gonna build an agent that runs agentically every single time." It has a better sense of which is the right tool for which job.

[16:16] Nathan Labenz: So in that one example that you gave, if I understood it correctly, first of all, it sounds like there's probably a ton of context that comes in the sort of test environment with that short prompt, right? It, it has to navigate, find the policy and parse that, and all those other things kind of require additional information finding and understanding. Also struck me that that sounds like the kind of thing that only happens once. So, or, you know, you might acquire a handful of companies, but you're not gonna acquire, like, the volume of companies that one would a- traditionally associate with a zap. So, like, how are you seeing usage change, or maybe what advice would you give if you're like, "Okay, I used to think that for me to go to all the trouble to make a zap for something, I need to at least expect that zap to run, I don't know, what, 500,000 times."

[17:07] Wade Foster: Well, that example is one... Maybe I misspoke, but that example is one that happens every day if not multiple times in, inside of a company. You closed a deal. If you're any good organization, you're closing deals all day, every day. There are so many workflows inside of a company that are kicking off all, all the time perpetually, and in some cases they're happening at a rate that humans can't keep up with. If you're operating at the scale of, like, some of these companies, they're c- they're selling to consumers that are having purchases happen, like, many times a second. You have to use automation. You have no other choice. You cannot put humans in the loop for these tasks.

[17:41] Nathan Labenz: So have you seen big shifts in terms of people moving away from blocking it out themselves and having AI do that? And have you also seen the scale threshold at which people start to think to use automation software come down substantially because the AI can do the setup?

[18:02] Wade Foster: Yeah. I think the big new, uh, like opportunity is to have the AI do the build. Like, you know, it is able to sort of work through the logic much, much faster than a human, and the idea of building a no code, it, it feels antiquated to me. It's like the new co- no code is code. What I think is we have learned is that humans still very much benefit from visualization of those workflows though. That visualization helps them, uh, verify is this thing doing what I intended it to do? It acts as documentation, so you can share with other people and say, "Hey, here's the thing I built. Here's what I'm doing." I- if you think of Zapier in the old school as like this thing that had a bunch of boxes and you're coming in and using that to configure it, by and large, I don't think that is the way people are doing it now or even in the future. Instead, they're talking to the agent and having the agent go make those edits for them. And then the other nice thing that i- is happening in the future is you're gonna see that stuff that gets hardened into a deterministic workflow, but even when it fails, you can actually have then the agent go troubleshoot and it's fall back and say, "Why did it fail? What happened here?" And then it can use its reasoning to f- to fix the, to fix the workflow or to fix that instance of the workflow. And so you start to see the, the agent almost take over the human role of building and maintaining it, but what is actually running is still a very deterministic workflow with AI interwoven in like the places that is most necessary.

Sponsor

[19:31] Athena: Athena matches you with a dedicated, top 1% executive assistant to handle your inbox, calendar, and daily workflows so you can save an average of 15 hours a week. Get matched with your EA today at https://athena.com/cognitive

[21:03] OutSystems: OutSystems is the leading agentic systems platform that enables enterprises to engineer, orchestrate, and govern AI applications on a single unified platform. Learn more and see how it works at https://outsystems.com/tcr

Main Episode

[23:02] Nathan Labenz: So when you go to the 40% or so success rate on Automation Bench, what can you bring that up to as you move from, you know, just asking Claude to do it to giving Claude Zapier and, you know, potentially iterating a little bit on initial failures? Like what kind of lift do you see and, and what sort of token savings do you see over time?

[23:29] Wade Foster: You're gonna have to wait for Automentio- uh, Automation Bench V2 for that, 'cause that is exactly the right question, is to see, hey, what happens when you give these models access to tools like Zapier? How much higher can you get those efficiency and... or how much higher can you get the, the scores, but also how much can you pull t- costs down? How much faster can they go? Because I think there's many dimensions when you a model access the tools and capabilities and certain things like that where it's gonna score better on these benchmarks at the end of the day. And that's where I think the application layer has a lot of room to go run, is to say what happens when you give access to the model to these extra things? And it... the model just gonna get better at performing against a, a whole host of tasks.

[24:08] Nathan Labenz: How do you put the model in position to be successful when something goes wrong and the model has to come in and like troubleshoot, debug? You know, there... I, I'm sure that you've done who knows how many things over time to try to set the models up for s- you know, within one generation of model, have it come back and like be able to fix the things that it got wrong the first time.

[24:30] Wade Foster: Yeah. I think there's two places that are interesting to talk about here. First is when you think about building these workflows. So the, the thing you wanna do there is you wanna make sure that the model can properly identify when it needs to be using AI versus where it should just be writing code. Uh, and there's many such examples where you don't want the agent to actually orchestrate the task. You just actually want it to run, uh, code that, that i- that exists before. And so a lot of that is when the, the, the agent is building its plan, you wanna make sure that the plan encodes like, hey, this is like what the optimized workflow looks like, and it has a good plan to do that. The second place is then what happens when things break, and how do you recover from that? Here we've done a fair amount of work just even in our own support org trying to figure out how do we actually troubleshoot on behalf of customers, and then how do we bake that troubleshooting back into the core product? And here a lot of it just boils down to w- one of the big learnings is we just have multiple agents run at it, and our current, we have this auto email program that's running right now, and for that it spins up five agents, independent agents that evaluate the troubleshooting situation. And we notice that when four of the five agents tend to agree, there's a pretty good chance that's actually the, the issue that it's hitting into it. And there's a lot of like data and measurement that just goes into that where you're just trying to honestly just hill climb and see like how do we actually try a different model, try a different prompting technique, how do we measure that stuff when we have humans that are editing, like auditing the output. And so when it fails to... And what those humans do is they basically either give a th- thumbs up approve or they give a thumbs up rejection and a reason why. And so those reasons why kick back in and help improve the overall system. And so you're just kind of going through this loop over and over again to continue just optimize your ability for that workflow to have a higher chance at getting the outcome that you want.

[26:26] Nathan Labenz: Is that loop and all the data that you've collected over time, which I guess must be quite massive, core to Zapier's defensibility these days? Like how do you think about what the, you know, what, what is the hill that you've climbed that will be other, be hard for others to follow you up?

[26:46] Wade Foster: Yeah. I think that's a big part of what it boils down to. It's trying to identify what are things that are unique to you, what are the things that others can't replicate easily. And so certainly for us, we're really good at automation. We have tons of data on what it takes to do. We hook into everything. And so how do we take that data to actually make our products better? And it has to be meaningfully better than somebody who doesn't have access to that could do. And I think this is where a lot of the incumbents have an advantage is if they're able to yield That to actually build a better product at the end of the day that the models alone can't do. And there's so much room for this because the models are, they're great generally out of the box, but all of us even experience this in our own just lives where you just, you hooking up your Gmail inbox, and now you start asking like, "Hey, tell me about... Help write an email." And it's like, it automatically does a better job of writing email because it sees how you write email. And you've done very little in the way of trying to tune that workflow. It's just by hooking up your company's data, all of a sudden the models get better. And yeah, I think this y- using your own sort of company data to build an edge is a pretty spot on technique these days.

[27:52] Nathan Labenz: So does that look like a big retrieval problem for you? Certainly in my email, right, it's like I've got a lot of history, and finding the right example to take inspiration from is probably, especially if it's just using Gmail APIs and doing keyword, you know, kind of constructions, that's probably just as hard, if not harder, than actually taking inspiration once you've found the right documents to take inspiration from. In my personal context, I've tried to help it out by, like exporting all that stuff, doing embeddings, various kinds of, you know, alternate search approaches so that hopefully the right content comes to the top more often. If I'm putting two and two together right, it sounds like at Zapier you probably have like a database of like a zillion things that have gone wrong over time, and then to help the inform the model of like how to fix for this particular situation, you gotta dig in and find analogous situations. What does that look like? Or is there an embedding model that would do a good job of that, or have you had to kind of innovate at the retrieval stack layer in order to make that work well for such a use case as Zapier?

[29:05] Wade Foster: Yeah. Coming back to the support example, a lot of it's just about taking the example that comes in, give it the old thumbs up, give it the thumbs down, provide a reason why, and just doing that over and over again. And so what that looks for our customers is just like giving them the same tools to do the same. A lot of this is not particularly fancy at the end of the day. It's just take the example, did you like it? Did you not like it? And just rinse, wash, and repeat.

[29:30] Nathan Labenz: Interesting. Okay. How do you think about competition in general? Like, it sounds... On the- on the one hand, we should, I think, all be worried about frontier model companies eating our lunch. Even me as a humble, you know, AI podcaster, I look at like NotebookLM and I'm like, "They're coming for me in my, you know, rather unlucrative niche." But you could, you know, say, well, those guys, they're only gonna sell their own models, so they're kind of a different type of animal. We don't have to worry about them. There's like a variety of new tools that are coming online to try to be the Uber tool. I've done episodes with Composio, for example. ZeroXYZ is kind of out there. And then there's like, you know, kind of other daily driver sort of agent builder type things. And then there's just kind of like other big incumbents, you know, that you've mentioned Salesforce, and it's, to some degree it's like maybe just big incumbents with lots of data, lots of resources kind of all end up colliding with each other. Like, which of those kind of classes of competitor do you think are actually the ones that you need to be most concerned with?

[30:35] Wade Foster: You know, it's a interesting... We're in an interesting period for sure. I think to your point, everyone gives a lot of attention to the labs and tries to understand what are they doing. And I think that is important. You wanna understand what are they gonna be great at, what are they gonna hill climb at. But I still remember PG's advice when we were going through YC where, you know, back in the day it wasn't, hey, what if Anthropic or what if OpenAI builds you. It was what would happen if Google built this? What would happen if Facebook built... That was always like the question. And the thing that PG tried to instill in folks was you're not often competing directly with Google. You're not going toe-to-toe with Larry and Sergey. You're not going toe-to-toe with Zuck. Oftentimes, in the things that entrepreneurs are trying to build, you're going toe-to-toe with a potential mid-level product director who's trying to get a promo- promo, might be there for two years and then bounce, et cetera. And the reality is OpenAI, Anthropic, they're big tech now. These are not small, tiny startups. Yes, they're obviously build, capable of building incredible things, and they're gonna be best at the world at these foundation, the- these models. They're gonna be incredible at that. And they will have good products elsewhere, but they can't build everything. They just can't. And so that's where I think, you know, it gets really a wide open field. And I look around and it is a little confusing because you've got everyone that does seem to be building everything. There's a, there's a sort of sea of sameness out there that is a real challenge at the moment. While at the flip side, you go talk to the average user of AI tools, and they're candidly not doing much. They might have used ChatGPT or Gemini. And so to me, I think most of us, our competition isn't each other. It isn't the, the tools that you talked about. It's do people actually know what to do with these tools yet? They just haven't adopted anything at this point in time. And the real challenge is can you actually get your hooks in somebody whose only experience with AI is using Gemini in a default Google search? Uh, and or if they're using Microsoft Copilot at work. That, that's where most people are. And I think that can get so easily lost in the shuffle if all, if you hang out on X all day, because on X we're all just hyper aware of what model came out, what tool is gaining traction, who just raised a huge bunch of money, and we're keenly aware of these micro differences between different products, and most folks just don't know that. And I think the, the challenge we have is really making sure we're sort of keeping like those two- ... competing challenge, like those competing thoughts in our head, which is yes, we do have to be better at some dimension than like all of this sea of sameness. And yet at the same time, the opportunity is massive to just go educate the masses on like how these tools can work. And there's pl- these markets are enormous. The market for automation was like orders of magnitude bigger than I ever thought it was when we started the company 15 years ago. It's probably a thousand times bigger than what we set out. And so there's plenty of room for us to go solve problems for customers who candidly they don't know about any of the competition you just y- rattled off. And so I think that's the biggest challenge for many companies today.

Sponsor

[34:01] Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr

Main Episode

[35:37] Nathan Labenz: Yeah. Certainly this... I, I'm working a little bit on a episode where I'm just gonna do things I've been wrong about and things I've been right about, and one of the things I've definitely been wrong about is I expected a lot more change to how things get done across the economy five years ago than we've actually seen. And if you, especially if you were to tell me then that like we'd have Astra and, you know, that it would've been roughly speaking like a smooth ramp to these capability levels, and yet we still see, you know, revenue has exploded at the model companies obviously, but like we don't see nearly as much change as I would've guessed.

[36:13] Wade Foster: Yeah. Life is kind of the same. I kind of go about my day and I talk to my friends, I talk to my family, and I watch what they do, and yeah, it's, it, their days are the same. And yeah, they use ChatGPT some to help with things here, meal planning or to help plan a vacation or to, to do a workout or something like that. Certainly not taking advantage of Astra level model capabilities-

[36:34] Nathan Labenz: Yeah

[36:34] Wade Foster: ... at all.

[36:35] Nathan Labenz: So what have you learned then about what kind of help people need? You know, if you, I don't know, you do, do you like talk about this on a, when you're taking walks in your neighborhood? H- how do you... I'm interested in your method, but also m- even more so your takeaways. What is it that kind of gets people over the hump? How do you help them see a new way of working? Are there any patterns that, you know, seem generalizable, or is it kind of idiosyncratic for every individual and small company?

[37:07] Wade Foster: There's definitely patterns, but there's a lot of the specifics matter a lot, and that's where it starts to feel idiosyncratic. And we've seen this for years. I remember for a long time one of the hardest problems with Zapier, even till this day, has been helping people with recommendations. What actually should you use this stuff for? And if I personally sat down next to you and said, "Hey, just show me what you do every day," I could come up with half a dozen examples of things that would immediately be things you were like, "Yep, I want that. Yep, I want that. Yep, I want that." But then how do you actually bake that in... Like, how do you give that experience of me or somebody who's knowledgeable about these areas sitting next to them, and how do you bake that into the product? And this is where I get pretty excited about where AI think can help cross that, I don't know, recommendations gap, use case gap, whatever you wanna call it. Because if you're able to point the tools at where you work, say, "Hey, I want you to go watch what I do every day. Watch what I do in Gmail. Watch what I do in Slack. Watch what I do in my browser. Watch what I do in my chat. And just tell me, what should I be doing different?" I started doing this workflow at the beginning of the year, and pretty much every week now I have new tools, new systems that start to automate bits and pieces of my job. And if you do that on a perpetual basis, you start to feel the difference. After a month or two you're like, "Wow, I, uh, there's a lot that's kind of running for me now that I didn't have before." And I think even though many of the things that are built are things that I kinda knew I should be doing before, it's the, it's the specifics of where it's like I literally watched the thing that you did here and here, and so I know exactly the tool that can get it done. It's that idiosyncratic part that makes it click in. And so I get pretty excited about how do you build, uh, how do you give these models like awareness of just how people go about their day? Because I think there's tons of ideas that are just kind of trapped, like latently lost inside of that, that most of us just don't wake up and think about. We are creatures of habit, and so we wake up and we go about our day the same way we did but the day before and we don't think, "Oh, there might be a 5% better way of doing it," or, "There might be like a 500% better way of doing this." It's just I have muscle memory. I know how to do it this way, and I'm comfortable doing it this way, so I'm gonna keep doing it that way even if it's not the best. And so AI sort of has to be so good that it kinda knocks us out of our comfort zone and we're willing to say, "You know what? I am gonna go try it that other way because that sounds so, so much better."

[39:31] Nathan Labenz: So how have you set that up for yourself, and how broadly deployed at Zapier is this sort of AI on your shoulder or screen record? I'm kind of, uh, expecting like a screen recording.

[39:41] Wade Foster: Most people, yeah, most people use Zapier MCP for this, so they just have all their tools hooked up into whatever harness of choice they use. Like I mentioned, we have our own internal hard- harness, but it could be Cloud code, it could be co- co- Cowork, it could be ChatGPT work, it could be Cursor, it could be anything, right? And they'll just have an automation that runs once a week, and it just kinda collects all these signals across all the work they've done. It seems like the event streams, and it can tell. It says, "Hey, I noticed you did this and this, and here's a tool that I think you should go build." And so most people have something like that set up and then they just tell their agent, "Okay, great, I like that suggestion. Go build it," or, "That suggestion's okay. What would make it great is if you made this tweak and do that. I really want you to go do that." And half, half the battle is just getting you to react to something. Even if the ideas aren't perfect, they only need to be like 50% good enough to get you to go, "Oh, I see where you're going for." And now that brainstorm process kicks off and you're able to run with it.

[40:39] Nathan Labenz: So do I understand correctly that it literally just uses APIs to look at your digital history for the last period of time, and then-

[40:49] Wade Foster: More or less

[40:49] Nathan Labenz: ... kind of collates that together and comes up with ideas? Interesting. Is this something you think you'll productize for Zapier customers?

[40:58] Wade Foster: Yes. I think it's very likely that'll come in some form factor.

[41:02] Nathan Labenz: Interesting. And do you think, I mean, it's a very It's an interesting way of doing it. Obviously, you know, I don't need to tell you, but, like, one downside of going that route with your customers is they'll have to attach all these different platforms that they use first in order for you to have the access to get any, you know, insight as to what's going on across all these things. Whereas with, like, a screen recording type of thing, you know, you have kind of one install and you just, like, look at what they do click by click and sort of, you know, make sense of it from a, a kind of top-down, I guess, perspective as opposed to what you're describing sounds a little bit more bottoms up of like, "Oh, I saw this in Drive and this in Gmail," and, and what have you. Do you think that... Is there, like, a principled reason or a empirical reason for going, doing one or, or the other?

[41:50] Wade Foster: We're good at APIs and we're good at that stuff, and so that was, like, just an easy, natural... Just was an emergent experien- experiment, an emergent property. I think for this experience to be great, you should use all the tools that you have available to you.

[42:03] Nathan Labenz: Yeah. It's interesting. I mean, it also just kind of s- reflects the alien nature of AI intelligence in some ways, where it's like, if I was gonna try to advise you, I would definitely want to watch you work. I would not be so helped out by, like, your logs, but AIs are really good at reading logs. How do you think about pricing in today's world? This is obviously very open question for a lot of companies.

[42:28] Wade Foster: The point of view that for the most part, seat-based pricing is dead or dying. I think it may still make some sense in some small areas, but by and large, when you're, when intelligence is such a core part of these product experiences, I don't see how a fixed seat-based pricing mechanism really makes much sense for that at all. And so inevitably, I think that lands you in some sort of usage-based or outcome-based pricing world. Uh, and I think products end up choosing which side of that fence they land on. I think if you're, you're going more the commodity route, you're probably closer to a usage-based pricing. This is the Sam Altman, "We wanna be a utility that's pa- you can have intelligence on tap," et cetera. Um, maybe if you're, like, a little more enterprise oriented, you're gonna try and say, "Hey, I'm a, I'm gonna price per out- out- outcome." And you see a lot of the, like, customer support tools doing this, where they're gonna say, "Hey, we're gonna, you're gonna, we're gonna bill you for resolved tickets," where they have such a clear demarcator of what success looks like. And so I do think if you can, if the product you're selling has the ability to have such a clear, agreed-upon fixed outcome, I do think that's probably to your advantage. I think the challenge is, at least most of the products right now, it's way more messy than that. It's you kinda stop one, one or two steps shy of truly delivering the outcome. You are, like, part of delivering a piece of the outcome. And so I think that kind of pulls you back into a more pure usage-based model. But either way, I think you kinda got this meter running where it's, it's basically selling work of some portion at the end of the day. And I, yeah, I think that's kinda where a lot of this stuff is going, is we're gonna be having to think through what is our budget for work to be done.

[44:17] Nathan Labenz: Do you worry about price discrimination from the model companies? Of course, you're, I'm sure, well aware of the ratio of tokens that you get with a Claude Max or an OpenAI Pro plan and how many more tokens you get, at least if you max them out, compared to what you can buy with the same dollars via the API. I've asked a number of entrepreneurs this question, and I just am kind of wondering, like, w- would you be supportive of some sort of rule that said, "Hey, you gotta charge everybody the same for tokens so that, like, an ecosystem, you know, has more of a fighting chance," versus, like, with OpenAI giving, like, you know, 20 to one token advantage, you know, that could be hard for third party, you know, value add services to compete.

[45:04] Wade Foster: Yeah. I mean, look, I, I definitely am, like... You know, kinda tend to fall on the side of, like, free markets, and these are companies that have the right to price how they like, but I think many of us lived through and are still living through, like, Microsoft's dominance and how they use bundling to their advantage to box out better products ca- ca- candidly. But because it's all just bundled there, it, like, plays to their advantage and, you know. And, you know, I think this is where, like, countries and, um, you know, folks sort of get to decide, like, you know, what, what do they think is monopolistic practices and what do they think is fair, a fair playing field. Um, yeah, ultimately, I, I, you know, I think I... M- my job is to, like, play by the rules on the playing field, uh, and not necessarily decide, but I, I definitely sort of lean more toward the, like, free market side and say, like, "Hey, if, you know, our, our job is to come up with an edge that sort of helps us to compete on there." Uh, and I don't fault any company for sort of wielding the tools they have in their, um, tool chest to sort of make products work for them, uh, work good for their customers, and help them maximize revenue. That's well within their right.

[46:10] Nathan Labenz: It's been interesting to see who's been willing to bite the bullet versus who has stuck to their free market principles. Kind of topic change toward operations and AI transformation within Zapier. One thing that caught my attention was that if my AI research agent is to be trusted, you put your chief people officer in charge of AI transformation. I believe last time we talked, you had kinda said, "Well, it's not any one person's job. It's kinda my job as CEO, but it's really everybody's job, so I'm not gonna say it's, like, one person's job." What changed, and how did you decide it would be the people officer who would, who would shoulder that burden?

[46:49] Wade Foster: I still agree that AI should be every person's job, should be my job. I think it depends on what stage you're at and what problems you're facing in terms of how you think about who you wanna bear the, like, t- tackle the next mountain, so to speak. And so for our first chapter, like, a big part of Zapier becoming AI fluent was everyone in the company like getting up to speed on how to use these tools. There wasn't an AI committee, there wasn't an AI group where it's like, "Oh, we're there, they kinda figure it out. The rest of you is business as usual." It's like, no, this is important for everyone inside the company. It impacts everything we do, so all of us kinda need to get on that. Now, as time went on, there was a couple interesting things that we started to observe. First, the AI fluency inside the company went up. Basically, we got within a year or so basically of ChatGPT launching almost like 100% of the employee base is using AI day to day. We are not having technical issues like adopting AI. That's not where we're bearing the brunt. What is starting to become one of the bigger issue is how do you take these models from individuals having success to actually using them to solve bigger and bigger production grade workflows across the company? And so the challenges start to look like a lot more like people issues, where it's like, okay, we kinda have to rewrite certain job descriptions. We have to rethink how we do compensation. We have to think about how these teams stand up, where it's like we gotta move this group. We kinda don't need this group anymore, but we actually need more people over there, so how do we retrain and re-skill these folks who have some of those skills but need to learn some new skills? And it turned out Brandon, our chief people officer at the time, was like really good at doing a lot of these things. His team was on the forefront of some of this inside of Zapier. And so the thought for me was just like, "Hey, you're doing a good job at this. Why don't you go help everybody in the company figure out some of these things?" It coulda just as easily been a CMO or a CPO or any, any number of roles. It just, that's how it's played out inside of Zapier, and it's been funny how much I get asked this question now because I think a lot of folks think I have a point of view on, oh, it must be a chief people officer or something like that, and it's like, no, it was really just this was like at this moment in time inside of Zapier, this sort of felt like the best person to go tackle it based on the problems we were facing, and I think that's the way you should do it inside your company. You need to look at what are your bottlenecks, what are your constraints, and go identify the person who is the right fit for that job.

[49:16] Nathan Labenz: Yeah. Echoes of, uh, Ben Horowitz's advice is tough. You've also done an interesting move of really trying to push people toward internal communications being public within the company by default, and I'm interested there in k- on a couple, like, finer points. One, how did you handle historical data? Did you like start that policy at a certain point in time and everything in the past was like left in the past, or did you try to reclaim some of that knowledge, which I assume would be like very tempting to do? And then do you have like different tiers of public as well? You know, 'cause it, it strikes me that like you might not want everyone to know everything, but you might want different groups to have certain, you know, different databases. So I'm just kinda looking for the double click on how you've operationalized public by default.

[50:04] Wade Foster: One interesting thing, Zapier's had this value of default to transparency for, gosh, forever, it feels like. And so by and large, we were already working in public for many such things. And so we already had a culture where there was tons and tons of public Slack channels, and people were like just talking about the projects and their day-to-day in those quite a bit. I think a lot of what we observed was there had been, as, as the company grew, there was some pockets of work though that started to find their way into private channels or private DMs and things like that. I think still by and large Zapier was very much more public than most companies, but you... Slack gives you the out- the readout, so you can see what percentage of stuff is happening in private and public and all that sort of stuff. And we looked at that and thought we could use a little bit of a reminder. And the second thing that also encouraged us to do this is just the fact that our AI agents were just so much more effective when they were able to see the context inside of Slack. And so one of the things we started to do on our executive team was we just had a little fun competition to see, hey, who can put most of their com- communications in sort of a public channel. There was no like, "Oh, you must do this," or anyone under this rate gets a bad review or anything that. It was just literally friendly competition. And we just noticed that, oh, more things can go in public than we realized, and this kinda seems to help the team. People know what's on our mind. Nothing is hidden, et cetera. And so we just started to en- encourage that all across the company. And there's certainly things that we still probably pull into private channels and things like that. If there's an HR incident or something like that, we're not resolving that in a public channel. If there is a critical security vulnerability, like we're not... Those are happening in private channels e- especially while the incident is active. And then once they get resolved, we tend to share out the learnings and things like that. So there's certain topics where, you know, you still need to set up these spaces where you can go resolve them i- in private. But I think by and large, most people far overestimate the number of things where that is required and definitely underestimate the power of what happens when both humans and agents have access to the full context of what a company is working on.

[52:22] Nathan Labenz: So speaking of security, this is obviously top of mind, and as I was thinking challenges that AI might pose to you, like you're holding potentially more credentials to more different services for more different users than just about anyone in the world, right? So I would think this is kind of a scary moment where all of a sudden, you know, where's Bedrock in terms of security? So how are you approaching that, and are you trying to like get into these sort of early adopter Glasswing and other clubs? Do you think that is like actually maybe a big source of differentiation going forward? And how scared should I be about cyber security, 'cause I've got a lot of credentials all over the place, Zapier and otherwise.

[53:07] Wade Foster: Yeah. We've held these credential for 15 years, right? So this has always been like a, an important thing inside of Zapier that we said, "Hey, this, the, the credentials are like a, a thing that we must, you know, treat with the higher- highest of stewardship." And so we've always put a lot of effort into making sure that we do a good job of protecting those for our folks. What feels different this, this time is that you do have like these mythos caliber like security models that are able to patiently just loop over issue after issue and find things. And so that is definitely like i- e- every, every software project has vulnerabilities. It's just someone found them yet. And the models just make it a lot easier to find those things. But the good news is they also make it easier to patch them. And so yeah, I think what smart companies are doing is they're basically wielding them for offense and defense. They're trying to find this stuff faster, and they're trying to resolve them faster. And so I'm not exactly sure how all this is gonna play out. Every day there's kinda funky stuff going on. Obviously the, the Hugging Face incident is one that was like straight out of a sci-fi book, right? But I think for most folks, my, my, my guess is it feels like there's this almost one time like investment to re-acclimate, and then you kinda get back to more steady state of like offense versus defense, like security posture as this stuff moves forward. But gosh, it's gonna be really interesting to see 'cause there's th- every day we're seeing new stuff.

[54:34] Nathan Labenz: Are you like taking steps as CEO to try to make sure you're on the inside of early access lists for new models?

[54:44] Wade Foster: Yeah. I mean, yeah. We wanna have access to the best capabilities as early as we can. I think everybody is i- in the same shoes would want to do the same.

[54:53] Nathan Labenz: Yeah. It feels like that actually could be a pretty meaningful point of differentiation going forward. Like if one company who's gonna hold my credentials is i- in all the clubs and another one's a startup that's not, I mean, that's a, you know, that's a big leap of faith to take on the company that like doesn't have the same kind of access to be trying to find and fix all these issues. In terms of spending, I saw you tweet not too long ago that you have some engineers spending $30,000 a month on tokens at Zapier, and it struck me that like we've kind of been on quite the yo-yo ride recently with token maxing and then, you know, budgets being hit and what do, what do we do about it. What sort of process do you have, or governance do you have for who can, under what circumstances with what approval spend tens of thousands of dollars a month on tokens?

[55:46] Wade Foster: Right now I would say that, that individual, those individuals are a bit of a li- an outlier. But it's still en- encouraged us to start to put, just build some tools to help people do some self-policing. And so mostly we don't have budgets yet set up for individuals, but we do have tools where they can see their spending and they can better understand, oh, what happens when I choose a powerful model versus when I choose a cheaper model on certain workflows, and they can see what those cost differences are. Uh, and we're, as we see people starting to spend a ton and ton on tokens, usually the first reaction is, "I just wanna go talk to them and say, 'Hey, what are you doing? I'm just really curious.'" And in some cases you have folks that are like doing some insanely productive stuff, and in other cases you have some folks who have got a mix of things that are like pretty productive and places where it's, "Oh, you don't need to be using Fable for this or Astro for this. There's a better way to do some of these things." And so when you have a almost 800 person organization, you're, there's just a big education effort in- involved. Over time, I do suspect that token budgets are gonna be a real thing though, and you're gonna have... Part of AI fluency is gonna be that. We're gonna say, "Ah, like this person is gonna get a higher budget than this person because they know how to get higher l- like output from those things." How do you actually operationalize that? I- we're still working through some of that stuff. But it seems pretty obvious to me just looking across the employee base that some people are like excellent at using increasing levels of spend, and some people are just not really thinking about it all that much yet.

[57:17] Nathan Labenz: Another aspect of AI fluency that I'm really curious for your take on is what you think is the right model for co-authorship or co-creation with AIs. And this is not a gotcha because I'm in the same boat where like I've consciously tried to almost like shock exposure myself recently to put some things out that I didn't rewrite every word of, and so my Pangram score at times, you know, says that my stuff is AI. Now I've seen also some stuff in various places from Zapier that has a high Pangram score.

[57:51] Wade Foster: Mm-hmm.

[57:51] Nathan Labenz: How do you think about and how do you set the tone for others at the company? You wanna be using these things-

[57:56] Wade Foster: Yeah

[57:56] Nathan Labenz: ... but we don't wanna be putting out slop. Like what's the line?

[57:59] Wade Foster: The way I think about it is I have no problems with people using AI for communication at Zapier. What I really have a problem with is low quality communications, and we definitely... Where this becomes a bit of a problem is that we live in an era where AI can, like a, a person who is u- exercising low judgment can create a high volume of very low quality stuff very quickly, and that can overwhelm a person when you're so... We've tried to put a few guidelines in place that help people think through ways to go about that. So f- one, you need to own what you send. If you wrote it, you probably should be putting more time into authoring the thing than the reader is reading it. AI use shouldn't be a way of transferring ship- transferring ownership of a task where it's like, "Oh, I was assigned this task, so I now had AI spin a prompt and I said, 'Hey, you now read it and deal with all this stuff and edit all those things.'" That's not a great way of going about it. You need to understand what you send. If someone is gonna start asking you questions and you actually can't, you're like, "I actually don't know what's inside of that," that's not good. You probably ought to be like making asks explicit. So if you need somebody to do something, if you need a decision, if you need feedback or labeling something as a draft and you want feedback on the thing, you need to do so. You, you need to go verify details. It's not uncommon for AI to hallucinate some of these details. Or it, maybe it's not hallucinating. It might pull dated information. Uh, so if you hook it up to an agent that has access to Zapier Slack or da, it could pull, oh, this project from three months ago is related but not the exact same thing. And if you're trying to pass that off, that becomes a real issue where these AI summaries take an AI summary of a summary and of a summary, and all of a sudden before you know it, it's actually passing on incorrect information. And so you have to do a good job of verifying the details in there. So to me, that's the real important piece is that you are still an active participant in the creation of the material. But if AI's helping you structure your thoughts and structure the writing at the end of the day, go for it. I don't have any problems with that. It can be tedious if you are not scrubbing some of the slop that are a part of it. The it's not this, it's that, the em dashes, the honest truth, the load-bearing point, all that kind of stuff. I do think that is something that if you're doing that a lot, especially if you're doing it in marketing material, it makes it hard to stand out. People get a little tired of reading that kind of stuff. So you do still need to have your own editorial hand on the steering wheel, so to speak. But I- to me it's not, using AI is not the problem. It's like low quality. That's the fight at the end of the day.

[1:00:33] Nathan Labenz: How has your team composition changed over the last couple years? You kind of mentioned earlier, like maybe we don't need this team, but we can re-skill. And are there any thresholds for AI capability, something that you're like, "Well, they can't do this now, but if they could, I could see that actually making a big impact on our hiring plans going forward from that point"?

[1:00:58] Wade Foster: What's interesting, I would say in some ways our team looks very similar to how I, it has in the past, and that's maybe a surprise to me. But in other ways it is pretty different. I think we still have engineering and design and product and stuff like that inside the organization, but the idea of like a classic traditional EPD, like that's largely gone. Um, you know, things are a lot more malleable, but the roles still exist. Um, you know, there's definitely been like a flattening of management layers, uh, but strong management is still crucial. We're not getting rid of managers anytime soon, but managers can handle a higher volume. Those are like a handful of things that are like interesting. Similarly, data analysts. Everyone inside of Zapier is kind of their own mini data analyst now, and so you don't need, quote-unquote, as many data analysts, and yet we still have analysts that are doing really critical, important work inside the company. It's just they're working on higher value stuff now. So the, it's this, you can feel like things shifting and yet at, in some ways it still feels pretty familiar at the same time. So the, the company, uh, feels quite a bit similar. But I, what I think is gonna be... You asked what are the models not yet capable that I'm excited of is I, I, I still do this idea, you know, the team shifting more into building the factory that builds the products and the company and the marketing, all that sort of stuff. And so you can start to feel where we are doing more and more of that, where it's like we have a software factory, we have a support factory, we have these workflows that are getting stood up where they're handling the inner loop and the humans are more designing that piece of the puzzle. Inside those factories, there's all sorts of steps that are like you, you come across areas where you're like, "The AI's not quite good enough for this yet. We need a human in their loop." But with every model release, with every just own iteration learning loop inside of Zapier, you can start to feel us chip away at that problem where you're like we're just getting closer and closer to something that looks actually pretty different than the orgs of the past, and that's pretty exciting, I think.

[1:02:58] Nathan Labenz: Anything else you would wanna leave people with? Anything I should have asked but didn't, or just inspirational closing thoughts?

[1:03:05] Wade Foster: If I were to plug something, I'd say go install Zapier MCP. You get a wholly modern experience of Zapier. So if you still think of Zapier as the old school, no code, clicky click boxes on a screen thing, I think you're in a, in for a treat. If you install Zapier into any of your favorite harness and try it out as your daily driver, you're gonna get a whole bunch of capabilities that you don't get with just the harness out of the box.

[1:03:29] Nathan Labenz: Zapier MCP. Install it, folks.

[1:03:31] Wade Foster: Mm-hmm.

[1:03:31] Nathan Labenz: Wade Foster, thank you for being part of the cognitive revolution.

[1:03:35] Wade Foster: Thank you, Nathan

Outro

[1:05:59] If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries either via our website, cognitiverevolution.ai or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts which is now part of a sixteen z where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at aipodcast.ing. And thank you to everyone who listens for being part of the cognitive revolution.