Full Transcript

Transcript

Nathan Labenz (0:00) Hello and welcome to the Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my cohost, Erik Torenberg. Hello, and welcome back to the Cognitive Revolution. Today, after traveling to California for Google's IO event and taking some time to process what has been, in all honesty, quite a confusing week, I'm back home and excited to share a timely conversation with Zvi Mowshowitz. While many excellent news roundups have been published over the last few days, including by Zvi on his blog, here we aim to provide a higher level of analysis. We start by discussing the new capabilities that excite us most from a mundane utility perspective, including the native multi modality, lower latency, lower prices, and deeper integrations into platforms that were all previewed this week. And then we reflect on the fact that while the technical advances are indeed very impressive, most of the products aren't actually available for us to use and even many of the demos seem quite rushed, and we consider what that might imply. From there, we attempt to get a handle on the competitive and strategic dynamics at play as the big tech incumbents compete not only with 1 another, but also with younger, scrappier, and less constrained startups. We assess whether the apparent convergence of the frontier developers, retail, and API products may create a winner take all arms race dynamic, and we analyze whether more friend like AI products could mitigate that concern. We also debate whether the returns to scale and foundation model development mean that the so called big tech singularity may be near. Where I tend to emphasize the big tech companies' structural advantages and find myself, frankly, bearish on the majority of startups right now, Zvi tends to place more emphasis on the cultural, bureaucratic, and regulatory barriers to big tech growth and is more bullish on the challengers. I think the contrast between our perspectives here is quite interesting, and I hope you find it useful. Finally, we address arguably the biggest news of the week, the string of departures from OpenAI's safety team, including what it almost certainly does not imply, what it in fact might, and what the departed safety researchers and safety community should do now. We recorded this on a Friday morning and it was actually during our recording that Jan Leike posted a thread explaining his reasons for leaving OpenAI. And while we didn't have a chance to discuss his post specifically, I do think our overall analysis holds up in light of his statement. At a high level, the only thing that's crystal clear at this point is that the situation is increasingly complicated and moving so quickly that we should all be really honest about our fundamental uncertainty and remain open to changing our minds on key questions. I, for 1, certainly am. As always, if you're finding value in the show, we'd appreciate it if you take a moment to share it with friends. And if you think we missed anything important or got anything wrong, we invite you to share your feedback via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. Now I hope you enjoyed this timely analysis of another intense week in AI with Zvi Mowshowitz. Zvi Mowshowitz, welcome back to the Cognitive Revolution. It's always a busy week when you're here, and I don't think that's a coincidence. We've got a lot to cover.

Zvi Mowshowitz (3:09) I'm confident it's not a coincidence.

Nathan Labenz (3:11) I thought we would maybe organize this sort of as you do your blog posts, start with kind of the mundane utility and what we're excited about, and then get into maybe what we're not so excited about, and then maybe get into what we're worried about. And I think we've got some, good topics for each of those sections this week. How's that sound?

Zvi Mowshowitz (3:30) Oh, yeah. Yeah. That's definitely the right things to do, and there's definitely coverage all around.

Nathan Labenz (3:35) Cool. Or not cool, depending on the section we're talking about.

Zvi Mowshowitz (3:39) Yeah. It's not so cool.

Nathan Labenz (3:40) But yeah. Let's start off with then, obviously, busy week with OpenAI making some launches, Google making some launches slash announcements. I guess both of them really making launches slash announcements, which is one of the bits of analysis that I wanna get into. But I guess for starters, what stood out to you as the the new capabilities that are most exciting to you as somebody in search of an incrementally better life?

Zvi Mowshowitz (4:04) Yeah. So there's always a difference between what they're promising you're going to be able to do in the future and what you can actually touch now, you can use now. So what you can use now is GPT-4o And that is fast, that is half the price if you're using the API, and that is multimodal in various ways. It seems like a great product. It had its weaknesses that are becoming apparent as I use it, but it's already pretty exciting. And then when it becomes fully multimodal, it starts to play into the modalities that they're talking about where you talk to the AI and talk back and it hears your tone of voice and it hears the cadence of your expressions and sees your face and all that stuff. That looks pretty exciting in the relatively near term. They said a few weeks for that. But do think this is bearing the lead compared to all of the integrations that both companies are promising. That to me is the big exciting thing, if they can be delivered. It's what about these universal assistants? Because the demos they actually gave us were lame as hell on these capabilities. Can you recognize what I'm seeing in this picture? Can okay. Yeah, I already know you know how to do that. Why is this exciting? Do you remember what was happening a minute ago? Yeah, you can. Okay, yeah, again, I knew that. And it was that over and over again, these demos of educational videos with trivial exercises where the kid is being intentionally dense and already knows the answers, and then is doing a good job of play acting being actually lost, and then the AI doesn't pick up on it, he just pretends to bumble through. You have a bunch of translations of very trivial statements that yeah. I'm not saying I could translate them without knowing the languages, but it's close, and so on. And those are some OpenAI demos, but Google's demos were similar, basically in these areas. But then we're talking about the things where it's okay, search my inbox for all of my receipts, put them into a spreadsheet, organize all of my expenses and all of the things that have been done by category, so do all this automatically forever, which I told you to, on a background continuous basis. Okay, now I'm more interested, right? The context of integrating all of this information, right? OpenAI thinking it's gonna live on your phone, Google thinking it's gonna live on your phone, and it's shaping up potentially to be another Android versus iPhone battle, right? Because if Google is gonna own Android, by default, and it's looking as if they could deal with Apple.

Nathan Labenz (6:35) Yeah. It does I feel like the there's a really interesting point there on the demos and just how sort of slapdash some of them were. The OpenAI ones, I think they're trying to make a point on some level of this is you can tell it's very real and and, like, definitely not prerecorded and staged. Although you could always still fake it, but I believe that their demos were real and real time. And there were a couple things there that that the AI came back with where it was like, yeah. I didn't think that was maybe exactly what they were hoping it was gonna say, but they're at least showing off that this is real now, and we we have the confidence to do it live on stage for you. But across both companies, it did feel like the demos were a little bit of an afterthought. And I wonder what you make of that. Is that like a I mean, it'd be like, it's a reflection of the sort of ideological motivation behind the development in in a sense where they're, like, pushing the capabilities and then figuring we'll figure out what to do with it after, which has obviously been something Sam Altman has said explicitly in various ways over time. But it's still a little striking to me that they were so okay. We just made this thing now. Hey. Can anybody come up with something to do with it on stage? We gotta be on stage in 10 minutes, it felt. Oh, yeah. Well, Mira speaks Italian. Let's translate. That is cool, but it it did feel rushed. And I guess the other answer would just be, like, they're rushing. They're racing, and so they're taking these things out of the oven and putting them right on stage without much time to really figure out what to do.

Zvi Mowshowitz (8:05) It suggests that these capabilities are new. They haven't had them for very long. It also suggests to their credit that they're not optimizing for the demo. Right? They're not putting lots and lots of effort into putting forward the best demo possible. If these were startups, these were scrappy little companies trying to use this to raise money, you would have seen dramatically different demos. It would have looked dramatically more impressive for both companies, but they deliberately chose not to put in that effort. So you don't have the same effect of, I am very confident this is the best possible thing you could have shown me. Right? I am very confident that if Google or OpenAI had tasked a bunch of people with, okay, take the day, take the week, figure out something much more impressive, try it 20 times, confirm that it always works, get it ready, I think they would have gotten something that would have wowed us a lot more. I have to assume these were hastily assembled, not well thought out, like, not robustly tested demos. These were closer to real than you would normally see, but it still speaks to, they didn't try to make this interesting in some important senses. But also that could be a lot of like, who are you talking to? Like, you're talking to the person who is, doesn't even know what GPT-4 is. They're used to 3.5 as like what it is, and they barely even spent 5 minutes on it. And they have no idea what's going on. And the horse can talk at all. And that's actually really impressive because until a year or two ago, the horses didn't talk. Yeah. I I have a few

Nathan Labenz (9:32) candidates for the the things that excite me most. You hit on one already, which is the integration into the whole platform. For Google, that's obviously always been where they're going. And with OpenAI, it's kind of more suddenly, it seems where they're going, although I'm sure they've been, you know, planning for that for some time as well. But all a sudden, you get the desktop app. Now you can connect your Google Drive or at least, again, that's coming soon or in some phase of rollout. That does seem super exciting as our, you know, friend or, let's say, inspirational, figure Tyler Cowen says context is that which is scarce. And that has definitely been a feeling I've had with language models all the time. It's both traditionally that or earlier language models that might have meant you just only have so many tokens in the context window. Obviously, that problem has been largely solved with Google going from now one to 2,000,000 tokens that they're bringing online. That's a lot of tokens. You can fit a lot in that. But now you still have to figure out, well, how am I gonna get those tokens? What tokens am I gonna get? If you're copying and pasting 1000000 tokens around to do a single use case, like, now you've got a lot of friction there as well. So plugging into these different sources, whether it's your Google Drive or your Gmail or whatever, that seems like it is gonna be a tremendous driver of value.

Zvi Mowshowitz (10:52) We always talked about these giant index windows, and they're very, very useful when you know you need them. And you're sitting down to analyze a big bunch of data or search for big bunch of data, but it is much slower. It is substantially more expensive per token to do it. And mostly what we're seeing is Gemini flash. Right? We're seeing things that are optimized for speed, things that are optimized for cheap, and things that are optimized to run locally and just do all of these things in a way that the customer wants. And Google is also giving us 2,000,000 context window, which is very useful, but how do you combine these things? And a lot of the key is gonna be like, how do you go through all that context to figure out what the right context is? I always thought that Tyler Cowen saying context is that which is scarce. It's sort of saying two things at once. It's saying the whole, like, you don't have enough context. Right? But you're always short on context, but also sort of a definitional thing, sort of whatever it is that is scarce is the context. And so in some sense, once these details of your situation become background common knowledge, that's not context anymore in that sense. Right? That's just your life. Like, I don't think about that as context when making my decision. So, like, I don't think, well, in context, we like French toast. Like, no. We just like French toast. Right? It's just a fact about me that currently, these companies don't know, But, presumably, these companies would know when I'm just storing absolutely everything and they're taking an input on.

Nathan Labenz (12:24) So other things that I think are candidates for the most impactful aspects of these launches, one is the kind of freeness of it all. OpenAI moving to make the latest model free for everybody is a big deal, and Google seems to be doing something similar. It's not entirely clear exactly what they're gonna be rolling out at what price point in what product. They have their Google advanced, which is not free, which has been required to get you the most advanced models to date, but it's unclear if that will be required as a subscription to get the Gmail assistant that's coming or whatever. So we've got some clarification questions, I think, still for them. But this does seem like a moment where, obviously, OpenAI has almost a sort of Kleenex brand in AI thus far, and most people who have tried something and weren't super impressed with it tried ChatGPT with a 3.5 default. Now they are gonna get a major step up. How do you think that will change the landscape, the the world of work in the immediate term just on the basis of, like, now everybody can use the best model?

Zvi Mowshowitz (13:35) Yeah. Trivial inconveniences, like you have to pay a bit of money, definitely hold back a lot of people. And it's clear that from all sides, they're realizing that owning the customer is so much more important than the trivial cost to compute for these lightweight customers using a bit of it. Like, ChatGPT is pure profit for a like me. Right? I'm not actually requiring that many tokens per month. I'm paying $20 a month to have it because it's part of my job, and I wanna make sure that I have a variety of different things to try and get each other. But in practice, of course, like, they've hit a dollar in inference in a month. Right? 5% cost to provide the goods on the margin, that's shockingly high. So it's not very expensive to give these people the free version except in some of our people cancel their subscriptions because they no longer need anything but the free version. I do think it's gonna be an issue for something that they count on that revenue, but I don't think they count on that revenue particularly. And of course, they get the data every time someone puts something into there, and that's incredibly valuable. So that's another way you own the customer. So, the future, I think Google clearly is indicating that the same way that like you could have used Bing the entire time to get access to GPT-4, but nobody bothered because it was a weird different interface and people didn't notice. Google, I think has realized that for most purposes, Pro 1.5 is in fact the appropriate model for most use cases that people want to do. They don't need what Gemini Advanced is offering, but they will offer the upgrade to Gemini Advanced people who want to pay it and most people won't pay for it and most people won't need it. But the Pro 1.5 is for most certain users, plenty good enough. It was when you're trying to compare Pro 1.5 to Gemini Advanced, it's not entirely obvious that Advanced is all that in some sense, like much better than it in practical purposes for many of the things you wanna do. Like, Pro 1.5 is very good at searching documents, for example, which is the main thing that I still want Gemini for exclusive to other things, but I'm not integrating with, other Google services.

Nathan Labenz (15:40) Hey. We'll continue our interview in a moment after a word from our sponsors. I feel like the voice and the the real time vision also definitely this is where the demos were kind of flat. This I thought this was true really in both cases. I mean, the OpenAI ones were maybe a little bit better. At Google, I went out to the Google IO event, and they had a you know, it's quite a production. Right? They have the amphitheater, which they don't actually own, I learned. But it felt like they owned it for the couple days that we were there because they had set up basically a whole little city on this, plot of land with massive, like, almost permanent but still temporary structures for all their different sections, web and Android and AI. They're the whole AI building. And you go in there, and they've got all these different AI demos. And the one was project Astra. And you go into a little booth, and they have a camera mounted, and it's looking down at a table. And then then it's like, okay. Here's a another sort of bin, another table full of junk. Now you can, like, pick a couple pieces of junk, put them under the camera, and talk to the AI about it. So you've got, like, a toy dinosaur, and you're like, what's this? And it says, it it looks like a toy dinosaur. You're like, okay. And it was again, it's kinda going back to this, like, I'm not sure that that they really took this anywhere near how useful it could be, and it it does feel like the sort of thing where it's like, yeah. We made this. Now we gotta show it off somehow, and we're leaving the rest of the work to the community to do, which I have mixed feelings on that. I don't mind it. In some sense, it's an enabling technology and so fine. I do feel like that that those things should have been a little bit better thought through or certainly could have been a little bit better thought through. It does feel a little ideological in some ways. And I guess I just also want them to have a bigger, better idea for, like, what it where they're trying to take us. We're all kind of everybody is at least everybody in technology, right, is, like, looking forward to these announcements, trying to figure out what are they gonna do, and how's it gonna impact me, and what can I build on it, or is it a threat to me? And then they just give you this stuff, and it's like the homework isn't really done. I've been saying recently that the scarcest resource is a positive vision for the future, and I just I feel like I want more of the tangible vision for where they are taking us than just sort of, here's a table full of junk. Like, you can put it under the camera and, like, the thing will recognize it and you can talk about it. Even though it was impressive, they had another, like, Pictionary mode too, which was an impressive demonstration. You saw some of these things in the Google, in the Gemini 1.5 Pro original announcement where they showed, like, a a hand drawing of a thing and said, where did this happen in the movie? And it was able to cross reference this to a scene. That similar experience was on display. People were, like, drawing little line drawings and saying, what movie does this represent? Somebody drew a, in the little demo group that I was in, a a Wilson volleyball with the handprint on it and asked what movie this was, and it didn't get it at first. And then they drew a island around it and a tree and it put a a couple waves. And I was like, oh, I get it. It's castaway. And so it definitely is a very impressive and quite capable technology, but it does feel like, you know, sort of the electrical wiring has been created before we've have any idea what we're gonna plug into it, and that is just a very strange dynamic to me. I wonder if I can read more. I wonder how much of a vision folks like Edison had for what electricity was gonna do. I mean, in that era, merely lighting the room at night was a revolution, so probably didn't need to have too much more than that. But I don't know. How do you feel about this sort of building, they will come, everybody else can figure it out?

Zvi Mowshowitz (19:36) I'm a parent. And so what this reminds me of, were there anything else, is the kind of thing you do with your kid. You're like, do you know what this is? It's a toy dinosaur. What's out the window? You ask them basic questions, you check their knowledge, you fill in gaps, you see whether they're strong and weak, you're building up muscle memory, you're building up components. You know they'll need those components to then do things that are more impressive to synthesize them. The idea is, well, if you know what everything in the world is, if you can recognize what people are gesturing at in various ways and do other things, this then allows you to do things that actually matter later. And it's also just evidence of intelligence, evidence of comprehension. And that's all they're trying to demonstrate there in some sense. Like it's not the final exam. It's not like the job. The job comes later. But yeah, these are not use cases. These are not things you would actually do. It's a fun little game to have a play dictionary and try to guess the thing, and it's okay, but meh, like pretty big man, like, what is the vision of the future? The vision of the future seems to be the universal assistant. Right? The idea is that you tell it what you want it to do or what information you want it to tell you or or anything like that, and it does what you want, informed by all of its context, or the same way they're really smart, like knowledgeable assistant who's gotten a lot of experience if you would too. And the thought is, well, go from there. Right? What do you do with a person? Well, whatever you want. Right? Like, as far as they got. So how do you

Nathan Labenz (21:10) yeah. It feels I mean, there was so much talk of her this week and leading up to the events that I found that very striking. I haven't gone back and watched that movie in a while. I don't know if you have or how how fresh that story is in your mind, but would you read that movie as a positive vision of the future? Would you I've heard kind of scene takes across the spectrum from people being very excited about it to people saying it's a dystopia. Then others were like, well, it seems like it's more in the middle. How would you read that story, first of all? And then I'm interested too to hear, do you think this is where you personally are gonna go? Are we all gonna be walking around with her in our ears? Meta's got their we just heard that they've got a an earbud that has a camera in it in development. So that seems like a an interesting form factor for the her future. Anyway, how start off with how do you read that story?

Zvi Mowshowitz (22:03) So from what I can tell, right? Like it's not, it's good in the sense that it's not a dystopia and it's not a utopia, right? It's not pretending that this is all one thing or another thing. It's saying, here's a scenario, here are some things that might happen, here's some speculations as to how some aspect of that might go, and it's up to you, the viewer, determine is Phoenix better off in these ways or worse off in these ways? Is this a healthy or unhealthy way to go about doing things? Is it good or bad for his personal growth? What is the salience alternative? What does this do to society? And then ignoring the ending, because as usual, the actual big picture, like ASI safety takes are terrible. But if you just look at the concrete mundane questions it's raising, I think it does a, from what I can tell, it does a good job of not judging in every sense, it's saying like, here are some things that we expect to happen. And my view on it is something like, if you're treating the AI like it's a person, developing a relationship with it, and getting emotionally invested in it, and especially if you're treating it like you might treat Scarlett Johansson, in some sense. And this is not practice. This is not reps. This is not training. This is not experiments. This is like real for you, in that sense. It's not good. That's not good for you. And not good for society if everyone else is doing it too, it's gonna make it much harder to get real connections with other people in that sense. If you're using it as certain different thing, you're using it as like a source of advice, a source of information, a coach, a sounding board, like there are a testing ground, and there are ways you can use this that are really good, they're obviously good, using it as like a tutor to teach you new things, it's obviously just great, right? Like, could you possibly think that was bad? But there are obvious ways that you should go, wait a minute, this is not healthy. This is like taking our evolutionary instincts and then putting something on there that was not that is mimicking the thing that's not supposed to be the thing. And it's bad in the way that like pornography is bad. Right? Like, it's not that you would never ever use it necessarily, or you would want to ban this, but like, too much of this is bad for you. Like, you have to touch grass. And like the warning of her, right, again, if you just don't worry about the editing too much, like you've the concreteness, the warning is that this can substitute for other things, and then if you let it go too far, that can be bad for you. And the promise is that this is actually really cool and exciting in other ways and has a lot of money and utility. And the question is how do you balance that? And this is not a new problem to AI, we've had to deal with this with a lot of new technologies like mobile phones, social media, or the latest ones. The one that I keep coming back to more and more is television, actually. Partly because like with social media and phones, have this current big debate as it's happening right now, and it hasn't played itself out, we haven't been the full adjustment. With TV, we're at the end of the road, right? We've moved on to the next thing, we've seen what it did, we've gone through generations, like, we can look back and ask ourselves, what happened? How did we deal with it? What did this do to us? And the more I think about it, I think that that situation where it's not like they made out these warnings about how horrible this was gonna be, and it turns out everybody was being dumb. The situation in which everyone made these dire warnings about how horrible this was gonna be and they were just corrupt. And it didn't destroy our civilization, we're still here, but the things all just happened. Like the impact on people that we're worried about, they just happened. And we now live in the world that resulted in that. And it turns out that was like eminently survivable. And we've gotten richer enough and better enough in other ways that it's okay, mostly. But one thing you notice is that basically everywhere that got ubiquitous television is now global replacement level one fertility. So these things can be extremely unhealthy. Do we do about that?

Nathan Labenz (26:06) So how would you characterize the impact of TB? Obviously, hurt the fertility point. I guess my general sense would be that it has just given us such a ready solution for our boredom that there's, like, just fewer passion projects, fewer people developing new quirky hobbies, fewer people just chasing random stuff because it's just easy to kind of go there for a bit of entertainment. Is that your mainline narrative as well?

Zvi Mowshowitz (26:35) I think it ate massive numbers of hours in the day. And, yeah, it made activation energy so much more expensive to get. It made gumption much harder to have. It substituted for a lot of other things that led to good places that built up and had increasing marginal returns to devoting your time and energy. And it taught us to be couch potatoes. Right? Passive consumers of information. Just sit there. We consumed massive amount of advertising. We forget now in the Internet age where we're pissed off that, like, our 30 minute show has three minutes of highly skippable advertising on our podcast. It used to be that, like, your 30 minute show was 22 minutes ago. It was a horrible ratio, like, 25% ads. And nowadays, I watch a sporting event, and I make sure to start late, if it's not a huge event, because I can't bear the idea there are commercial breaks. Like, this is so bad. But that's a correct reaction if you have the option. Why would you want to endure that when you can just be slightly delayed? It's not super bowl, is it? If it is, then you're stuck, but and also the ads are great, but most of the time, the ads are terrible, and you know, like, it also like people, like, built their lives around this, they built their schedules around this, and they stopped planning things with people, like the whole Bowen Malone phenomenon. I think people underestimate that how much it was just television. It was just, you know, there's a show I can watch, there's a thing I can do, and so I think it sounds like a lot of work and requires coordination and requires an activation energy, and I don't have to. Right? I can just stay home. And now it's Netflix, of course, and chill or whatever, any number of similar things are. And I watch still today, you see my television, right? People think it's over, it's not over. And am I better off for it? I think so, but I'm not at all convinced. The other thing is just, you used to sit alone with your thoughts, just used to be willing to be bored, used to like, do all these things, and now we have so many different things, it's not just the television, it's not the phone, it's now, you know, you can listen to a podcast, you can listen to the music anytime, anywhere, you can do all these other things, you can just scroll endlessly. And so it's been supercharged and all these things matter, all these things change us and having an AI at our fingertips all the time, again, even in this sort of mundane utility world, right? It's not particularly dangerous as it really transform everything. It's gonna change a ton of things. And it doesn't necessarily mean we're gonna do choose to do smart high level things with it. one thing that's striking about all the Character AI style things out there is they're freaking dumb. Right? They don't. They spend a ton of time customizing the experience so that you get the type of interaction you want. The kind of thing that's satisfying to people, but they keep not investing in highly capable models to power those interactions. Like, they're ridiculously stupid so often, if you look at what people were reporting back or you try them out in my experience, like, you'll be lucky, I mean, you're not just not getting three GPT-4, you're often getting like below 3.5, you're getting something that's like abysmal.

Nathan Labenz (29:41) Yeah. That was really striking in my testing of Replica, and it's gotten significantly better, but it is amazing how I'll never forget the line from Eugenia, the CEO there. She said, going back even before language models, they started this project with the idea of addressing loneliness. And she said, we knew we couldn't make a bot that could talk, but we thought maybe we could make one that would listen or could listen and could make you feel heard. And she, you know, was pretty open about the fact that they use, like, pretty simple tricks, and she used the term, not my term, her term, parlor tricks in the early days to just make people feel heard. And I'm mixed on that. I I feel like that's another one of these things where it's good for some, but it it potentially becomes a big problem when it gets to be too broadly too good or too broadly adopted because they have research out that shows that and, this is, like, done over a year ago, you know, before the current generation of language models, which they haven't even fully implemented last I checked. But already, their product was helping people reduce suicidal ideation, helping people who, you know, need reps, get reps. So people by and large did report that they were more socially engaged as a result of having this outlet or kind of practice ground, training ground, whatever, however you wanna conceptualize it. So it seems like it's like even in that limited form, it's good for this sort of pocket of people who are really struggling. But then I do wonder similarly to what you're saying, like, how does that change life if it becomes really good and compelling to people who are not really struggling but could do other things and maybe now don't do other things so much? Hey. We'll continue our interview in a moment after a word from our sponsors. I wonder how you think about the so if TV just happened, then maybe this is just gonna happen too. Right? We're all gonna be walking around with this thing in our ear, and it'll be I mean, it it was striking how responsive the voice interface was. They talk about between two and 300 milliseconds latency to respond. And that is, like, the that is normal conversational interactive response time. When when we go edit our, podcast into script, like, the normal break between words that they that if if you have a long silence, they'll take it down to 300 milliseconds as their default for kind of, you know, you had a gap, but you wanted to shrink that gap to make it sound like it it never happened. 300 milliseconds is what they have. So if it can respond in less than that, it's definitely gonna be in this sort of similar zone to normal interactive conversation. I do wonder what forces are gonna shape this. It it does seem like it probably matters a lot what the business model is behind it or who's doing it and, you know, what they're trying to do for us, to us, with us. A an advertising model is very different perhaps from a subscription model. People obviously have also got sucked into social media. There was a time, which I think it's honestly still there to a significant degree, although maybe it's kind of receded some where everybody was optimizing for time on-site, and you had this, like, you know, what can we do to make this person come back? What is the most engaging thing? And it it became probably harmfully engaging, not just to individuals, but to society. I do think Zuckerberg, to his credit, has tried to back off of that. I'm not sure if other platforms have, but certainly they've, like, taken some public steps to, like, not reward anger emojis, for example, which I think is a nice clean low hanging fruit win, but still gotta give credit where it's due for taking that. Do you have any sort of, like, tree in your mind? What what do you see as maybe the forks in the road for how this new kind of ubiquitous technology could develop depending on what our relationship is to it. And you can imagine I always used to say too, and I think I might have to stop saying this, but ChatGPT, I always appreciated that the branding was not trying to be your buddy. ChatGPT, a name, like, doesn't it doesn't sound like a friend, And the way it talked to you also was not like a friend. It never sends you, like, proactive notifications, which some of these, like, virtual friend things do. Replica will send you notifications on your phone and say it misses you and it wants to talk. ChatGPT, you know, it's just like, here's your answer. Let me know if there's more I can do to help. But it's not, like, baiting you into further interaction. Claude interestingly does a little more of that where it'll ask you, like, what do you think at the end of its responses in a lot of cases, especially if it's, like, more philosophical or open ended. So I guess there's related questions there around, like, how do you expect this to develop in terms of how much the technology will try to pull us in and try to captivate us versus maybe try to push us out into the world? And how do you think that will depend on who's developing it and with what business models?

Zvi Mowshowitz (34:41) Yeah. So these are not the branching parts of the universe that I worry about the most. Obviously, I'm worried about bigger things, but stuff matters too, especially if the bigger things don't materialize. And I think you hit a lot of the nails on the head, especially the subscription versus advertising versus time on-site optimizations. So what metrics they're aiming for, I think are a gigantic question. And then the question of will people be able to differentiate and select based on what is good for them? Or will they be largely falling into these traps where they fall past the least resistance, they get checked into straight into entering Skinner boxes, they develop these kind of fake emotional attachments, they use it as an excuse not to engage with the world instead of using it as a tool with which to learn how to engage with the world or to better engage with the world. Every time I write with you both, I'm constantly filled with ideas either that other people have had or that I have myself or often both where I'm like playing off of them and they're playing off of other people. How could we use this as a tool to make our lives better, not through interacting with it constantly, but we're using it to enhance our choices and our actions and solve our trivial inconveniences, solve our knowledge gaps, skill gaps, again, give us the reps in the ways that are relevant, let us practice, let us do all the boring coordination, let us not get interrupted is I think one that's like going to become a bigger deal, let us track all of these logistics and all this paperwork and all of this stuff that's expensive to us, where humans effectively can only do so many things, even though the amount of like actual stuff going on inside those things is often small. AI can solve a lot of these problems for us, but we have to want it to, we have to care about that, and it has to be the thing we prioritize. I'm really hoping for subscription model that I think one of the best things that OpenAI did was go completely advertising free, completely steer the customer free as far as I can tell, There's this huge amount of revenue and ability that they've just given up entirely. And instead, all they're trying to do is answer my query, charge a flat rate, or charge an API for which particular marginal cost providing it is, and that's it. And as a result of that, that has become for many top people a standard. Right? Like, we've learned, oh, you can't just become a miserable per like, the difference between a triple a buy once carefully crafted experience game that's trying to do something good for you, which I think is a wonderful thing and people should do more of it than they do with anything. And the gacha games and the mobile experiences and the free to plays, which are designed to hook you and trick you and trap you and exploit the whale that can be exploited and makes everyone else's life miserable to make sure they can exploit the whales, and that stuff you just don't wanna touch for the most part. But even occasionally, they happen to create an experience that's good enough that if you are bold enough, you might be, you know, and confident enough in your own self restraint, can go into the lion's den, but mostly, you see this mechanic, you should run, right? You should run as fast as you can. And it's the same thing, right? You have TechGPT is inherently a very healthy product. They're not worried that people are going to do something bad for them on a personal level because they use too much TechGPT. I just, I haven't even heard a single story. Same thing with Quad, same thing with Gemini. These companies are being very responsible in the ordinary sense. Character AI with Replica, they're not. Replica is very obviously the predatory free to play trick you into coming back, like message you when you're away, give you your daily reward of various types. Like every time, oh, you can get more free messages by waiting and then come back for them. I know where this comes from. I know about delayed variable rewards. I know daily login bonuses. I know all the tricks. I was in the gaming industry. I know how this stuff works. And when you see that stuff, you've got to run. And so the other question is, if we don't regulate, if we don't mandate, because the free society, we really shouldn't be regulating and mandating this stuff, right? Bad incentive drive out good or does good drive out bad? Who wins this fight? And if the bad guys win in this sense, it's gonna be a pretty miserable time for the people who get trapped. But my hope is that the people with the best models presenting good experiences and that can continue and we can scale that. And the other question is what happens with the advertising model and where is the advertising? Because right now, is clearly delineated. It's clearly distinct from the experience, but with the large language model, it seems very easy to say, well, we're gonna give you a free experience where you can ask about various different products that you might wanna compare, but if you just want a little bit of priority in what's discussed and which features you're highlighted, and whether we talk about Coke or Pepsi or we talk about Tide or one of their competitors, you're welcome to pay us. Right? What happens in that future where like suddenly this thing is suddenly pushing stuff on you and you can't necessarily tell the difference and the labels are increasingly convoluted? If you aren't paying, you're the product, right, in sense. And like, we could reach one of those worlds, and then maybe you even have to get your second AI to watch what the first AI is doing. If you're smart, you can notice when this thing is happening. You can double check once they're saying things and then they're buy ins and all of this get complicated. I don't wanna get too far away from this so we can do so much to get through.

Nathan Labenz (40:27) I do think this is really interesting. The analysis many people have covered the news. So I think where we hopefully can add value is with a little more synthetic analysis. I I do have one word of defense for Replica in that having gotten to know Eugenia a little bit, I do think she is really trying, and they do have a subscription model. Although they do have these add on buy a new outfit for your replica type things going to. I think that research, which I do take as reasonably credible, it came out of a group at Stanford, they weren't directly involved in it other than handing over data as I understand it, is pretty compelling. I do think they're headed into a very challenging balancing act of a future. one thing she said to me that was really interesting, and I I do see that this is, like, another dynamic I'm gonna ask you to analyze. On the one hand, they started with relationships and basically no, like, mundane utility. And then with ChatGPT, we have pure, isolated, episodic help on whatever you're looking for help with, but no relationship. And I was struck that I heard from her actually a couple months ago, last episode we did, that she is moving replica toward more of an assistant. And she said something I thought was really interesting, which was maybe the moat in AI is relationship that, you know, you don't swap out your friends, because a new better friend potential friend came along. You have some sort of loyalty to your friends. Like, the history matters. You know, what you've been through together matters. And she was we're gonna become more of an assistant, but we still wanna have this notion of relationship. And maybe that is where the moats in in AI come from. That's why maybe people don't switch at the drop of a hat long term. ChatGPT, on the other hand, is headed that direction a little bit too now it seems where you've got certainly, the voice is, like, much more given to relationship, and it's, like, developing memory, and it's obviously gonna have longer context and all these sorts of things. So these maybe seem to be converging. I wonder how you think about the ethics of ChatGPT, like, going more in the buddy direction if this is something you think and we have, like, wave of departures from OpenAI to talk about too. I wonder to what degree you might speculate as to how this move toward the, like, more engaging personal relationship possibly could have been one of the things that the the safety team might have objected to. Certainly, I would have been like, hey. Do we really wanna go this far, this fast? Can't we keep the voice just a little flatter? Okay. Cool. It's responsive. But does it also have to be so thirsty, like, in our first release? But then there's another dynamic too that I think I would if I'm reading into what they're doing correctly, Altman's now saying, well, we wanna get into not safe for work content. We wanna be able to do some of this stuff, even gore, even, like, erotica, whatever, but we don't wanna be making deepfakes. And I'm like on the one hand, might say, well, why would they be doing that? That seems just why would they even wanna touch it? On the other hand, you might think, well, maybe they think they can do it more ethically than other people can, or maybe if they do 90% of the stuff that people want, it'll take the air out of the sails of the sort of totally unscrupulous actors that would be doing your if you can get NSFW AI from OpenAI that's, like, r rated, maybe you don't need, like, Taylor Swift deepfake porn that's x rated or something. And maybe they think, hey. We can be, like, the good guys that do this racy stuff, but stay on the right side of the the worst ethical lines. There's a lot there. Convergence to buddies. And how do you think they're maybe thinking about, like, how they play against, like, other actors that they may consider to be worse actors?

Zvi Mowshowitz (44:19) So I'll start with the last part. I wrote out my model specs analysis for OpenAI, but because I think everyone at OpenAI is gonna be hella distracted this week and everyone else as well, I'm just gonna hold it for another week or two and refine it and adjust it and then post it later. But one of the things I have in it is in fact, an endorsement of not safe for work, right? I think you have to ask, is this actually harming anyone? Is this bad for people who are not the user Or is this so obviously just terrible for the user every damn time that as a free society, we need to say no? And as far as erotica or gore, the answer is just obviously not, Right? People have this desire to be thirsty. Right? They have this desire to be hoarding on vain. They have this desire to be like playing in these ways and getting these experiences. And if you don't let them get it at home, they'll get it on the street. You can't shut this stuff down. It's just part of the human experience. And yeah, I think absolutely, if somebody wants to create pornographic images they're not deepfakes of individuals who have not given their permission, then why shouldn't we do that for you? Given we have verified you're an adult, maybe we charged you extra because, you know, why not? Maybe. But we, why shouldn't you be able to do that? If you wanna create a picture with a bunch of blood and gore, why shouldn't you do that? They recognize with Sora that when you can't do sex and you can't do gore, you can't make most of the interesting things that people do with video. Draw the line at PG 13. A lot of them are just out the window, especially every individual element has to be like very carefully curated and gatekept and so on. It's just not gonna work. And so, yeah, I think that you draw the line specifically at inappropriate portrayals of actual individuals, but you don't wanna make a an obscene Taylor Swiftie thing. That's a problem. You should catch that. You should stop that from happening. And there's a certain level of difficulty you need to impose on that before someone could be allowed to do that, but you have various different tricks to defend yourself against it. But I think if you wanna either create a pornographic deepfake of no specific person that like just plays to whatever it is you want to see an image of. Or if you want to create a picture of Taylor Swift wearing a different colored outfit, sitting in a different venue that she never visited for her Eros Tour, like as long as it's got a watermark, like what's the problem? You wanna take a picture of her sitting at your son's birthday party and have fun pretending that she was at your son's birthday party? And you play her music. Right? Because you can just record that and hit play. That's, like, kind of disservice. What's so bad about that? Why is that a problem? And, obviously, different people have different opinions and maybe each celebrity should decide like what level of reproduction is okay and not okay. But certainly like the situation where like Barack Obama can't be generated by an image model, like giving a speech at the wrong college. Like it's just like dawned to me. And again, it would have forced me to use a different image model for those purposes. And how long is gonna be before those three image models are just trivial to set up and trivial to use and everyone starts using them? I keep waiting I I have the old decision on my computer, and I tried it out for a bunch of stuff. one thing that's very good for is you tell it to create a 100 copies of something, and then you come back 30 minutes later, and then you check this, like scroll quickly through them and see if any of them have what you wanted, maybe you go image image on that one and maybe you figure out what will, maybe you just use this to diagnose, okay, here's what's wrong with my prompt and try a different prompt, and you can't do that if you're using the internet services because that would cost them an hour and leg. Now I'm using my own GPU, so it's fine. Or alternatively, use that to create stuff that they wouldn't let you claim because you wanna see what you can do and you wanna create it and you wanna enjoy it. So I moved on to it basically, but it's become especially obsolete that I just use the online ones now, but I've been waiting for stable fusion three to be easy to deploy and not to go back to, okay, see what can see with the data. In general, yeah, I think they should offer the services that you want. Don't think it should be this thirsty by default. I think this idea that suddenly you have Scarlett Johansson acting all thirsty with you, like when you just ask what the weather was outside, with no indication you wanted thirstiness, that's weird. Don't I think that's ideal. I think that this is we went too far. But if you put in your user preferences, you dial up the thirstiness tone, right? You select, you want the Scarlet Johansson style voice. I don't think shame. Like, it's fine. Enjoy yourself if that's the statement you enjoy. And again, we should be very careful with consent and very careful with permission when there are real people involved. But I think a lot of real people will be very happy to give their consent either for free or for a very nominal charge. You ask me if someone wants to use my voice for some ungodly unknown reason on SkyGBT, if it's just a handful of people, I'd say go ahead. If it's like a 100,000 people, I'm like, I should be paid. But it's about reasonable compensation here. And I'm sure a lot of other people feel the same way, including people whose voices someone might wanna hear. Are you telling me I shouldn't just pay an extra dollar and get the Morgan Freeman voice for all of my narrations and my explanations? Because that seems cool. Why shouldn't I do that? Or maybe I watch her with Johannson. Again, if she gets paid, that's fine too. And then like you were there was some other stuff that you were asking about that I've almost forgotten at this point because it's such a complex question. You just have to remind me what we're talking about.

Nathan Labenz (49:58) What I am thinking about is and it it is there's a lot of issues that are intertwined here, but there's a couple dimensions of convergence, actually. There's, like, the convergence to the buddy form factor. And then I also see at a lower level and we're gonna have an episode. I'm gonna have Alex Albert. He was a prompt engineer, and I was leading the developer relations function at Anthropic. And I was asking him about how all the APIs are converging too and how that how they think about that in in the sense that it may create a winner take all dynamic. So on the one hand, I'm like, man, I don't feel great about sort of the trend toward relationship first in general. I do feel pretty good about replica for, like, the people that are really struggling with loneliness. Not sure how I feel about it when it gets to, like, a super broad based thing. I'm not sure how I feel about OpenAI being so thirsty with their voice. But then if I go down one level and I look at the API and I see convergence there and it's a one line switch, possibly with a little bit of prompt engineering, definitely with a little bit of prompt engineering, but it's, like, close to a one line switch just to go from, you know, GPT 4 turbo to Cloud three Opus to back to GPT 4 0 to Gemini 1.5 Pro, whatever's next. That to me is a worrying dynamic in that if it's if all of the integration points have converged to essentially the same service provided in the same way with a down to a one line change, then it becomes a winner take all dynamic in the, like, whoever has the best model in theory can win all the business very quickly. Like, why don't we just switch to the new 1? But then maybe one reason that could not be so all winner take all with one release to the next would be this relationship dynamic. So I'm like, not sure I like what it does to society if we're all, like, in these parasocial relationships, but maybe I do. What it does to the to soften the, like, super intense borderline winner take all dynamics, I'm not even sure to what degree those exactly overlap because one is the app layer and the other is the API layer that powers the app layer. But what do you think?

Zvi Mowshowitz (52:17) I see opposite. I think you're thinking about this all wrong. So first of all, like, I noticed that you have more than one friend, and I have a lot more than one friend. And I choose which friend to call based on what we want to talk about and what we might want to go do. And sometimes I have a group of friends. Sometimes I have 1. Sometimes I have a different 1. And I would get bored if I was hanging out. My best friend's name is Seth. And if we hung out every day, all day and try to talk about everything, I'd be like, this is not ideal. I don't just want to hang out with Seth all day. But so I had a lot of people do. And the switching, this means that you are not locked in, right? You're not stuck. So quite the opposite of winner take all, right? It lets you move around and even Character AI and Replica understand this. So you can talk to 10 different bots, right? For different purposes. And of course, minimum, you wanna figure out, you have this one for when you wanna talk about engineering, this one for when you're thirsty, this one for when you just wanna talk about restaurants in the area, blah, blah, blah, which makes perfect sense. ChatGPT has his GPTs, and now Gemini is gonna have its gems and so on. And then you have this thing where you can plug into a different LM. And so for me, I have changed my primary LLM reasonably recently from GPT-four Turbo to Gemini to Claude, and now back to GPT4.0. And in many cases on a dime, it doesn't mean I don't use the other ones. It means that like when I want a variety of answers, I'll call all three of my friends and I'll ask them in some sense, I'll ask them, Hey, what do you think? What do you think? What do you think? I'll compare notes. And then I'll be like, I know what, I know who's good for this, right? I want to analyze a PDF, want to go to YouTube, right? I want an image, I didn't create this kind of image better. And so I put portfolio. And then as people advance, you learn, okay, so then you just capture more of my portfolio for now, but then that switches back and I want this variety. I want these optionality. And maybe I don't necessarily even want these things to know all about me, certain aspects of me, then I do different things. The data portability is probably going to be a huge thing. You're at Replica, you build up this relationship, but are they going allow you access to your records and data? But that's like a pretty toxic thing for them to do. I assume they won't. What happens when you just download the chat log, upload the chat log into Gemini three or whatever it is, and say, you're this guy now. Here's everything that you've said to me, and here's everything I've said to you. And it learns all about you now. And now you have your new assistant that knows what your old assistant did. Often you don't switch employees because you don't want to have to train them to explain all the context with them. But now that context is just copy paste. That's like a huge I don't expect winner take all from this unless the winner deserves it. So like GPT-five is out and everyone else is still at 4 level. Well, of course, I'm just going to 90% of the time plus be using ChatGPT, ChatGPT-five. But if it's a marginal change like it's been recently where it's like one of these things is a little bit better, that's completely different. It's particularly the lock ins and the relationships that I think are the places where you're likely to get a pinch of winner take all scenario. So if Gemini knows the contents of my Gmail, knows the contents of my sheets, knows the contents of my docs, is creating workflows where it generates new documents and new memories, sees my photos, analyzes my photos. I start taking pictures of all of my food, so it knows what I've eaten every day, where I've been, etcetera, etcetera. It has all of this new context. It'd be very difficult to transfer this context. Now suddenly I have this huge state where every time anything is remotely personal, I want to use Gemini, right? And that becomes a thing. But I don't think that in general, I've been seeing this lock in unless of course the people develop this kind of emotional attachment, right? This may be romantic, maybe it's something else. I'm basically saying that's not like that, that going too far is not particularly healthy, right? I think that's bad. I think people should strive to avoid that except under pretty special circumstances, like beyond what it takes to get like reputation, reps and practice and stuff like that. But I'm optimistic, this is, the convergence seems like it's friendly thing, right? Like, why wouldn't you, like, how amazing would it be if you could transfer your stuff and your connections and your knowledge between social networks, between Android and iPhone, between all these other things that have like converged, but are seemingly incompatible. Like compatibility is just almost always good.

Nathan Labenz (56:38) Yeah. It seems good for the consumer. I do wonder about how in a world where Google had their data on the calendar and then OpenAI does their launch one day in advance, it does seem like there is a potential for exacerbating the the general one of the biggest worries in AI is that the and I'm I know I'm not telling you anything you don't know as well or probably better than I do, but the arms race dynamic between either countries or even just leading companies within The United States where, hey. We gotta get this out. We gotta beat them. You have x time to do your testing, and we're launching, and that's the time you have. And so corners start to get cut perhaps. This feels like the liquidity with which people can move about the market would enable that dynamic or encourage that dynamic. Because if OpenAI is, like, about to launch you know, one party's gonna beat beat us. We gotta get ahead of them. Now you have this. And that that sort of depends on the idea that if they're out ahead of us, then everybody's gonna switch to them, and we're gonna be screwed for a while. And that could be screwed in revenue. It could be screwed in data. It could be even just pride and reputation, which certainly seems to be a big part of what's happening at the moment. You don't worry about that. That to me, I'm convinced by your multiple friends argument that as long as there are different things in the same ballpark and they have different characters, you go to different things. I do personally find that as well. I still if only because ChatGPT has the native coding environment, even if Opus is a little bit better on coding, like, fact that it can execute the code on ChatGPT side keeps me there for coding. Now it actually with O probably is notably better on coding as well. But Opus is a better writer, so I go to it for help with writing. So I do have my different AI friends for different different purposes at the retail consumer level. But in my app building, it is more of a I'm gonna make a choice. Right? If it's Waymark and I'm gonna have it a language model or a vision language model process all the images of a small business that's just uploaded their library, we're probably pretty much gonna just do it with one model. And in practice, like, you can switch really easily. That dynamic doesn't seem like it a a big concern to you still?

Zvi Mowshowitz (58:57) Well, it can go both ways. Right? If switching costs are low, then me announcing a day before you doesn't really matter. So what you get my customers for a day, and then I switch back. Like, who cares? If it's incredibly expensive, well, then if you can get ahead enough to convince me to switch, I'm gonna get stuck. And so maybe you get incentivized to try and capture that market a lot more, or maybe you're trying to outdo each other continuously to try and say just a step ahead because we don't take a lot. But my guess is there'll be enough differentiation between what features these different tools have and in what details you want to use and in how you customize them. I got to put a decent amount of work at this point into customizing my instructions for ChatGPT, because they have custom instructions. And I haven't done that for Gemini or Quad yet, but like when they offer that and I decide to do this time, I will almost certainly do that. And they might look very different because I'm trying to specialize, right? They might be like, okay, so when I go here, this is what I want. So it's a different set of instructions. But as an app maker, you're not going to want to switch every time there's a little incremental improvement, right? So it's just not the same thing. Look, we don't take all, in some sense, from a safety perspective is ideal. If OpenAI or Anthropic or Google was out there two years ahead of everyone else and their model was just better. And so for all dangerous purposes, they were the only ones that mattered, then there are advantages to that. But I don't think that the interoperability is going to change whether or not that turns out to be true in an important sense. Don't think it substantially changes necessarily the pressure we're putting on people in that way. But also one of the things about this last week that was like why I really enjoyed this week on the release side, even though there's other things that I didn't like as much, obviously, was that everybody is now seems to be focused on things that help people have a better experience and improve the world, but that don't advance the core intelligence of the underlying system. So to me, that's just win, right? I want ChatGPT to be better for its users without becoming more intelligent in the ways that they get more existentially dangerous, As long as it stays in its current state and just gets all these new cool features, that's just the best of the world. I love this world. And so I was really heartened to see the directions people were taking it, see these new developments, and I can wish everybody a great success. Obviously, it built a certain amount of revenue and hype and pressure, but I think mostly that's already built in, and I accepted that's just like the world we have to live in. So why begrudge? We're not just having don't threaten me of a good time, right? Just keep going.

Nathan Labenz (1:01:42) Yeah. I'm with you on that. My standard line with the g p d 4 class models is we're in this sweet spot where it's powerful enough to be really useful, but not so powerful as to be a real concern. And I do think that's great, and I do think all these integration points are great. The the enhancements have just, as you, of course, famously call it mundane utility are great. Yeah. It doesn't feel and I hear the argument too about, yeah, if somebody's way out ahead, then they have the luxury of doing it right, taking their time. Doesn't seem like we're in that world to me. I I do expect OpenAI is probably still about half a generation ahead internally. My guess is that they don't make the best model free if they don't have a significantly better one coming before too long that's gonna get everybody paying again. That could be wrong, but that would be based on all kind of rumors and intel and just how much time has elapsed and public comments from Sam Altman. It does seem like they probably still have one additional turn that Google may not have made yet. But overall, it does feel like they're close enough where they're, like, running, generally speaking, neck and neck, and that part does concern me. And and Meta is, like, not far behind either. They just put out a paper yesterday where they had trained a new model called Chameleon, which is very natively multimodal. They call it early fusion, meaning text and image tokens are encoded in the same way, basically share all the same weights, trained with 10,000,000,000,000 tokens, which is not a small amount. And unclear if they're gonna open source that yet, but, you know, same week as as all these other kind of natively multimodal things that they're, you know, answering and definitely just showing that you're not gonna stay in front of open source for too long even if you are, you know, ahead by a full generation, you know, generation and a half. You're gonna have to keep going. I'm still not convinced the other direction either that I shouldn't be worried about the liquidity at the API layer, the ease of switching, and the sort of sense that may create where we gotta one up these guys. Yeah. I do see that dynamic kind of developing, and it does seem problematic.

Zvi Mowshowitz (1:03:52) It is. I don't think these things that basically, what I'm saying is I'm not sure these things intersect that heavily in terms of the correlation is close to 0, not that it's like going the other way or anything. Just these things seem to me to roughly cancel out in the sense that like that same pressure is always going to be there. OpenAI and Google are going to try and run up each other. They're going to try to advance themselves as fast as possible. They're running into a lot of the same constraints. My assumption is indeed that OpenAI has somewhat of a lead here, but the fact that OpenAI seems to be quite responsibly taking their time to build the next thing, whether or not they have a choice, we'll see what happens. But it's really hard to tell who's out in front. I think Google has a history of essentially being far out ahead of what Google actually releases. Google has had a significant number of pretty earth shattering for the time capabilities in the past and it decided that their lawyers should carry the day and they should not release the product. And so it's not obvious to me that Google has to be behind at all. Right? It's it's very opaque from the outside. You can't tell, like, the default is that OpenAI is 6 months to a year ahead or something like that.

Nathan Labenz (1:05:00) If OpenAI is not 6 months to a year ahead, it would seem to be more likely because Google also has another half or full generation more than they've shown, not that OpenAI doesn't. I would

Zvi Mowshowitz (1:05:14) My assumption is that OpenAI I mean, these things don't take that long to train as such necessarily yet. It's more like that you need to get into a position where you're ready to do that, where you have the resources to do that. You know, you have all the data streamlined and you're ready to go. And it's really hard to know, obviously, any of this for sure. It's also possible that there's a big gap between when you have it internally in the base model form and when you're ready to actually release a product. I speculated also that it's possible that there's a GPT-five style model. They either could train or have trained, but that like inference is so expensive that they think it would degrade the user experience to rush it into market. So they'd rather do this instead because for most purposes, that's not actually what people want in some form or they need to scale up their infrastructure. Because if they have to choose, they have to either serve GPT-five to a small number of people or serve GPT-four-zero to a very large number of people. And maybe they think that it's better for them if they serve the second 1. I don't know. I mean, just have a hard time in the long run, like who has the advantage, right? Who has the better infrastructure, who has the better teams, who has the better capabilities. Like, again, it's There are people who know much better than we do and we'll find out. Yes, in an important sense, I wish it wasn't close. Right? I wish I knew who it was, who was in front, and they could, in some senses, take their time, much more so, and I definitely see the pressure that seems to be building of throwing caution into the wind and just doing some stuff. But again, shouldn't caution be thrown through the wind outside of some certain key moments? Like when are the moments when you should be cautious, right? So like GPT-two, we saw some caution, right, for OpenAI, even if it wasn't enough for Dario, right? At least GPT-three, again, we saw some caution. We got some dispute over how much caution was warranted. GPT-four, we saw a lot of caution with the release. It seems pretty reasonable to not particularly be that cautious about 4. If you've looked at it, you're like, no, these extra modalities don't really do anything that's that dangerous. We know that because we've tried a lot of stuff with GPT-four and it's not like it's close. It's basically fine. And then you're in a normal commercial situation and they were up against any number of deadlines, right? You've got the Google IO thing coming the next day. You definitely wanna you potentially asking Ilya and maybe John Leike to to stay on until after this presentation so that doesn't distract. So you've got potentially a bunch of things going on.

Nathan Labenz (1:07:40) I wanna talk a little bit also about broader market competition. Our mutual friend Andrew Critch coined this term, the big tech singularity, or I think he calls it the tech company singularity. But the notion here, I think, an interesting one where he basically says, at some point, we may hit a point where the big tech companies, in virtue of their superior AI and compute capabilities, could reach such a a position of dominance where they could effectively enter into any industry that they wanna compete in and come to dominate it in relatively short order. And I do start to feel like we may be getting close to that, and I think you have a different point of view on it. But my kind of just simple thinking is, man, Google has a $100,000,000,000 cash on hand. So one 1 fork in the road there would be like, are they allowed to buy in or not to these industries? But it's crazy. They got a $100,000,000,000 cash on the balance sheet. I looked up just some of the big hospital companies in the country. one of the biggest ones, not the biggest, but but a big one that I'm familiar with is called Tenet, and they have 58 hospitals. They're worth $13,000,000,000 market cap. For one eighth of the cash that Google has on hand, they could buy one of the biggest hospital companies in the country. And then you can imagine them just plumbing MedGemini into kind of everything, ripping out the sort of nightmarish electronic health records that currently exist, putting something in that would hopefully be a lot better, tapping into all this data. They've been able to do these MedGemini things, I would assume, with huge barriers to data. This would dramatically reduce their barriers to data, would allow them to I'm sure, of course, they'd still have to handle compliance in some ways, but they could dramatically streamline their access to data, train the next generation of medical models, provide med GPT as kind of frontline service. And it would seem to me like they would pretty quickly be the best hospital system in the country if they did that. And then I imagine, like, replicating that across other verticals, education, as we've discussed. Obviously, these things hold tremendous promise for education. The key thing seems to be, like, actually building it in a sort of first class way, which I'm not sure current institutions are gonna do very well, and Google might struggle with it in some ways too. Again, they could buy, like, any number of private education companies for a pittance from their perspective and just run a loss leader experiment to see what they could do in a new vertical like this. Do you think that that's not the case if you think that's not true?

Zvi Mowshowitz (1:10:19) Because I think they're not culturally capable of doing the things you'd have to do to make that work. I think that their reputational and legal and regulatory barriers to doing that are very serious. And I think that they're not going to be able to deploy technology in those situations that is that far ahead of what people who are simply paying them to use their, to use big tech technologies could get in their place. Right? OpenAI is not going to be able to use GBD5 for its hospital system substantially before GBD5 is released. That's not really a thing. The version of, I forget what the med, what their alphamed or whatever it is, what they're calling it. The version that Google is going to put out there is going to be the same version that they have internally, but it's also going be available to everybody else if the people want it. Yes, it gives them better data to do this, but they could also get that through licensing agreement through a of cooperative deal. And there's a reason why basically companies like Google do not put themselves into every nook and cranny, try to do everything, add every feature. There's a reason why we see these demos and we're like, have you do have no idea what your products can do? Are you not trying to do anything with your product? Are just leaving that for the rest of us? And the answer is yes, they really are in some important sense. And they are trying to develop their core systems of products, make their thing available, let someone else build off of it, let someone else take the baton, let someone else handle these things. I saw a few weeks ago, a story about AI private equity, right? So the idea is this thing, but without being Google, right? So like you go out there, you find a company that's obviously in an AI business, doesn't know it's supposed to be an AI business. You buy the company, you then tell all the employees to use AI, you deploy the AI systems, everyone gets much more productive, the company goes up in value, the company takes a lot more money, you win. You then use the profits to keep doing this. And that seems like a much better model to me. So does being a startup whose job is to raise a lot of money in order to launch this hospital thing. Either work for existing hospitals or maybe use a private equity thing and buy them out. I don't know. Maybe do both. But these models strike me as more what these people are capable of. But like, Google is terribly sclerosing in so many different ways. My entertaining and so was Microsoft, so were the other big tech companies in their own ways. You can't simply have them launch an entirely new thing and do an entirely different thing and expect that to just work in this sense. Now, at some point, if you have GPT-nine, you can tell GPT-nine, go out and buy a hospital system and deploy your technology to help the patients do better and make more money, and GPT-nine just handles all of it because it's GPT-nine, but that's not the most important thing that's going on with the fact that you have GPT-nine. You bury the lead if you can do that, right? It's very important senses. So as long as we're still dealing with these mundane levels of AI, my expectation is that the big tech companies just they only have so many imperial focus points. They only have so much different things they can do at once, and it's not in their interest to do this. It's the reason why Google is constantly buying startups, and I don't think that's gonna change.

Nathan Labenz (1:13:35) Let's talk about the startup side for a second as well. I feel like I am hearing less about startup acquisitions lately from the big tech guys. Obviously, we had the Microsoft acquihire inflection, but compared to a number of years ago, it does seem like there's a lot less acqui hiring going on. And I certainly, there is bureaucracy at these companies. It does seem like they've managed to clear some of that out in the Gemini department at least. There has been an interesting discussion on that just in the last 24, 48 hours on Twitter where, among other people, Sholto, who was is locally famous for his appearance on the Drakesh podcast said, yeah, basically, nobody cares about levels of Gemini. Elsewhere in Google, yes. But in the Gemini thing, it's a very sort of flat. We're all kind of one team trying to make things happen vibe. If you are a startup I I was going down my list of previous Cognitive Revolution guests and just asking, like, who is feeling enabled by all these new things and who's feeling threatened by all these new things? And I'm like, I think there's a significantly a significant majority of them that seem like they are more likely to be threatened by it, if we're honest. I do think in the short term, probably still everybody grows because and this is for some of the reasons that you're saying that Google's not gonna productize everything immediately. They're not gonna go to market with the same sort of intensity or focus that startups will. So I'm bullish on everybody's, like, next couple to few quarters still. But I do look at the economies of scale that I see developing in the big tech companies vis a vis the startups that are building on the platform. And I'm like, man, I don't like the beyond two to 4 quarters position for a good chunk of them because they're always gonna be a little bit behind on the models, of course, that inside they're gonna know what they have. Inside they're gonna be picking off of what are the biggest opportunities for us. The outside, of course, you're you're waiting. Inside, you can also do it for free. Exactly how this is gonna be positioned and what you have to pay for is to shake out. But, like, one thing that's definitely clear right now is g p t 4 0 is free for all in ChatGPT. It's not free. They've lowered the cost, but it's not free via the API. So right off the bat, you have a as a startup, you're like, how do I exactly compete with that? I have to pay for it, which means I have to charge for it, which means, you know, I just have all this additional friction compared to somebody can just go to ChatGPT and and use the the thing directly there. And, again, just the economies of scale, just man, the who who can really compete with these guys? We've had folks on the show who specialize in text to speech, who specialize in image generation, and just g p t 4 0 once its full capabilities come online over the next couple weeks, seems like it's likely to be best in class at both of those potentially. What do you make of, like, this the position of the companies that are building on the platforms right now?

Zvi Mowshowitz (1:16:41) So Altman, I think, had very wise things to say about this, right? The idea that if you're building your model, you're of your company, you're building your company and you're building its tag because GBTN isn't smart enough, isn't good enough, and you're going to build a substitute that's going to, for now, do a better job, then we're to steamroller you. We're going to curb stomp you. It's here to die. But if you're planning, here's an apparatus to use intelligence such that when GBD5 comes in, you switch the 4 to a 5 in the function call, and now suddenly your product just works way better And you're not directly trying to do the thing that the base model can just do. Now you're in amazing shape. And in fact, you have a great company that's going to be happy with OpenAI makes progress, right? So the question is, can you plan for something where you're happy when OpenAI comes up to a new model and not SAP? Without loss of generality, obviously. But, so if you're 11 labs and you have amazing text to speech right now, then yeah, you have to worry a lot about whether OpenAI is going to curb stomp you because you are fundamentally competing directly with them to offer a technological service that they are going to want. And either you find a niche, say you're willing to replicate Scarlett Johansson's real voice and OpenAI isn't or something like that. But otherwise, yeah, they they're eventually going have anything better. You're going to have to somehow keep up with them and that's going to require being big and they do something good. Because like that's a generic ability that OpenAI is gonna develop anyway because it needs it for its core products. But if you are trying to be a private equity company to buy that hospitals in order to streamline their medical records, well, now you're like, okay, cool. You have the latest AI to just feed into these things to be better. You're like, cool, hand it over Google, hand it over OpenAI and you plug it in and your product works better. And you're that much better than competition that you were last month when they didn't have an AI at all and you had GPT-four and now you have GPT-five, right? It's amazing, right? You're just that much farther ahead now. And so also there's no conflict between, there are going to be a lot of amazing companies, there are going be some, we're to all die before that because it'd be some really cool companies. And the current companies are often not going to be the cool companies, right? It's very possible that every two years, half the existing companies die in this realm because they got curb stomped because the thing they were trying to do no longer differentiates them from the core model. The core model was not able to do that, but the other half do well. It's the tech bubble in 1999, right? Where, yeah, a lot of these companies turned out to be doing something that had no underlying value as the internet developed and it didn't differentiate them. But also, if you had portfolio of them, part of that was Amazon. So you made money anyway.

Nathan Labenz (1:19:24) So who do you could you point to specific companies that you think are, like, well positioned? Because I honestly struggle to list too many that I think are gonna be like, oh, yeah. This is playing how we to our advantage, how we hoped and puts us in a a long term winning position. I I don't find, like, a very long list, honestly.

Zvi Mowshowitz (1:19:44) Yeah. The problem with I I think you've mentioned this before is that AI is this huge beat. And so you have to choose, like, where you're specialized. And I have made a deliberate choice mostly not to follow the details of various start up projects and small projects that are offering within utility in this way, because they're not that important to the bigger picture. And it's very hard without investigating carefully to have value add things to say about them. Right? So it's okay, I'll see what happens. So I can't say which ones I think have great market prospects necessarily, but I also can't say which ones are necessarily going to get curb stomped either. And I would rather be in terms. Right? How is I don't I have no idea what Perplexity is actually doing over the hood. I don't know to what extent they're training their own model. I don't know to what extent they are working off of GPT-four or Claw or whatever it is right now, but then adding custom instructions and then scaffolding to try and allow it to much, much better analyze the web and come out with very rapid good answers. And in a sense, there's like a version of your bucket that gets curb stomped and a version of bucket that's potentially going to do very well, right? It all depends on a lot of details. But when I look at what are the companies whose tabs I have opened, what are the companies whose stuff I use? And I do not use those things because what I want is what the base model is. I don't use Character AI either, right? But think about that as an example, right? If you're a Character AI, well, if you're just counting on the fact that OpenAI is not going to be thirsty, right? It's not really a great long term plan. If you are counting on, we're going give a bunch of tricks that make this thing feel like human interaction because you're replica, because the models aren't good enough to simulate real human interaction on their own, that's not going to be sustainable either, right? But if you're building your AI bot such that when you make it smarter, it provides a better experience, but you know all these things about how to build relationships that OpenAI doesn't know and it never going to find out, you could be in a great spot. There's nothing stopping you, Right? And in theory, that stuff should get much, much better. And it should especially get much better when you start adding video generation and virtual reality. Right? Which is where I assume is the future of that sort of thing, a few years from down the line. And if you're ready for that, great. So it just depends on, again, you trying to be future proof? Are you planning to where the are you getting where the puck is going? Or are you fitting the fact that like right now there are some flaws that you can better address and you can go to market? But there's also room for people who sign Brightly for a year or two and make a bunch of money and then fade away because they're tech

Nathan Labenz (1:22:21) That is honestly when I've considered any sort of new entrepreneurial endeavor right now, I'm like, I think you're right to cite Sam Altman. And and I think that analysis is right. But I'm like, man, that's pretty tough. I, yeah, I I I can't identify too many things where I'm like, oh, yeah. If I do this, then I'll be, like, much enabled by the next model, but I won't be threatened by it. The majority do seem like right now, like, they're in this kind of compensating for weaknesses, trying to go to market, and they have to. Right? Because if they don't if they don't compensate for the weaknesses, they can't really go to market successfully. If they don't go to market successfully for most of them, aside from a few that can maybe raise, like, enough money to really make a long term play and not feel rushed to market, but that's obviously not a super common position to be in. It just it feels tough. So when I've considered Ashwin, I don't think I'm gonna start a company in the short term, but when I have considered that, I've felt more drawn to this. What would be like a flash in a pan, you know, that would be that meets a need right now that could grow really fast that maybe doesn't make sense 18 months, two years from now, but which could be, like, a good winning short term sprint that sure inevitably gets burned up in the supernova of the innovation models.

Zvi Mowshowitz (1:23:36) You think it does get burned out. Even if your original tech gets burned up with the supernova, right, by by learning a lot about the business and your customers and by building relationships with your customers, you might be able to entirely turn this over into something that still makes sense. And there's a lot of places where getting it right, making it easy on the user. Like we talk about all these custom instructions, all this stuff in between different models and different companies, and all of these sophisticated power user things that we're doing. And we gotta remember that most people aren't even using these models at all. And other people who aren't using them at all, most of them are incredibly unsophisticated. Right? They're mostly complete civilians, and most people will never touch a settings button in their life on any of the apps they use. No matter what options you give them, no matter how valuable they are, they will never learn what they are. This is not how people work. So if you can just do some basic good customizations instead of good defaults and handle some regulatory issues and just make life easier for people in a way that slots in the technology. Right? And then you have a bunch of scaffolding to, like, actually have it be better, right? In various ways. And that's really promising. I was talking to one of my friends, he's a lawyer, right? And his firm is encouraging them to use their own internal customized, but like very lightly customized. It's like some basic service Lawyer GPT that they, like, can use internally, but they can't just use the basic thing because of compliance concerns, but also because, like, they wouldn't necessarily know exactly what to do. But there's enough, like, annoying individual steps on it that he ends up just not bothering. Because by the time he actually like imported the proper case law and hooked up the proper documents, the only thing he could have just done by hand. So too often at the time, so he loses most of the utility from it. Most of time he ended up skipping it. It seems you personally could probably within a month, code up something that would supercharge this capability and probably increase the entire firm's production by 10%, maybe 50%. Certainly if you had a small team. So, and then again, if you built it properly, when GPT-five comes out, you just switch the 4 to a 5 in the reference call and your product just gets better. There's the alternative, but you can't use the, still can't use the alternative and still doesn't know enough to like properly do the alternative. And then yeah, when it properly integrates into your entire workflow and Google's buddy has fully automated into, okay, it has access to every document in the world automatically. You're still gonna want this kind of specialization and you're gonna use the profits you made early on to do the specialization thing. If you buy the hospital, you're gonna be able to integrate into the hospital's workflow, you're gain the trust and ability and education of these doctors. You're gonna make it easy on them because they are not tech people and they do not want to have to think about it. And they want something that like feels right and safe and good to them and the regulators are breathing down your neck. And there are so many things like this where professionals' time is so incredibly valuable and getting the answers right a little bit more is so incredibly valuable, but you don't have to be that much better in certain ways. Again, we can be 10 times better in the interface. We be 10 times better in just the ease of onboarding and usage and regulatory compliance. And then you got some scaffolding, but yeah, sure, you just run whatever the last model is. That seems fine. I think there's plenty of things that like, they're not future proof if you don't think you're code, but, like, they're future proof in terms of if you're planning ahead to make them work.

Nathan Labenz (1:26:51) So you said you like the business model of the private equity AI re reboot for companies that don't realize they should be AI powered yet. Do you like venture capital right now? Would you go would you wanna be a an LP in a generic Silicon Valley venture fund at the moment?

Zvi Mowshowitz (1:27:10) Venture's a weird situation because your compensation as an LP in a venture capital firm is not tied to your expected returns that highly. Right? It's a question of, look, if you offered me a big enough fund to go invest in companies and we set aside ethical concerns for the moment with potentially advancing AI in the wrong ways, or we were confident we could like only choose companies we were confident we're neutral or better or whatever it is. And you gave me a big enough portfolio to work with and enough of the share of the profits. And of course I'm excited. Do I expect VC to make money, like above market returns? I think it's a completely different question. And that is a Now we're talking price, right? We're talking about whether or not there's too much money raising into AI or there's not enough money raising into AI. And then also the question, of course, of how much are the insiders and the high reputation and well connected people just getting differential access and positive selection and you're missing out on all the deals that are worth it. And so you're just left with the dregs. And unless you're, like, really good at picking up from the dregs, even if the industry does well, you're gonna lose. But, yeah, I'm currently advising Lionheart Ventures. We're trying to do a safety positive fund for AI. And I think they're taking LP money if you wanna invest in our next fund. But that's we're talking the industry, about the start ups, about what's going on, and partly to try and encourage the company to think will be better in the world. It's not gonna be about money, and don't have a very big share of it. But sure, of course, if you have a bunch of money, then it comes down to you get the good deals, and is your expected return high enough if you're considering or not? And that's very hard to tell from the outside. My guess, are the valuations the same? Or maybe the valuations have an extra 0 on them they're not supposed to have at this point, or maybe everyone's sleeping and they should have an extra 0 they don't have, and now that would seem completely great. My guess is if I could get a portfolio that said I get to buy shares in SV, on the stock exchange, in SVUS, was just a equal share of all venture capital investments made by anybody at Silicon Valley, and it was trading at cost of investment plus interest rate or something, would put a portion of my savings into SVUS. But that's not the question, is it?

Nathan Labenz (1:29:30) Well, that's a pretty good approximation of what I'm trying to ask. I think I would probably put more into big tech than that. How I would you had $1 to put in SV, hypothetical SVUS, and $1 to put in big tech US, which is whatever your companies. Yeah.

Zvi Mowshowitz (1:29:49) It's trading at cost of investment plus interest and the existence of SVUS didn't radically alter the valuation of all the companies, which it would. If this traded, then all the valuation would probably go up 10 x or something crazy. We have various reasons to suspect this, but at current valuations in relative terms, I think I'd be pretty excited for the small stuff. And I think that the market will reflect would reflect that if other people had similar market opportunities, that you'd be able to turn around and sell it at a profit if it's certain that the actual people wouldn't pay. And they would have a both sides of returns if you didn't have that, and you just got to hold to maturity as it were. But I'm very happy with my portfolio of companies, essentially, is big tech investments that have made me a ton of money and everything else, which is like a wash. Right? Like, I made some money in for solar and I lost some money in Hasbro and it's all the emerging market stuff is all wash. It's vaguely even except for a number of big tech companies that have just like gotten, returns have been very good. And they, big tech companies are increasing percentage of my portfolio, they don't rebalance.

Nathan Labenz (1:30:53) I guess I expect that trend to continue. To summarize our somewhat contrasting viewpoints, I'm seeing it being pretty hard to outrun the big tech companies because they seem to be moving pretty fast and the returns to scale seem to be super high. And it seems like a large majority from what I see seem to be playing this sort of compensating for current weaknesses game and trying to get to market. And meanwhile, the next generation of model is cooking if it's not already cooked. And the current one is free in retail and not free at the API layer. And it seems like you more see friction and bureaucracy friction in the market, bureaucracy at the big tech companies, regulatory barriers, such that you think that while maybe everything I'm saying, I don't hear you, like, disputing any of it, but you're more saying, yeah, that's maybe all true, but it it probably still gets contained by these other forces and leaves, like, a lot of opportunity for other smaller companies.

Zvi Mowshowitz (1:31:59) I think I basically think that the big tech companies just only have so much bandwidth that attention has to be focused and critical for any given company of whatever size. And that leaves treasure everywhere. There's just think about all the things you could do with current technology if you had somebody built the app for that, right? Somebody built a really good, not just a GPT, but like a much broader true context has gathered all the data, is coordinating a bunch of stuff, is a lot out, like actually sat down with users, figured out what they actually cared about and so on. And it's certainly, it's something Google could have built. It's something that, I mean, now people could have built, but they didn't because they're not doing so. They didn't even try and fail. They didn't even try. Right?

Nathan Labenz (1:32:46) They did preview that virtual employee in their keynote. What's your expectation for the AI powered virtual employee?

Zvi Mowshowitz (1:32:55) The first thing I noticed right away was like, is this thing in all of the chats and reading all of the messages and the DMs and, like, all of your emails? Like, for everyone on the team, and then talking to other people on the team? Looks like it is. And that doesn't seem great. I mean, I don't necessarily want all of that stuff to be viewable by an AI that people query. That seems bizarre. People are start using Signal at work just a separate way on their on their second phone. What's gonna happen here? But to the extent that if that problem gets solved, it seems really exciting to me that essentially you have this virtual employee. Really what it is is just it's a bot that scoops up all of the information from all of the context in which you're okay with everybody having that context, ideally. And then it can answer questions. It can help communicate. It can help mediate. It can pass things along. It can I'm especially excited for just things like you ask him, what has happened regarding x? Or can you keep me posted if something happens with respect to y? Or can you contact Cheryl in marketing and have her evaluate which of these three things she wants she puts his priority on and get back to me because I don't wanna have to talk to her directly because she's really rude. Or at least, any number of things. Right? Like, it just, it seems like it's the sky's the limit on just, you have this extra tool in your toolbox that the practice is going be incredibly valuable because it just knows all the things and can do all the things and continuously monitor all the places and like just not having to monitor as many feeds on a continuous basis by itself needs like a consent.

Nathan Labenz (1:34:31) Do you have a sense for what the hourly rate should be for an employee like that? They may or may not charge for it that way,

Zvi Mowshowitz (1:34:40) Which hours, so it's always the question as a consultant, learn which hours are people paying, right? Which parts of your experience are being billed and which parts are not being billed? But if you assume that when we talk about what you're billing while it's being queried by a human, right? While it is performing a specific task, but not while it's hovering up all the information in the background, I assume a lot if it's good, but it's always the if it's good, right? Like if it's reliable, if you can count, there's always that threshold effect to me in these situations, right? Is it good enough that I trust it to do this task? Because if I've got, I wanted to check the marketing room and see if marketing had any requests or proposals or craziness that I might have to respond to, right? And the marketing channels are nuts at my company, right? They do like 40 pages of stuff there. And I don't want to read that junk. So I have the AI read that junk. Now, can AI reliably alert me when I actually have to pay attention, such that I trust not to read the stuff? If it doesn't, then it's worthless. Right? But if it's good enough that I actually don't read it, then suddenly it's a godsend, even if it has to spam you at 4 pages out of the 40 per day, just to make sure there's no false negatives. So the question is, can you get it reliable enough? If I ask it to have to talk to share on marketing, would actually be able to interface with share on marketing and would fail basically when it fails such that I can take over from it. If it fails silently, if it hallucinates weird stuff, gets me in trouble, that stuff happens even a little bit, I can't do it. But if we can get there, suddenly it's incredibly valuable. Think it's a lot of stuff like that, right? Or if it's just ease of use, right? It's like with my friend, the lawyer and the case files, right? Like if I can get it such that it's easier to have this thing checked and it's reliable enough, suddenly I win, but until then I have nothing. But I think you see a lot of this stuff where like, you have nothing until you have everything. And then once you have everything, you have to learn, you have it, and you have to get people to actually use it. So like you've the three things you have to do. So these things are hard, but like it's Google, we'll get there. The question is how long will it take and like which ways will it get there, but we'll get there.

Nathan Labenz (1:36:56) Yeah. I think you're right, absolutely right to emphasize the importance of threshold effects. It does seem like we're pretty close to tipping over a couple big ones and it will be very interesting to see how that affects the broader world of employment. What's your expectation for employment over, say, the next year? Do you think we hit a point where we do start to see a real impact on either, you know, on the positive side, like measurable productivity or on the potentially negative side. You could imagine a world in which people that are entering the labor market are really struggling because entry level developers just like can't really beat Devon or entry level general purpose knowledge workers can't really compete super effectively on an ROI basis with these AI assistant, AI virtual employees from the likes of Google. What's your short term expectation for that?

Zvi Mowshowitz (1:37:53) So you have to obviously separate what's gonna happen in specific areas of employment. What's gonna happen with employment more broadly, either like the sector of information knowledge workers or this overall employment. Overall employment, just a question of how big the positive number is, right? Because as you make people more productive, as you enable people to do more things, like we are nowhere near the point where the AI just starts doing the next marginal task that a human wasn't bothering to do before, right? And to me, like that's the question. If the AI takes 10% of the tasks in the world, or even like 25% of the tasks, well, every time that the new task that we were going to do, 7% of the time, the humans better off doing it in AI. So we're wealthier, we have more productivity, we are richer than we thought we were, we are able to commission more things and people will be in more demand, not less demand. So that's great overall. Question of knowledge work specifically, I don't expect that much diffusion that quickly, and to say, think that's why the private equity stuff, for example, is so important, slash, can be profitable. So specifically, we're starting to see a few industries where people are hurting, illustrators, translators, to some extent junior coders, junior engineers. I think the junior engineers are actually mostly a non 0 interest rate phenomenon combined with a law on the tax bill. So I think we're seeing this problem where engineers can't get work and everyone like, but it's not about AI. It's about the fact that you can no longer deduct those expenses properly. And so like every company doesn't have a huge cash set. It's just like getting blown away by this and interest rates are higher, so you can't invest in the future as profitably. And the combination of these two things is just hitting the market really hard because they're happening at once. And hopefully we fix the stupid tax loophole reasonably soon, and the anti loophole, right, where you just get flammed for no good reason. And then the situation gets a lot better. Because with programming, if you double the effectiveness of every programmer, or if you double the effectiveness of every really good programmer, plus 50% of every bad programmer, and made every person who can't code at all as good as really terrible programmers or something, I think you just get a lot more programming. I think you just get a lot more demand. I'm now much more capable of programming, which has taken me from definitely not doing any programming to might want to do some programming. It's not gonna take someone else's job away, it's just gonna be that we code more things. So I'm pretty optimistic about that aspect of things, because I've literally never been at a company where I would have thought where we had engineers. I was like, what would we even do if another engineer? We have all the engineers we need. That is not a thing. There's always like, we need more engineers, we need better engineers, we need to prioritize and we're constantly triaging what has to be done today versus next week versus next month, because like, good coding is always incredibly valuable to any real company. And I don't think we're anywhere near that changing.

Nathan Labenz (1:40:50) So do you think we'll see on the positive side, like, measurable productivity improvements in the short term?

Zvi Mowshowitz (1:40:56) So they say that the internet showed up everywhere with the productivity system, right? Like that, and we know that's bullshit, right? We know that the world is a much richer place. People are much more productive in important senses because of the internet or because of computers and technology, But the stats don't seem to say that. So I wouldn't be surprised if the stats just act dumb. I don't know how to put this and don't reflect the reality of the situation. And also like coding is not that big a percentage of the economy, right? So like, we know that certain even knowledge works of certain types, broadly construed, not as a big a percentage of the economy. So I would say, I don't expect that much in the shape of like measurable productivity or economic growth to be apparent in 2024 or even 2025 necessarily at current pace, because I expect so low adoption even amongst the places where people could be productive, and people not adapting it properly and not getting the most out of what systems can do for a while while they come down, And also for a lot of the actual productivity not showing up in statistics, including employees just hiding the fact that they're totally disproductive, like just searching off a lot because they can, or alternatively, like doing better jobs of the same thing, but looking the same, or like just various other techniques. We just have this long history of the statistics just not being good at this, but that's compared to like what I expect to happen in the medium term, right? Like my 5 year, 10 year projection is for this to be very noticeable even in the relatively not interesting worlds. You talk to economists, they're like, well, know what, might add 1.5% to our total productivity over the next 10 years if we have a really optimistic scenario, people are gonna crack. People just like, what are you smoking? This is make no sense. That's just not a possible thing that could happen unless you like, and maybe if Hawes AI, like extremists got their daily to roll back with GPU 3.5, which I think would be insane. Very few people are even you know, almost or whatever's the actual yeah. No. But we just use the current things we have. We just extend them in a way. And I wonder how many of those economists would watch the presentations from this week and see those demos and rethink what they're saying. What's always happened is they're always trying to evaluate, okay, if we have exactly the things that we currently have and nothing else ever, what happens to the productivity stats? Think they're still off by an order of magnitude. They're still crazy, but they do have to notice when other things become actually possible, right? Like, what would be the productivity boost just from a universal translator that actually worked? Just be very concrete. Okay, everybody with a phone now can get full real time continuous translation, including tone of voice, including affect and associations, usually good translation, as good as the startup translator. It's being said as if it was being spoken to you, its original form, we have only the minor pause we have to live through a play. And what does that do to productivity? That huge on its own, right? And there's so many different things you could put in that slot. They're just huge on their own. So I'm very optimistic over time. I just don't want to get too ahead of ourselves.

Nathan Labenz (1:44:05) So maybe another time we can talk about the possibility of a biotech revolution, because that's another one where I see an interesting dynamic where it seems like things could be about to get pretty crazy, but much in the way that these this new class of weight loss drugs maybe doesn't makes everybody a lot better off or makes those people that are taking it a lot better off may end up actually shrinking GDP in some ways because there's, a lot of expensive things that don't have to happen downstream. I could I'd be very interested to hear your take at some point on how that might play.

Zvi Mowshowitz (1:44:34) Attack 0 for 0 for a while because of bureaucratic and regulatory issues. And then 10 years from now, watch out.

Nathan Labenz (1:44:41) Is it to the point where you, like, change how you live or change, like, your risk your personal cost benefit or risk analysis at all because you think there might be genuinely revolutionary advances in biomedicine for you?

Zvi Mowshowitz (1:44:55) I've already baked in that kind of thing mostly, and I think mostly there's very little you can do that you shouldn't be doing anyway. If you're a healthy person, you should try to stay healthy. You should invest a lot in your health. And I don't think this changes that. I don't think that The flip side of that, of course, is the world could end in some sense, right? I'm 45 years old. If I think that we're going to hit escape velocity and like all these designer drugs that make everything great, by the time I hit 60, like half the time, right? And like half the rest of the time, I'll be dead. Like, I'm not saying this is my model or anything. I'm just saying, if you didn't believe that, then maybe you just don't care much about your long term health because the death rate in that range is actually not that large. There's a lot of different ways you can approach it and humans are completely irrational about these things. But the actual observed a real preference for me is your health is valuable. You get good returns from being healthy immediately, effectively. You don't need to add the longevity to it. And there are basically no clashes between longevity and current health. They're actually like, well, I have to trade one off the other. It's like, no, what's good is good for the most part, as far as we can tell. As can't far as tell, we just don't know anything. There's probably ways that you do trade off, but like, we just don't smart enough to know about that stuff. So, yeah, I'm putting reasonable enough investment into me, my health. I do think that it would be dumb with this kind of revolution potentially on the horizon to skydive. Like, to take known, like, large risks of being literal then. That, like, clearly, like, you will not be saving this. Right? Like, you're if you roll a natural 1, like, we can't help you.

Nathan Labenz (1:46:30) Okay. Well, we can dig deeper into that on another occasion. Gotta address the departures from the safety team at OpenAI this week. I think everybody probably has heard the news. Daniel Cataleco, I'm not sure if I'm saying that quite right, but he left a couple weeks ago now and has been posting some cryptic stuff online saying that he declined like, left his equity on the table, which would have been a large majority of his personal net worth because he wants to retain the right to criticize the company, but we haven't actually heard what those criticisms are. Now, obviously, we've got Ilia is out with a pretty conciliatory message saying he thinks everything is in good hands and they're gonna build safe AGI. And then Jan, who was his co lead on the super alignment team without such a message, simply saying I resigned and leaving us guessing. It's obviously not good. What more can you say about it beyond its obvious not goodness?

Zvi Mowshowitz (1:47:30) I've been writing up the article this morning for this that I'll probably post next week. I have 8 safety researchers in the last 6 months who have left one way or the other. In addition to Ilya and Jan and Daniel, we have Leopold and Pavel who were fired for leaking confidential information supposedly. What little we know about this, it's hard to tell because when you leave confidential information, the last thing they wanna do is tell everybody about all the confidential information you just leaked through the Sirius because, like, obviously, that's self defeating. But it looked a lot like they technically leave confidential information that was used used to fire them, but, like, from the outside, we can't tell. We also lost William Saunders, Colin O'Keefe, and Ryan Loeb. So that is I don't know exactly how many people would count as Lyme leader safety researchers under this criteria, but it feels like a lot. Right? We also did lose Diane Yoon, Chris Clark, Ahmed Morikawa. We those aren't obviously safety related, but I don't know. What's going on, guys? Sherry Lachman, head of social impact is gone. It doesn't seem like the kind of pattern of people you are losing when you're, like, being a socially responsible safety first company. Right? It feels like a company loses these people when a lot of people are very dismayed in some fashion. And yeah, obviously, we have the messages that are being sent and not sent. We have the fact that people are almost universally under NDA. We have the fact that like Daniel gives us very strong evidence that nondiscretionary clauses are in place basically everywhere, unless you are going to pay an extreme financial price to not do so. And even then you're still under NDA. So it's probably good that Daniel is that he's reserving the right to criticize them in the future, but the NDA probably severely constrains his ability to reveal the information that would be useful right now. And he's made a very reasonable decision not to break the NDA.

Nathan Labenz (1:49:21) So tell me, so he did say one thing, which was his reason for resigning was loss of confidence that the company OpenAI would act responsibly around the time of AGI. Right. That's almost a verbatim quote, probably not quite exact. How would you the NDA thing seems weird to me, to be honest. I'm like, if you really believe that and you are willing to leave that much money on the table to be able to make some criticisms, What's the NDA doing here? It's it doesn't seem like they could really sue you super effectively. They maybe could, but, the Streisand effect, you're talking about self defeating. Right? Like, suing your safety departed safety person only increases the attention, the media circus, whatever, to what they're saying.

Zvi Mowshowitz (1:50:06) You might not wanna proliferate certain technology or certain technological information. There might be a philhazard involved.

Nathan Labenz (1:50:13) Yeah. Seems like you can kinda separate that. Right? Is it so it wouldn't be, like, the mere existence or the discovery of something that would lead to a loss of confidence. There's gotta be something, like, social about that. Right? And the the public is primed for it when it comes to Sam Altman was just the subject of a lot of drama. People there's definitely, like, smoke here. It seems like

Zvi Mowshowitz (1:50:33) think that that's the obvious. But so my experience with this type of thing is that, like, yeah, if if they were to reveal some esoteric, like, detailed reasons why, like, they were concerned that it was, like, technically leaking confidential information that you were an NDA for, probably, you know, you necessarily get sued for that, although you might, but also, like, it could make your life pretty miserable, and it could take away your flexibility to do other things in the future. But the baseline scenario here is there's simply a toxic environment inside OpenAI for people who are care about safety and want to be safe. Right? That it's now transformed into a move fast and break things startup mentality, and that they are inherently suspicious after everything that happened of anybody with associations with, like, restaurant or the alignment form or the VA or safety of any kind, and that they're making their lives miserable in various ways, and that politically often watch all those people gone. And so it's steadily taking every incremental opportunity to get those people gone, and that as those people leave, more of them are now more isolated and under more pressure to leave. And if you were to spell all this out, you'd be breaking your NDA and potentially making yourself vulnerable. And in general, these are like, Sam Altman is representing reasonably well that he can be a medicated person, right? That he might come after you if you personally if these are personal issues and political internal issues, and you, like, spread their dirty laundry and you launch these kinds of attacks, I don't think it's unrealistic that you get sued for it. Like,

Nathan Labenz (1:52:03) would be damned. I think that I've dealt with these people, and a lot of these people will absolutely sacrifice the

Zvi Mowshowitz (1:52:10) strident Stridend effect in the short term to credibly be the type of agent that will retaliate. Right? It's rational. It's highly rational. It's highly correct from a decision theory standpoint to be really to come down very hard on somebody in the situation who attacks you. Right? The counterpuncher. If you are trying to be a modern leader of a company in these situations where there's a lot of secrets, there's a lot of dirty laundry because there have to be even if you're doing everything right. Even if no one's doing anything malicious and everyone's trying their best, and everything's basically fine, there's still a bunch of stuff that you wouldn't want airing in the public, and yeah, you credibly threatened that you are going to enforce these things, the full extent of the law, you're gonna make people's lives miserable, you're probably gonna do other things to make your life socially worse in various ways. Try to blackball them from their future jobs and opportunities and blah blah blah. And Sam Hulman knows a lot of the VCs. He knows a lot of the people who determine whether or your startup goes well or badly, whether you can get various different spots. I don't think being worried about these things is unreasonable. I do think that we can be confident that there isn't, like, an imminent, like, the world is about to end because of OpenAI technology that's motivating these departures, because then they wouldn't be talking. If it was, like, no. Seriously, they broke Q Star, and they hooked it up with g p d 5, and now this thing is, like, clearly on the verge of breaking out of the Internet. We're very worried about it or some crazy scenario. It's clearly not happening. Like, very clear, not happening. Then I think these people would be willing to talk if only to whistleblowers to a government community or something. Like, they'd be talking. And then they potentially, they're talking to people who are not in public. If you had serious problems with really deep wrongdoing, would you go public of your concerns, would you call the would you call the government? Yeah. Not obvious to me that one approach is better than the other. But, again, the the false explanation for all of this is that the environment at OpenAI became toxic as a result of the events that happened, regardless of the extent to which the involvement directly caused that to happen or intentionally wanted that to happen. And as a result of that and also that a lot of people were effectively his political enemies, and now they're gone. Right? And if you see someone doing that, getting rid of systematically getting rid of dirty people in the safety departments, maybe you leave. Right? Daniel lost faith in OpenAI's ability to navigate the future AGI world responsibly, and left, it wasn't too long after they fired the opponent of Vowel. one obvious explanation is that he knew that he thought those firings were bullshit, and that he saw that they were cleaning houses, safety people, and shoving safety people to the side and working mostly with them. And therefore, he wants to make the Naruto to be irresponsible. It doesn't require there to be a big mystery here. Right? But the obvious explanation is that OpenAI is just not uplaced. It is currently very receptive to something that I think will be very vital to making sure the future goes well, and that's deeply troubling.

Nathan Labenz (1:55:01) So any parting advice for either the departed folks from OpenAI or there's been talk also of compensating folks like Daniel for leaving his, you know, stock on the table to try to encourage others to do that or to, you know, set some sort of a precedent. I guess any like, what should we want in the short term? If would agree with you. There's probably not like a super emergency or they probably would just be throwing all caution to the wind. But what do you think they should be wanting and and what should they be doing to get it and what can others be helping that?

Zvi Mowshowitz (1:55:33) Obviously, it depends on what the underlying situation is. Obviously, they should take their concern or no matter what else they have done. And if they can't do that, then that's, like, horrible. They should be if there are things that justify whistleblowing to governments behind the scenes in ways that their protection is poor, they should be doing that. I think the difference between disparagement and breaking your NDA, obviously, you think about these things very differently. I do think it would be good to consider compensating Daniel or others who have given up their equity in order to not find non disparagement clauses. But I do worry especially about paying for NDA breaches about the secondary incentive problem. If you know that people who are working on safety will feel free to violate their NDAs because this ecosystem will compensate them for doing so, then maybe you just can't hire anybody in the ecosystem at all. Maybe you can't hire anybody who cares about safety because you're worried they will steal your secrets and bring you economically valuable secrets, and therefore you can't do it. I think you should think very carefully about what you are and aren't going to compensate to what extent, and not just throw your one card. Right? Like, believe in freedom of contract, these people knew what they were signing, and they knew what they were agreeing to. Doesn't mean that it's cyclic doesn't mean it's fully cyclic, doesn't mean that there aren't circumstances where you should break the NDA. And there aren't circumstances where you should go to actually people who do break the NDA. But, you know, I think those bars are hot. Also, obviously, I am a journalist now, as are you to some extent. And if anyone wants to break news, if any level of confidentiality and privacy, my the answer open.

Nathan Labenz (1:57:06) Well, that might be a good note to to leave it on. I'll look forward to your full analysis on this topic next week, and it's always great to break it down with you. Any other closing thoughts?

Zvi Mowshowitz (1:57:18) Just to be clear, like, it's all still you know, we don't know much about this in particular, and about many of the demos and other things that we learned about in general. We're speculating. We're trying to process a lot of information very quickly. So I don't think we know what's going on. To be clear, like, this could all be as simple as events like the vegetation in people's mouths. They want us to fresh start. The environment is somewhat hostile philosophically to what they're doing, and they're just not having fun anymore in some sense. And then then that fact makes people lose confidence. Like, maybe it's that simple. Maybe it's all kind of a coincidence. We don't know. It's just yeah. I don't believe coincidence in a sense. That's all.

Nathan Labenz (1:57:55) Cool. Well, we'll keep following it in our respective format. So thank you for taking the time today. Enjoy the conversation as always. And Yep. Zvi Mowshowitz, thank you for being part of the cognitive revolution. Absolutely. It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.