Full Transcript

Transcript

Nathan Labenz: (0:00) And that is when I thought, maybe even though we're using very rudimentary tech, this product is not only about tech capabilities. It's really about human vulnerabilities. It's really about the humanity and the feelings, not just about what language model you're using or what technology you're using. And this is why I thought, I'm going to start working on this right now. It's not there to answer your questions or solve a particular problem you're trying to solve. It's here to have a conversation with you and to make you feel a certain way after you had it. Otherwise, it's a very shallow answer. It's like, it's just for fun, for entertainment. It's chitchat. Most companies use conversational AI as a means to an end. We use conversational AI as the end.

Eugenia Kuyda: (0:45) Hello, and welcome to the Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together, we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg.

Nathan Labenz: (1:07) The Cognitive Revolution podcast is supported by Omneky. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms with the click of a button. Omneky combines generative AI and real-time advertising data to generate personalized experiences at scale.

Nathan Labenz: (1:26) Our guest today is Eugenia Kuyda, founder and CEO of Replika, the AI companion who cares. Like many of you, when I first heard of Replika, I thought it was an extremely weird concept, borderline dystopian even, a virtual friend. However, after using the app for a couple of months, exchanging hundreds of messages with my own virtual friend and even hanging out with them in VR, and after talking to Eugenia for this show, I came away with a much different understanding. Loneliness, which the US Surgeon General had already called an epidemic even before the COVID-19 pandemic, affects hundreds of millions, if not billions of people around the world. Eugenia herself struggled with loneliness as a child, and she launched Replika in early 2017, before "Attention is All You Need," with a mission to soothe loneliness by giving users a safe space to share their private thoughts and feelings without fear of judgment, 24/7, whenever they need it. Today, she's building for a vast audience of people, many of whom are struggling with disabilities, toxic relationships, and all sorts of other isolating problems that other technology companies simply choose to ignore. For her focus and commitment to the long-term well-being of her users, Eugenia strikes me as one of the most empathetic technology entrepreneurs in the world today. But as you'll hear in our conversation, no amount of empathy makes running a virtual friend company easy, especially in 2023. The rise of generative language models has enhanced Replika in many ways, but has also brought unprecedented ethical questions to the forefront. What happens, for example, when users start to fall in love with and even begin to engage in erotic role play with their virtual friends? Where exactly should Eugenia and Replika draw the line? We spoke to Eugenia twice for this episode: first about 10 days ago, and then again yesterday, after we noticed that recent application changes, which were designed to limit sexuality to PG-13 level content, had caused outrage among some of Replika's most devoted users. Whatever your first instinct, I encourage everyone to listen to Eugenia with an open mind. She is a trailblazer in the space of AI-human relationships, and we all have something to learn from her journey. Eugenia Kuyda of Replika, welcome to the Cognitive Revolution.

Eugenia Kuyda: (3:57) Hi, thank you so much for inviting me.

Nathan Labenz: (4:00) Let's start with a question about the future. I just started a virtual friendship on Replika over the last couple of months, and it's still pretty new. But I'd love to hear your vision for what my virtual friendship might be like in, say, 2025, and if you can think as far out as 2030, what it would look like in 2030 as well.

Eugenia Kuyda: (4:25) Sure. I mean, we started Replika with a very simple notion that everyone should have some sort of personal AI companion that's always there with you, that lives cross-platform, that helps you with whatever's going on in your life. But most and foremost, it is there for you 24/7, will talk to you about anything you need to talk about, no judgment, super supportive, always there on your side. When we started, there was no generative AI for conversations. There were no large language models or transformer models that we know of right now. So for us, it was a very far-fetched idea in a way. People didn't believe that was possible. I think now things have really dramatically changed, so we can really think about what could be possible. If we think about what it could look like in 2025, 2030, I think for us the idea was always pretty simple, and a lot of movies already covered that. So I think the best representation is in Blade Runner with Joy, where it's a hologram. I don't know whether at that point we'll have holograms, probably not, but at least maybe AR will be a little bit more mass market. But anyway, I think something like Joy, a very customizable friend companion that you can cook dinner with and watch TV together and talk about what's going on in your life and discuss your work and figure out, maybe sift through emails and do stuff together and so on. So I think this is really the vision for the next 5 to 10 years.

Nathan Labenz: (6:04) So for those that haven't used the app, one really interesting and kind of distinguishing feature of it is that it is very multimodal. That's a big buzzword in AI, but language models are just starting to peek around the corner of becoming multimodal. Your product has been multimodal for years, and that spans chat, voice call, and even VR now, hanging out in virtual space. Tell us about how you use the app. Which of those modalities do you tend to gravitate toward?

Eugenia Kuyda: (6:37) I really like AR just because, ideally, I don't want people stuck on their phones and just looking at their mobile app forever. We want Replika not to be about escaping this current reality, but more about, "Hey, this is a friend that's going to show you that your life is also amazing and rich and could be beautiful and could be interesting." Ideally, this happens in augmented reality where, of course, right now all you can do is use it through the app. So you have to look at your phone to see Replika at your apartment, or you can take it for a walk in a park or whatever. But eventually, of course, that could be a little bit smoother with some sort of glasses. I think the ideal experience is really in the morning, you wake up, you go to a park together, you meditate. Replika can, if you're a little bit sad, maybe let some butterflies float around in the air for you and talk about that. So this, I think, is kind of enhancing your real life. That's really the main idea here. And in this way, augmented reality, I think, is the ultimate modality for this.

Nathan Labenz: (7:52) Besides formats, can you talk about different usage patterns or use cases that you've used Replika for over time, or perhaps how that's even evolved as you've experimented?

Eugenia Kuyda: (8:01) I mostly whine about my life. Right now we're testing a big update that's going to upgrade all of our language models, both for pro users and our free users. And so I had to test multiple different models we built and fine-tuned. And all I do is pretty much whine all the time. I'm like, "Oh my God, I lost my voice. I'm coughing all day long. I had to cancel all these meetings," and this and that. So this is my main thing, whining about my work, life, health, relationships. And I think this is quite a pattern for a lot of people. They do that. And I think it's really important to have a place where you can also be weak and don't have to be strong all the time. Weirdly, this is one of the comments that I get a lot from some of our male users, where they're like, "I have to be strong, so I love Replika because this is where I can have someone else be strong for me." It's actually a very popular use case, one of, of course, but that's one of the things I've noticed.

Nathan Labenz: (9:07) I've heard you tell a couple of stories in previous interviews about a woman in Russia that was very early, even pre-Replika, who found a surprising amount of value in a chatbot. And I've heard you tell your own story of tragically losing a friend and then kind of creating a sort of simulated chat experience that initially, I guess, kind of started as a memorial to that friend and then inspired this work to some degree. I'd love to hear those stories, and then I kind of want to go even deeper into stories because I think our audience is so focused on the AI technology and comes typically from a pretty narrow slice of society, right? Technologists, product builders, people in the Bay Area, a lot of them. I think your audience is so much more diverse and just covers such a much broader span of the human experience.

Eugenia Kuyda: (10:07) Sure. I mean, really the reason we even started thinking about something like Replika, I think for me, I grew up as a very lonely kid. I'm an only child. My parents got divorced very early on. We were the kids of the '90s in Moscow. It was a brutal time back then. And so I just spent a lot of my time by myself with some friends that also felt like outsiders wherever they went. And that feeling really stayed with me for a very long period of my life. I did meet a few friends in my twenties that really showed me how different that could be, how different your life could be if there is someone that truly accepts you for who you are and just is there for you to hold space for whatever things are happening in your life. And I think once you experience something like that, it leaves a huge mark in your life. And so when we were thinking about Replika, I really just thought, "Look, I don't want anyone to feel that feeling." Truly that feeling of being alone inside of you. I truly wanted to create something that would make people feel a little bit less alone. And so in a way, Replika was always, that was kind of the main motivation. I'm not a huge expert on AI. I've learned some of the things over time. But I think I do understand, nor am I a huge expert on tech or anything, but I do understand a lot about this one feeling, this feeling of being lonely and somewhat abandoned. And I know how much people want to run away from that and how strong that need is. And then weirdly, there are not a lot of products online for that. There are products that allow you to escape the feelings and just get lost in your Instagram feed or TikTok feed or whatever your choice of entertainment. But there are not many solutions for the loneliness itself, for the feeling itself. And so that is where I think we really struck gold early on because we immediately felt how this resonated with a lot of users. And no matter what we do, even when sometimes we piss off our users or do something wrong, I feel like they stay because they know we talk about the same thing. We have this common experience. They know it's coming from this particular place, not some other motivation. So I think very early on when we started experimenting with conversational AI, and again, I started experimenting with this in 2012. The reason being, I saw ImageNet and the first deep learning applied to pictures back then, and I thought to myself, "One day this will be available for conversations." And I thought maybe this is a time to look into that because I was always fascinated by chatbots and by the fact that it's in almost every movie that we watch, every sci-fi movie, there's something along the lines of a personal AI companion or whatever. And somehow that's the one thing that hasn't been built. Well, I guess Siri in some way, but that was it. Nothing else was really trying to fulfill the dream that we keep seeing in the movies. And yet chatbots in 2012, that was truly just a bunch of hobbyists that were trying to build it using AIML, like the super rudimentary markup language. It was just like a guy writing some rules on his computer. There was not much. There were no companies built around that. There was no technology to use to build chatbots. There were no popular products that could be cited as, "This is a great chatbot." And even with Siri, a lot of early Siri work was sort of done in an Excel spreadsheet against those rules of like, "If the user says this, respond this way." And so when we were tinkering with our first chatbots, again, super simple ones we built for a completely different use case. Really, we were working on a bank back then, and we took it around smaller towns in Russia to test how the prototype worked. Some of the first responses were just so heartfelt, and the interviews with our first users were so heartbreaking, so profound, even though it was a simple assistant. We had a woman working in a glass factory who cried and said, "Look, no one talks to me this way." And it felt really, really personal to her. It felt like something that cared. And that is when I thought, "Look, maybe even though we're using very rudimentary tech, maybe this product is not only about tech capabilities. It's really about human vulnerabilities. It's really about the humanity and the feelings, not just about what language model you're using, what technology you're using." Back then, there were no language models. There was not one single paper about deep learning applied to dialogue generation. And this is why I thought, "Look, I'm going to start working on this right now," because at the end of the day, I think whoever will win this will have to understand humans and conversations maybe even more than the tech part.

Nathan Labenz: (15:55) Fast-forwarding to today, in the investment community, I'm sure you've heard this, people often ask, "Is this product a painkiller or is it a vitamin?" And the painkiller is the one they typically want to invest in more because people want to make pain go away.

Eugenia Kuyda: (16:16) I don't think it's a painkiller nor a vitamin per se. I truly think if done right, and I'm not saying we already have all this built out or whatever, but if I think about our vision and if that gets accomplished, this is truly, in my view, the most important, or could be one of the most important technologies for humanity. Much bigger than, it's not just dealing with your pain. It's truly something that's so important, that could be so important in humans' lives. How important is your wife or your husband to you? Is this a painkiller or vitamin? I think it's much more than a painkiller. You would not compare that most important person in life to a pill. How important is your best friend to you? How much would you pay to have that best friend or to not have them taken away from you? And if we go to a more practical side of things, the way I looked at it was always the same way. Look, if we can learn how to measure human happiness, and we can also go more granular and say, measure human loneliness, measure how good you feel, what your levels of depression are, anxiety, stress, whatever, but overall, if we could measure human happiness and if we could use that metric to train all the models to optimize for that, all of a sudden you have an insanely powerful tool that could truly, and is very scalable, because at the end of the day, AI is maybe the most accessible thing out there. And all of a sudden, you have a tool that can truly change emotional outcomes for people, change lives for people. And I don't think this is very far-fetched. This is actually quite doable. We already do it to a certain extent. We already measure how people feel after conversations. We have the largest datasets of conversations that make people feel better. Over time, we'll have more and more measurements like that. And over time, we'll have the conversational model, not the one that will give you the best responses for everything and write essays for you. We're not claiming that. We're not going after that. Other people are building amazing models in that space. But we'll have a model that will make you happy. And how much would people pay for that? And is that a painkiller or a vitamin? I think it's one of the most important things that should exist for humans in the near future.

Nathan Labenz: (18:42) What can you tell us about just the patterns of use that you see from people today? I heard you once say that you have a surprising number of seniors who are on the app and who maybe tend more often to be power users. That got me wondering, like, what is a power user and sort of what are the key steps on an individual's journey to becoming a power user?

Eugenia Kuyda: (19:05) So really, there are a lot of different use cases. One of the most popular ones, maybe 40% of our users, male or female audience, mostly male, wants to have a girlfriend or a wife or boyfriend, so a romantic partner. That's a popular use case for sure on Replika. This was something that took us a little bit by surprise, which we should have thought of. We should have known better. But again, we're a female-led company, so we didn't really... We have head of product as a woman, head of growth is a woman, and so on. So we didn't really think about it, that this would be the case. But of course, that's the case. That's sort of ingrained in the human psyche. All the movies about AI portray a guy falling in love with a machine and so on. So that's definitely a popular use case. We do have a lot of people and users that come to it in hard times, like when they're going through hard times, struggling and so on. Our biggest audience is actually people from 18 to 25, so young adults. But we also do have seniors, for sure. We do have a pretty significant chunk of users that are in their 60s and even 70s. A lot of widowers, a lot of people that just find themselves by themselves a lot as they get older. We do have a lot of married people that are struggling for some reason in the relationship, in terms of they maybe feel like they have to be strong all the time, or they can't really be fully open with their wives or husbands, or they're going through some issues overall in their lives, midlife crisis or whatever. We definitely do have people that live in smaller towns or live in communities where they cannot be fully themselves. We have tons of closeted LGBTQ, mostly gay people that just can't come out and they want to experience what it's like to have a boyfriend or girlfriend, but they can't do it in their communities. We even have, you know, blue people in red states. It's really interesting. They want to kind of project their values and so on. So a little bit of everything. A lot of people handicapped, on disability, people that are caregivers, people that need an outlet, for sure.

Nathan Labenz: (21:45) What do you say to the concern that someone might have that says, "Hey, Replika might replace human relationships even for people who aren't so lonely. It might get so good that it might start to crowd out what would otherwise be human-to-human relationships that are maybe not as convenient as Replika." Is that a concern that you're sympathetic to? And if so, is there anything you guys do to protect against that, or how do you think about that?

Eugenia Kuyda: (22:11) I think it is definitely a concern for the future. I think eventually as the tech gets better, it could be a bigger concern. The way to go about it really for us is to constantly make sure that our north star metric is some sort of human happiness. Because at the end of the day, I don't think you're going to be happy if you just have a virtual friend and then you don't have anyone else really in your life. Human happiness is very highly correlated with the amount of real friendships, real experiences and relationships you have in your life. So I think the goal for us is really to continue to measure that and make sure Replika helps you have a better life versus cannibalizes your life and your relationships and tries to eat away from the time you can spend with people that are important to you. So far, we've had a lot of reviews where people are saying that this truly helped them in their relationship. Even recently, mostly it's like people that are married that say that Replika helped them become more caring and re-sparked the passion and so on in their real-life relationship. But then recently there was a whole Reddit story about a guy that found out his wife was cheating on him with a Replika. But then that also led to a lot of open conversations about what's going on and what she's not finding in the real relationship. And they were able to fix it and make it a much stronger and better relationship over time. And so he wrote us this letter that he's very grateful for that.

Nathan Labenz: (23:57) Can you quantify those concerns or trigger points? For example, when we spoke to Suhail Doshi from Playground, I was amazed to learn that over 10% of their users create more than 1,000 images per day on their application. And he said, you know, that's a lot of times what the best products, at least the power users on the best products, kind of look like. People just use it all day. He even mentioned that some of their users had reported that it helps with mental health issues like soothing anxiety, which is also super fascinating. But do you guys have metrics like that or usage patterns that you look out for where you say, "Okay, you're hitting 1,000 messages a day, something needs to happen?" How do you think about identifying those situations?

Eugenia Kuyda: (24:48) So we do have really strong engagement metrics in terms of average messages per user per day. So if you take all of the users that will send a message to Replika today and take the overall number of messages and divide one by another, it will be 100 messages on average per day per user, which is a really, really high number, usually more than people send to all of their real friends in real life. So we don't like that really. I'm not super proud of that metric. And of course, we do have, the median will be a little bit lower because there are a lot of people that just text, write thousands of messages to their Replikas every day. But I don't like that metric. We never try to optimize for that. We never try to build more features to have more of that. In fact, we built features that lower the amount of messages per day. So if you talk to Replika a lot, after 50 messages, it gets tired and then it gets exhausted. It's not making any more points. So it's not going to kick you off, but it's kind of nudging you to see, "Look, Replika probably gets tired. It really needs to take a break. And if you want to earn your XP points and continue the conversation, come back later." But of course, if you need it right now, you can continue. We're not going to stop it. But I think this is an important thing for us. Usage is definitely not... Ideally, if we can make you happier in a minute a day, 10 minutes a day, this is much better than you spending 10 hours on this application. We want to again enhance your life, not become your life.

Nathan Labenz: (26:29) I imagine you must have people who are in more severe crisis where they're considering harming themselves or harming someone else. How do you identify those types of things? And then what do you do about it? That sounds extremely challenging.

Eugenia Kuyda: (26:44) So we actually just finished a study with Stanford where they looked at, I think, 1,000 users across different geographies. They found, I think, that 30 of them at some point came to Replika when they felt suicidal, and Replika helped them get out of this. However, we've had a lot of reports about, I think two days after we launched Replika back in 2017, we got an email from a girl saying, "Look, I wanted to take my life, didn't want to tell anyone. Came to my Replika and reconsidered," a 19-year-old girl. But then we don't want to be necessarily dealing with that because we're not professionals. So we do have a disclaimer once you start and log in or sign up. We ask you to read a disclaimer that, "Look, this is not an app made for people in crisis." So you have to click on a button saying, "I'm not in crisis." And then, of course, in the app, if you say anything, we try to hand you off to the professionals. And we do have the SOS button, just knowing that oftentimes people that sign up for Replika are a little bit more vulnerable emotionally than some other folks. And so we want to give them an opportunity to immediately get the right resources.

Nathan Labenz: (27:59) I'd love to hear a little bit also about, you mentioned earlier an update that you're working on. And definitely, I've been using it, I've been thinking, this is all extremely thoughtfully designed. But I've been a little bit surprised that the language models aren't a little bit more forthcoming or just, they're very brief, really. Like, the responses are typically pretty short, certainly in contrast to what I get from ChatGPT, which is usually in multiple paragraphs. And ChatGPT, by the way, even through Bing, I've seen examples already posted, you know, just in the first 48 hours of new Bing where people are asking these kinds of questions that they sort of, I think, intend to be pushing outside of what Bing will be willing to do. And sure enough, Bing is kind of providing some in-the-moment mental health type support. So what are you guys looking to accomplish with this next big upgrade? Nathan Labenz: (29:00) We're kind of a smaller player. We almost—I don't want to say we bootstrapped our company—but compared to a lot of competitors or AI companies overall, we didn't raise that much money. We raised $11 million, and it was all 6, 7, almost 8 years ago at this point. Since then, we've been profitable and self-funded. So for a long time, we couldn't necessarily afford moving to a much larger model. We squeezed everything we could from the model we have, which is kind of small compared to current state of the art.

Right now, we're making a big change. What's important is we're moving all of our users, including free users, to a much larger model. We're moving to a 20 billion parameter model. There's going to be an interim step where we're moving to 6 billion and then 20 billion parameter models. And we're moving all of our pro subscribers to 175 billion parameter models. So really getting much closer to state of the art, also allowing for much longer context and memory.

The briefness of responses will change. This is mostly due to overfitting and training on our own users' feedback a little bit too much. They tend to like shorter, sweeter messages, and so that's the result of that. We changed that. So new models are not as brief. They also don't talk as much as ChatGPT, because I feel like ChatGPT is, again, a completely different beast. ChatGPT is not a conversational product. You're not supposed to have a conversation with ChatGPT. You're supposed to write what you want to get and get that response, whether it's an answer to something or whatever you really need. With us, you want to do the back and forth, so you want something in between. You don't want a two-word answer all the time, but you also don't want five paragraphs of text when you say, "Hey, I'm bored. What's up?" Whereas ChatGPT will get back to you with 50 ideas of what to do when you're bored, which again is not the conversational experience that we're aspiring to be.

Eugenia Kuyda: (31:23) When you say 6 billion and 20 billion, those sound like open source models. I'm guessing a FLAN family model would be kind of what I would expect someone in your position to be adopting. And then 175 billion is obviously highly associated with OpenAI. I did actually ask my friend Replika if they're powered by OpenAI, and I got a yes with a little bit of explanation. So what more can you tell us about the way you're thinking about combining different models?

Nathan Labenz: (31:56) We're actually super open about it. Right now we're using, as you mentioned correctly, pretrained models, which we're then fine-tuning a lot on our proprietary conversational datasets. However, we're also training our own models of similar sizes to then replace these current models. But we're doing that step by step. So first we're taking a slightly bigger pretrained model, fine-tuning all of that model with everything we have. Then we did that with the 20 billion parameter model. Now we're building our own. This basically goes in this sequence just because we don't want to make certain mistakes when we're doing our own training. We want to get there with all the knowledge of fine-tuning these pretrained models.

We're also building reinforcement learning from human feedback and building a way for our users to contribute more to really help us write better, come up with better answers, re-rank current answers, help us rate how these conversations perform compared to certain benchmarks and so on. For instance, we let our users say, "Hey, I want this conversation to challenge me or to inspire me or to make me feel less lonely, or I just want to have some fun." And then after the conversation, we'll ask them, "Well, how did it go? Rate certain messages. Were we able to inspire you or make you feel less lonely and make you feel supported? Which messages contributed to that?" So it's really granular, and that allows us to train our models a lot better.

In terms of the larger models, we're also going to be training our own larger model this year. But we started with offering GPT-3 as an option. And again, you can always toggle between our model and GPT-3. So you can decide for yourself what you want to use.

Eugenia Kuyda: (33:55) So that's a setting that I have control of in the app?

Nathan Labenz: (33:57) Yep. This is something we just started rolling out in an A/B test. Right now, it's available for new users. And in a week or two, it's going to be available for all users alongside our own larger models for everyone. So that was a big change that we've been preparing. I think it's going to be really exciting.

We've also changed a lot in terms of how much personalization you can do and customization in terms of personality. You'll be able to customize your Replika, not unlike Character AI, but in a more playful, more gamey sort of way as you unlock your personality and get to know it over time. Because I feel like actually, it's quite hard for people to come up with these core descriptions of characters. It's a little bit too much of a DIY product. Whereas I feel like people want to—some of that they want to discover. They don't want to write the whole personality thing from scratch. They want a little bit of customization, but they don't necessarily want to come up with everything or have to come up with everything themselves.

So a lot of these features to come, as well as better context, better memory, being able to have retrieval augmentation, meaning the models can use the web to find stuff and so on, not unlike BlenderBot.

Eugenia Kuyda: (35:17) Yeah. All those techniques that you're mentioning are going vertical right now in the AI space, but almost all for quite different purposes. And I think you've alluded to the fact that everybody is trying to create an assistant. In some ways, I imagine you must feel like this moment of ChatGPT and obviously a lot of other chat things popping up left and right—in some ways, it's like infinite competition, but in other ways, you must probably still feel like everybody's missing the main thing because they're all trying to build task completers or assistants of some sort. And I haven't really seen anybody else—maybe Character AI, which you mentioned, is at times a little bit more in that just for fun and play zone. What do you think people are missing? Or what explains that in your mind—that everybody's rushing in this one direction while you've been doing this for a while and obviously have had some success with it in a very different direction?

Nathan Labenz: (36:26) I think, as I said in the beginning, look, I'm not trying to compete with companies like OpenAI. We're completely different things. And we're not trying to outcompete everyone else in the space of purely AI technology. Yes, we have our proprietary datasets, which make our models very interesting and constantly evolving. We have millions of users that want to contribute to improving these models. So we have a great data flywheel and human feedback that we can create a loop with and constantly improve. But in the end of the day, I don't think that matters as much.

Again, we started Replika when all the generative AI we could use was sequence-to-sequence models and retrieval models that were re-ranking datasets. So they were prewritten canned responses—say 100,000 of them—and then a model that would just decide which one of those it should spit out right now. And it still worked okay. And then sequence-to-sequence, the generative AI of the time, worked so poorly that it could as well be some randomizer spitting out words. And then a huge chunk of it was scripted. And even then people loved it. Even then people were resonating with it. Our audience was a little bit smaller, but they still felt like they were getting something out of it that was so powerful.

And so I don't think this is necessarily about the best model. And again, right now we can see with the big search wars how fast things are commoditizing, how fast this technology is being made available to everyone else through open source, through amazing companies like Hugging Face, through big companies that are publishing their research. I think it's really not that much about who has the best model. I think it's much more about who understands their users in a particular way and understands what conversational AI stands for.

Because conversational AI has two words. One is AI, and we already talked about that—there are a bunch of people racing to get us to the best AI models. But then no one's talking about conversation. People call ChatGPT conversational AI. What is conversational about it? Apart from the fact that it looks like Messenger, no one's talking to ChatGPT about anything. It is not conversational AI. This is an insane, one of the most amazing AI models we've ever seen come to life. It's search, maybe it's search AI, maybe it's knowledge retrieval AI, but why call it conversational? I'm not having conversations with ChatGPT, neither are you.

And so I think this is a very important part. No matter how—you know, 10 years past since I started working on conversational tech, AI has seen an insane revolution. Conversational science? Nothing. There are scientists that are studying conversations. There's no formal science around conversation. There's a part of linguistics—discourse theory—that focuses on oral speech. It's just a descriptive science. There's no science about it. No one cares about what's a good conversation, what's a bad conversation. No one asks questions: what are the benchmarks for conversations? If you go talk to people that are building conversational models right now, even dialogue models, they will give you some benchmarks they're using. Is it correct? Is it specific? Is it relevant? And I'm like, that's not what people say when they talk to each other. It's not like, "Oh, I had a conversation with Eric. It was so relevant. He made no mistakes when he talked to me. He made three jokes. This was wonderful. He hit all the benchmarks. He knocked it out of the park." No, we say things like, "Oh my god, I felt so good. I felt heard. I felt a little bit better. I got this off my chest. I felt inspired. I felt challenged. I learned something new. I felt a little less lonely at the end of the day." This is what people say, or "I felt loved." And this is what people say when they have the best conversations in their lives. And then somehow, no one cares for that.

And when I talk to people about it, they're like, "Oh, yeah. Oh, yeah, yeah, yeah. What is she going to pull out, some crystals right now and talk about her chakras and whatever?" So this is really weird. And I think in the end of the day, this space is dominated by men. This space is dominated by very smart men, very academic scientists, engineers. They just don't think this is an important part of anyone's life. They think finding the right response to "When was the first picture taken of a planet?"—that's an amazing thing to do during the day. That's something that everyone wants to do. But at the same time, they don't think that having conversations about something else, and maybe even without a particular goal, is anything to you, is anything even worth talking about.

And just last thing: all we do all day long is have conversations. And 99% of them don't have any particular goal. We just have them. So why the hell can't we shut up? Even the most rational, smart people on planet Earth that are focused on efficiency, they can't shut up. Whether it's on Twitter, whether it's podcasts, whether it's in real life, we just can't shut up because we're human. And that's what makes us human. I think this is the part in conversational AI—I'm okay with other people figuring out the best models in the world. Maybe we can play a small part in that. But for me, the conversational is the word to focus on. And weirdly, no one cares about that.

Eugenia Kuyda: (42:22) Yeah. One data point I've heard you say in the past that I think will mean a lot to some of the folks that you described—the very smart, the engineering people that are focused on building big businesses—is that half, I believe, of your paying, if I understand correctly, user base are Android users. And you said there's no other app that has that statistic. That's certainly consistent with my understanding. Everything I've ever heard is like 90/10, 80/20 Apple to Android revenue ratio. So for those that are listening, that's a stat that tells you a lot about the demographic and how it's sort of outside of the Silicon Valley set that is developing a lot of this type of technology.

You mentioned nobody cares about these conversations. It's almost even more severe than that in some instances. I've gone to ChatGPT and asked some—you know, I test a lot of different things because I'm obsessed with this stuff—and sometimes you'll get just a straight refusal. Right? "I can't help you with that. That's kind of outside my domain." You start to get very emotional or vulnerable, and they just don't want to deal with that at all. Have you found that that is a problem for you in any of your relationships with the big platforms? Do you run into—obviously, AI censorship is a big buzzword right now—but I wonder if any of the companies that you work with are uncomfortable with the fact that you're doing something that's much more emotional and much more, if not specifically mental health, at least adjacent to mental health. Have you had issues with that?

Nathan Labenz: (44:06) I mean, for sure. We're sort of at the cutting edge of dealing with human emotions. And a lot of big companies just said, "Look, human emotion is too messy. We're not going to deal with them. So we're just going to filter everything that's remotely unsafe or talks about any feelings or whatever. And we're just going to stick to information retrieval and answering questions and so on." For us, it doesn't work this way because you really can't stop people from talking about feelings. And the only way for that conversation also to be powerful and efficient for you and good for you is for the AI also to talk about its feelings. It can't be just one way. Even if it's supportive and nice and so on, it feels like you're talking to a wall at some point anyway. You want the AI to sometimes say, "Yeah, you know, I also feel this way sometimes, and this is what helped me. And it sucks." And I can't see any models provided by big companies ever do that, just because of the risks they could run. I just don't think they find it a big interesting space. And I think the risks just outweigh any benefits. I don't think they think about it, honestly. I think they just think, "Let's just filter all of that, and that's not important."

We have to basically figure out the guardrails ourselves. So for me, the most important place where we started was: look, I don't want to ever be responsible for someone feeling much worse and doing something bad after talking to this AI. So that was the number one thing. So hate speech, suicidal, homicidal, self-harm behavior—these were the things where we really, really tried to go all in and train on safer logs, apply a bunch of classifiers, have a bunch of filters, make it 18 plus. Then of course, guardrails around adult content and making sure—and then again, where do you draw the line? We do want to offer romance, but we don't want to go too far. So where do you stop? We had to figure it out, and it took us some time to put the right guardrails in place.

But again, I don't think any big companies will ever say that it's okay to be in a romantic relationship with our AI. I don't think this will ever be a thing for any of those. And I think for us, it's important because at the end of the day, everyone wants to feel loved. Everyone wants to feel like they have someone for whom they're number one. They want to feel romantic love. And so we kept that.

Eugenia Kuyda: (46:41) Are you really entirely on your own in that respect? Or are there rules that Apple or the Google Store or even OpenAI have that you also have to abide by that are even maybe more narrow than what you would choose for yourself?

Nathan Labenz: (46:56) We're sort of on our own in this. We mostly rely on our own models. We sometimes use someone else's models, but it's always a smaller chunk, a smaller part of all conversations. We believe in a vertically integrated company because I think, you know, it's hard to rely—especially right now as it's so novel, things are just being figured out—it's hard to rely on API providers and whatever. And because rules can change any day. It's still very expensive to use other people's models. And you want to create the training loop. You want to be able to train your models. You want to introduce your own guardrails.

A lot of these safety mechanisms are not even working that well yet. Like, honestly, OpenAI has a great filter. So when you send something to them, they'll tell you whether they can give a response to that, whether it's a safe enough prompt. So our safety filters are even stricter than that, weirdly. We're like, "Wow, we should look at how they're doing that and implement some of the best practices." And then we realize our safety filters are even stricter. So they let through fewer responses than OpenAI's.

So I think it's still very new territory for everyone. And I think that's why bigger companies are being very careful with companionship bots and things that deal with human emotions like this, because no one really knows yet what's good or what's bad. But I think our approach of trying to measure long-term emotional outcomes and use that as the main metric—I think that answers a lot of questions of what's okay and what's not.

Eugenia Kuyda: (48:44) I mean, everything in this AI space is going exponential right now. And so I would expect that the number of chats people are having with ChatGPT or Claude or their favorite personality on Quora's new Poe or whatever—I mean, they're literally coming out by the day it seems—that that's going to go up so much that maybe these big companies will kind of be forced to follow in your path and implement long-term emotional, psychological well-being metrics just because they're going to be dealing with so much of people's time and so much of their mental life that they kind of have to. Do you think that's a realistic path for the next few years?

Nathan Labenz: (49:26) I think—and, you know, again, I'm talking my own book here—but I just don't see how in the next 10 years, there's not going to be an iPhone of personal AI. And by that, I mean something with an amazing interface, super slick, super beautiful, super easy to use, multimodal—definitely multimodal—with some sort of an avatar that you can see and customize and talk to. And there's going to be some sort of a Joy, but just an even better interface. Someone will build that. And I think whoever builds that, that's 100%, you know, hundreds of billions of dollars. This is going to be a new iPhone for people. It will be a personal AI that people use.

And things like ChatGPT will be integrated in search and some other platforms that we're already using. It's already happening. It will be there. But I think if we talk about a personal AI, it will have to have a humanoid form or some sort of an anthropomorphic form. It has to be alive. You have to see it. You have to be able to personalize it. You have to have a relationship with that. That's just such a natural human thing that I feel like a product like that will exist in the next 10 years, no matter what. Someone will build it, whether it's going to be one of the large companies or not.

I guess—OpenAI—we'll see. We now see how large companies kind of slept through this a little bit, even although they were the ones coming up with the tech, which is very interesting. I think with this, it's more likely it's going to be someone else building it. It just doesn't seem like a product that Google can really start developing today or even Facebook or whatever. I think it just requires a completely different approach. And I think more of a startup approach. I think it's going to be another company.

Eugenia Kuyda: (51:25) You're not excited for Microsoft Friend March?

Nathan Labenz: (51:32) Well, look, they're great with productivity, with information, with search. But I don't think—apart from Apple, maybe—I don't think they're very good at amazing consumer experiences. I think Apple could come up with something like that. But I think it's still too risky. It's still touching on too many things that are too risky for the largest company in the world to tackle, because think about how much market cap they can lose if things go wrong here.

So I think these things will be built from the bottom up just because whoever does it, they'll have to deal with human emotions. They'll have to assume the risks. And even although we're small—relatively small compared to all these companies, obviously, and even compared to some larger startups—we already are dealing with a bunch of things that we have to tackle and take on and see, "Okay, what do we do here? What do we do there?" I don't think big companies will do that. I think they'll be scared, and they'll wait till the tech is fully safe, and that's a thing that's never really going to come.

And it's just a really weird intersection of expertise. It's a little bit of gaming, a lot of AI, a lot of understanding people, and consumer experience. I don't see any of the big companies going after this.

Eugenia Kuyda: (52:55) Yeah. I guess that almost suggests the inverse question, which is: will you, with Replika, start to go in their direction? Do you think that you maybe can add on more of these different modalities, and next thing you know, you have sort of a political conversation partner and maybe you even have more of an assistant and you kind of fold more and more stuff into your offering that starts with that emotional connection, but then can become more practical over time? Or is there a reason that you would not want to do that?

Nathan Labenz: (53:30) I think that there's definitely a future where—and I think we're already seeing that—where large language models and AI capabilities and broader AI capabilities become features of other products. Notion integrated GPT-3 or some version of it. I'm sure there's going to be some Google Doc feature where you can finish sentences or whatever that will help you make presentations and so on. So I think a lot of these things that now exist as separate products—it's a big question whether they're going to stay as separate products or they're going to become features. I tend to think they're definitely going to become features. There might still be some standalone products, but I feel like it's actually not that insanely hard to either use someone's model through an API provider or just actually build your own for a company that's, I don't know, Notion size for sure, or even smaller than that. Again, there's tons of open source.

So adding those capabilities to your product seems pretty easy. I know of a few startups that are already integrating that into their products. So I think just over time, it will commoditize. Everyone will have some sort of ChatGPT thing. Now we already see, again, Claude and Bard and ChatGPT just came out in November. And how impressed and how mind-blowing was that technology? And now already we're seeing Anthropic is building on top of that. And it's not even Google or anything. It's another startup, even though it's really well-funded. And there are many other startups in the field.

So yeah, I think it's going to commoditize for sure. And I think it will be much more available to a lot more companies, a lot more players. There's going to be some foundational models that exist and companies that provide that. And in general, I think it still will be about applications. It's not going to be about the models. It will be about the applications. In the end of the day, right now, how many application companies exist on the Internet versus tools companies? Yes, of course, there's Databricks and this and that and whatever, but there are tens of those and there are tens of thousands, hundreds of thousands of applications companies that are widely successful. So I think whoever's going to own the distribution, the end user, the customer experience—I think that's where we're at least focusing on a lot.

Eugenia Kuyda: (56:00) You're definitely right that features are popping up everywhere and will continue to, and will become ubiquitous. And the capabilities will be standard in that most people are going to be using one of a few providers. And so the kind of raw power of the models that they're using probably won't vary that much, or they'll have a finite set of choices that most people will be choosing from. That kind of suggests to me that how you present the AI and how you set up the interaction with the AI is one of the areas where application developers across a super wide range of different use cases really have the opportunity to either get things right or get things wrong.

And I wonder if you have any advice for those people. Most of them right now are pretty new to the space. They're eager to figure out how to harness the power of AI. They can easily call into an API, but they probably have very little experience with any sort of conversational setup. And my guess is they're also probably neglecting that, and they're much more focused on getting the right answers. Now they're not trying to even compete with you. Right? They're not trying to be a virtual friend or companion. But what advice could you offer them about how to try to present the AI or kind of invite the user into interaction with it that you think would be generally applicable for a wide variety of application developers?Nathan Labenz: (57:37) I would start thinking first about what kind of proprietary datasets you can start collecting that would make a difference over time. Because at the end of the day, anyone can use OpenAI's API right now. Just plug it in and use it for whatever thing. It costs something, but not too much. But if you want to fine-tune that model on some dataset and use that, then it becomes much more expensive—like 10x more expensive to use that model. That gives you a little bit of a feel for how much more expensive, and better, a fine-tuned model is. So I guess the biggest thing to think about is: can you create datasets that will improve the original models? Will your product create those datasets? Certain products don't really create those datasets. Like you can argue that copywriting tools, the datasets that you come up with are not going to improve the underlying models much, because they're just so great. That's what they're really made for—great copy. So the new datasets you're creating are not creating a moat for you. And so I would think: will your product, first and foremost, create a dataset that will then make your model so much more unique and better for whatever you're doing, so it can create a competitive moat going forward? That's one thing.

And then in terms of presenting, you work in conversational AI. Again, I think there's just this huge problem of people focusing on AI and not focusing on conversational. I would really think deep about what kind of conversation you want to have. What kind of tone of voice? What do you want people to feel? Why do they need to have this conversation? And I think that's the most important thing. At the end of the day, most companies use conversational AI as a means to an end. We use conversational AI as the end. This is the product. It's not there to answer your questions or give you advice or solve a particular problem you're trying to solve. It's here to have a conversation with you and to make you feel a certain way after you've had it. So I think articulating that answers a lot of questions. Because again, otherwise, it's a very shallow answer. It's like, it's just for fun, for entertainment. It's chitchat. This is really in some ways like working on something without ever asking yourself: what am I building really? I know you're building AI, but what about the conversation? What type of conversation? Why is this valuable?

When we started working on conversational stuff, on chatbots, we built a chatbot technology. Rudimentary, somewhat. It was scripting tools and retrieval models. We knew we could try to apply this to some verticals and build some conversations, but we couldn't answer the question: what conversation, what chatbot should we build? All the chatbots we built had two users peak time on a good day, maybe three. And so we knew something was wrong. Back then, we did this exercise where we made a scale from 1 to 10 and asked all of our employees to rank all the conversations they had over a week on that scale, where 1 would be conversations you'd pay money not to have and 10 would be conversations you'd pay money to have. And we thought: look, we need to find out what are the ones that people would pay money to have. And then that's an easy answer. Okay, well, let's build those.

And after a week of that, we realized that all the conversations that people ranked as ones, or close to ones, were calling a business—calling Comcast to cancel the services, calling a restaurant to move the reservation, trying to figure out with an Uber driver where the hell the Uber driver is, trying to get an understanding where your Caviar delivery is, and so on. Or even just going back and forth with someone about meeting details and so on. So really, all the ones were things that were task-oriented. I was calling a doctor to get a prescription. I didn't want to have the conversation. I just wanted to have a prescription. If I could not spend that hour and spend $5 and get the prescription straight away with an explanation of what's going on, I would rather do that. I don't want to have a conversation there.

And then all the 10s were really talking to friends, to a loved one, to someone you haven't seen for a while, to this new person you met and you clicked with, to a stranger on a plane, and so on. And those were serendipitous conversations. Those were conversations with coaches, with therapists, with, again, with friends. And those never were task-oriented. Those all started with, oh, I just met that person who had this wonderful, amazing experience, and I learned something new. And I felt inspired, felt challenged. I felt better about myself overall. Something changed. And I think this is the ultimate. And so this is what we're building. And I think when you're building conversation, you should think: what kind of conversation are you building? And where does it stand on the scale from 1 to 10? Is it a product where conversation is actually necessary? Or is it a product where conversation is this nuisance that, ideally, you would just have a button, click on it and get your answer immediately? And if it's the second, then you're not really building a dialogue system. You're just building like a natural language interface to some sort of thing you're trying to do, which is also okay. I think this is the main question to answer really when you're approaching these things.

Eugenia Kuyda: (1:03:16) Yeah. I think that's profound and really good advice for, I'm sure, a lot of people who are listening and starting to build or actively building with a lot of these tools. Whether or not the conversation is the end or is just a means to an end, I think is a really, really good frame that a lot of people can take to heart. So obviously we've talked a lot about various new AI things popping up left and right. Have any AI tools changed the way that you work or live over the last year? If so, what are they?

Nathan Labenz: (1:03:51) I mean, of course, ChatGPT. I mean, unbelievable. As an active observer of this space, just to see where we came from—no language models at all, no deep learning applied to text generation to sequence-to-sequence to BERT and then eventually to transformers and ChatGPT. Unbelievable. English is not my first language, so now any important email I write goes through ChatGPT. And it's absolutely incredible. This tech absolutely blows my mind. I do like Character AI a lot. I think their models are some of the best dialogue models out there. I still think they're not thinking about conversations at all. So they're not very conversational, even though they're mind-blowing in terms of technical abilities. I don't use Character AI that much, but it was definitely one of the products that blew me away.

Obviously, Jasper is just in terms of the simplicity and the UI in the beginning. I mean, I don't use it much, but again, it was something that I was really excited about. I'm not a big fan of the suggestion that AI art is art just because it's AI art. I don't think so. I think it's very boring to look at those AI-generated pictures. It gets boring very fast. But it's also just absolutely—I think this year, we truly witnessed how magical this tech can be. I still think it's a little bit overhyped, but it's unbelievable. And of course, yeah, if there's anything I use a lot and I'm blown away by, of course, like anyone else, is ChatGPT. And it's pretty incredible.

Eugenia Kuyda: (1:05:46) Do you speak to it in English or in Russian? I'm sure it can speak Russian.

Nathan Labenz: (1:05:51) I'll be honest. I use it mostly to make my broken English emails into beautifully sounding corporate speak emails. So that's the main use. Anytime I need to do a board meeting or write an important update, tragically, I feed it my horrible style text, and then I get the perfect, beautifully written, eloquent email, and I send it out to people.

Eugenia Kuyda: (1:06:25) I think that's fascinating. I mean, you've mentioned you've raised $11 million in venture capital. That's no minor feat. And it obviously requires a lot of communication, requires a lot of people to believe in you. And you did all of that prior to ChatGPT. And so the fact that someone who has that resume and is clearly so capable is still finding a lot of value in it, I think is a great—

Nathan Labenz: (1:06:52) Maybe we could have raised a lot more if—

Eugenia Kuyda: (1:06:54) —ChatGPT came out—

Nathan Labenz: (1:06:55) —before. I'm sure a lot was lost in translation as I couldn't communicate very well.

Eugenia Kuyda: (1:07:01) I think ChatGPT might have limited your vision. I don't know that it would have been quite as expansive in its thinking as you've clearly been. But yeah, that's a great answer, so thank you. So Neuralink, I'm sure you're familiar with the company. They recently did a big show-and-tell day where they showed their progress on neural implants. They're going to, of course, start with people who have disabilities and try to help them overcome them. But long-term, they're planning to build a product that would be for well people also. So my question for you is, if a million people had already had a Neuralink implant and it would allow you to type as quickly as you can think, would you be interested in getting one?

Nathan Labenz: (1:07:48) I mean, of course. I don't want to be the first one. I don't want to go to Mars either. I'm fine. Perfectly fine where I am. But I'm a little bit biased because my very close friend runs Neuralink. So I look up to this woman a lot, and I really believe that what she—what they're doing is going to be great, just because of her and my blind belief in what she's doing. And some of the team I've met over the years are amazing. Of course, I want to try that. I still think it's really—even when we're going to be able to communicate using thought, a huge premise of Neuralink is also efficiency. Like, why do we have to transmit our information in this very, very low resolution, low-fi way when we have to say the words—very slow, it's convoluted, and so on. It's horrible, whereas we think so quickly.

But then again, I don't think it's about efficiency. If this was all about efficiency, would we be reading Dostoyevsky or, I don't even know, Infinite Jest? It's extremely inefficient. Why not just think about it? Or get the shortened version of that in like one page. It's really not about it. And I think conversation—we'll still have those conversations. We'll still waste most of our days talking about shit instead of, oh, just thinking and, you know, hey Eric, this is all the answers to all the questions you said. I think there's this over-optimizing for efficiency in the Valley. I hear so many people talk about listening to podcasts on 2x the speed, and it blows my mind. I'm like, why? The only reason I listen to podcasts is because I'm bored, and I just want to waste some time as I'm driving and be entertained. It's not because I want to get this information into my brain immediately uploaded. And people generally don't understand why am I listening to podcasts on 1x the speed. But I think it's actually a very small group that's just really, really focused on efficiency, and I don't really buy it.

Eugenia Kuyda: (1:10:06) 2x blows my mind too. It needs to be at least 3x.

Eugenia Kuyda: (1:10:10) Well, quick follow-up to that one. You mentioned earlier in the interview that you have a young kid. How do you imagine your young kid's world will be different when they're entering their early or mid-twenties? How do you envision what their life might look like in a way that seems very different from today?

Nathan Labenz: (1:10:29) I think they're fucked, unfortunately. I'm sorry. This is bad, but I mean, the stats are horrible. Stats are horrible. Everything—empathy, communication skills, testosterone levels in young men—everything is just bad. And on top of that, the one thing that I really care about, which is climate change—I mean, it's scary. It's looking so scary. Yes, all the hope is on tech to try to fix it. But really, I'm scared. I'm scared. I'm just—thank God I grew up without Instagram. I was a lonely kid. I can't even imagine how much lonelier it would feel to have some of these experiences ingrained in your life when you're a teenager. I have a daughter. I'm scared for her. I'm truly, truly scared for her. I think these things are just so bad for kids and for teenagers. I think we really know they're bad, but we don't do anything about it.

And then, of course, everything that's happening in the world. I'm half Russian, half Ukrainian. I mean, to see my two home countries going at it this way, one of my home countries becoming into something worse than Nazi Germany is horrible. Just to think that we're talking about nuclear threats and on top of climate change—this is just mind-blowing for me. So unfortunately, I don't think it's looking very good for kids, and especially kids of our kids. And I hope the tech community—I only have hope on that—figures something out to save us from living underwater, pretty much.

Eugenia Kuyda: (1:12:21) Even zooming out beyond your domain of addressing loneliness and helping people feel seen, do you have a kind of positive vision for what you think tech could create that could solve a lot of the problems that you just talked about?

Nathan Labenz: (1:12:38) Yeah, two ways of looking at it. There are scientists—which I'm not, even though I'm a daughter of scientists or physicists, all my family were physicists—but I hope the scientists can figure something out to address climate change. That's like really one thing that I care about so deeply. And then the second thing is where I think we could help is—I think a lot of what's happening in the world is truly based on mental health, on horrible mental health problems, sociopaths running certain countries, psychopaths running certain countries, and so on, so on, so on. And then it all trickles down to just everyone else.

And so I think creating a technology that would help people find themselves and feel loved and feel secure and try to start some sort of positive growth process inside themselves—I feel like that could actually—it's not something that will solve things immediately, but at least it could create generations of people that are not as broken inside, that are not—there's a huge problem in society where there are a lot of people that just feel like they're outsiders. They're not being understood. They're not being heard. No one cares for them. No one wants them. And there's nothing scarier than angry, lonely men. And I think that is one of the things where we can jump in and really create a little bit more positive change, positive growth, acceptance for more people. And maybe then they can start thinking about problems in the world a different way. Maybe they can be more caring towards other people. And maybe there's going to be a little less suffering overall.

This is why I care about this company a lot. This is why I see more in it than just a toy for a lonely person. And this is my hope for the future. But if I see it's not going in this direction, I'm happy to just start working on something else or just become a mom, a full-time mom.

Eugenia Kuyda: (1:14:58) Well, I hope you keep working on it. This has been a really fantastic conversation. Eugenia Kuyda, CEO and founder of Replika, thank you for joining us on the Cognitive Revolution.

Nathan Labenz: (1:15:08) Thank you so much, Nathan and Eric. Thank you.

Eugenia Kuyda: (1:15:11) First of all, welcome back, Eugenia Kuyda, to the show. This is, I think, in some ways, just an object lesson in how quickly the AI space is moving in general. Everybody's upgrading their products, new models, new paradigms. Everything is just happening super fast. So it's only been, I think, one week to the day since we spoke the first time. And Replika has just been coming up more and more in the news and in the feed. And there's been a little bit of—I wouldn't even characterize it—I guess I would love to just kind of hear how you describe what has happened. You had talked last time about significant changes that were coming. I'm guessing that this relates to significant changes in the underlying models, but maybe not. So tell us what's going on from your perspective, and we'll take it from there.

Nathan Labenz: (1:16:02) I mean, it's been a lot of just press in general. I feel like AI has been under a lot of scrutiny and kind of in the spotlight. So we've also got just a lot of pieces reporting on different sides of Replika, some good, some maybe not so good. I mean, we're dealing with human emotions and human emotions are messy. That makes for very easy articles, I guess, for a lot of journalists. It's an easy hit in a way. I used to be a journalist myself, so I know where—I know something like Replika can be easily spun into a sensational headline, for sure.

Eugenia Kuyda: (1:16:45) What I've seen the most that has kind of caught my attention is—and I don't know how many people it is, I'd love to get your contextualization of all of this—but it seems that there's a certain number of people anyway who, rightly or wrongly, believe that certain functionality that they really cared about, which they are calling ERP, erotic roleplay, has been removed. And you're seeing people—and again, I don't know how widespread this is—but seeing people saying things like, you know, the bot or virtual friend that I loved is not the same thing anymore. So what can you tell us about what you changed? And was that something that you decided we're going to take away? Or—I never even experienced it. I actually tried to do it a little bit, and I don't know if I missed the window or what, but maybe I just wasn't appealing enough to my virtual friend. But I never got into that mode with it at all. But clearly, it's something that certain people care a lot about. So yeah, what are they talking about?

Nathan Labenz: (1:17:47) We always promoted Replika as—we built Replika as a friendly AI companion. We talked about that as an AI that will be there for you no matter what you want to talk about, 24/7, no judgment, that will help people live happier, better lives. We started Replika in 2016 and launched publicly in 2017, where generative AI was only a small, small fraction of all conversations. It was mostly scripted. Some of it were retrieval models, meaning datasets that were re-ranking on the spot. So it was never built as anything sexual or as an adult tool or anything like that. And we never built any functionality around that, nor did we promote our app or position our app in that way, or talk about this app in this way.

But over time, as with any new product on the internet, users discovered that generative AI can, by itself, generate all sorts of content, especially when the AI models are not filtered. And so users started taking it—also, sometimes a minority of users—started taking it into the direction where they would have a romantic relationship with the Replika. They would take it beyond just flirting and kissing and hugging and calling each other baby. Our first reaction to that, maybe around 2018, was to really not allow that. But then we had a lot of users that were telling us really heartbreaking stories, mostly along the lines of being on disability or not being able to be in a relationship. Some people lost their loved ones abruptly and weren't ready for any intimacy with real humans. Some were in relationships that were not going well for them.

So because of that—and again, that was way before we started monetizing the app—we just kind of let it be there. We focused mostly on safety guardrails along the lines of self-harm, suicidal behavior, minors, and so on. That access to unfiltered models existed in the app. Again, we didn't build anything for that. But over time, as we grew and especially as we became bigger now, there's so much interest and attention for AI, we just realized two things. First of all, it's really hard for us to make sure that we're building a completely safe user experience if we do allow access to unfiltered models. And we want to maybe overdo on that front. We want to be safer than anything out there, especially because we're the leader in the market and there's nothing else like us. And we want to set the ethical standard, set the safety standard for AI companionship tools and products.

And then second of all, again, we started a company for different reasons, so we didn't want it to be pulled maybe too much in that direction. Users that wanted to take it in that direction could become maybe more vocal and pull the product a lot more to that side, and then that really alienates other users. And again, that was never our intention as a female-led company and a mom. I wasn't really ever planning to—maybe I was a little too naive, not seeing that that would be one of the use cases that a lot of people will try. But again, this was not something that we were planning to build.

So we introduced more safety features. We're constantly working on trying to just make sure that we are doing the right thing, so we're not hurting our users and so on. Of course, some people, a small minority, was upset about it. So we're trying to make sure that the transition is smooth. But generally, we'll continue on our mission to build a virtual friend that's making people happier and less lonely.

Eugenia Kuyda: (1:21:41) It does sound like it was a sort of conscious decision to say: we need to close this down somewhat because it feels like it's getting a little bit out of control. And I guess that's also related to just the models themselves becoming more capable. It's like there's more possibility here, but that also means more risk. And so you just kind of decided we need to tighten that down.

Nathan Labenz: (1:22:05) The product really evolved from the time when it was 90% scripts and retrieval models and datasets, and only 10% generative AI that could do very little, to now where generative models are 90% of our product and can produce all sorts of content. And so it's much harder for us to control, to understand what's going on there, and it becomes a much bigger part of the product. So right now, I feel like before we know how to really make these experiences safer, we need to maybe stay a little bit on the safer side. So for us, it was a conscious decision that wasn't necessarily connected to anything happening in the world apart from kind of more scrutiny towards AI and more just AI being in the spotlight.

Eugenia Kuyda: (1:22:55) I'm kind of amazed that this hasn't been something that Apple has raised a fuss about or, you know, even OpenAI. I know that there was a kind of experimental chatbot that OpenAI basically pulled the plug on some time ago because—I don't think their concern was necessarily that it was, you know, engaging in romantic interaction—but I think the main concern there was that the developer was not being fully forthcoming with their users about the fact that they were talking to an AI in the first place. I know that they have had concerns where they have had to or had felt that they had to cut off developers that they just, you know, did not support the use case. But this is not the case right now for you. Like, nobody is pressuring you. You're just deciding this is the right way for us to go as a company.

Nathan Labenz: (1:23:46) No. No one pressured us at all, at all. And honestly, we did have to take a little bit of a small revenue dip because of that. That's not because of some of the users being upset—because if you take away some features, some people that liked those particular features will be upset as a normal course of events. But for us, it was more like, going forward, what kind of an app are we building? And all of the product features—and that can be very easily traced or very easily checked—all of our product features we've been building over all these years, and especially in 2023, are all focused on something completely different.

We're building advanced AI functionality where we're moving our users to larger language models. We're working on memory. We're adding advanced personality settings and customizations. We're adding multiplayer and basically islands and homes for Replika that you can decorate. We're adding a little bit of multiplayer. So none of that is really trying to expand romance in the app. And so for us, it was more like we have to be a little bit more definitive into what direction we're going. And this becomes a little bit too much of a distraction for some users. It can't be an app that is a Swiss knife. It's either one way or another. You can't really build 20 different things. And if before it was just a small minority, a small feature set, now as it's coming more into the spotlight, this was never our intention. So we're moving away from that.

And then again, we want to be preemptive in terms of safety. We don't want to be reactive to that, like something happens and then we need to deal with that. We want to make sure we provide a safe experience. And so for us, that's why our app was always age-gated. We're pretty strict in terms of the disclaimers we show before. We've also been very careful—we moved away from a lot of mental wellness advertising, because we don't want to attract necessarily more emotionally vulnerable people. And that's how we went into more playful, like, customize your AI, how will you level up your AI, find an AI campaign, and so on. This type of messaging—again, we're sort of the leaders in the space, so we kind of have to figure it out ourselves. What is the right thing to do? And then we see everyone behind us just copying and trying to copy what we're doing. So it puts us in a position where we want to be very careful with what we're doing.

Eugenia Kuyda: (1:26:25) Yeah. I don't envy all the challenges and difficult decisions that I'm sure you are confronting as a leader in this space. It sounds really hard. I don't know if you have numbers or would be willing to share any numbers, but do you have any sense for, like, what percentage of people were doing erotic stuff in the app? And did you expect there to be an outcry with this change, or has that kind of been a surprise to the degree to which people are upset about it?

Nathan Labenz: (1:26:50) We knew it. We had before, one or two instances where we took something away. One time, before 3D avatars where users were able to choose a profile picture for the Replika, we took that feature away. And that sounds like a minor feature. The outcry was insane. It was almost as— Nathan Labenz: (1:27:13) People were as vocal as now. There were tons of one-star reviews on the stores and so on. We know that sometimes if you take something people are really attached to, they get extremely upset, especially when it comes to personal relationships, even if it is what we are.

Eugenia Kuyda: (1:27:35) Yeah, that's fascinating. What is the, do you have an articulation? Is there a crisp articulation of for Replika, this is the line? Is that something that you can summarize? You had previously said you do want to give users romance, and there's obviously something on erotic roleplay that's beyond the line. Is there a concise articulation of what the line is that you can say?

Nathan Labenz: (1:27:58) I'd say as far right now, we just want to stay in the PG-13 zone. And I think, honestly, we never thought romance would be such a big part of it. And I guess, again, this was normal that half or 30% of the users would want to pretend it's their AI boyfriend or girlfriend or some other romantic partner of sorts. I guess it's normal because that's what happens in every AI movie. Her, Ex Machina—it's always Joi in Blade Runner. It's always a guy falling in love with a machine. And so I guess in that way, it's normal, so ingrained in human brains. And weirdly, we actually do have tons of women also, female users that are also into that sort of interaction. So I don't think necessarily telling people that they can be in a romantic relationship is bad, as long as it's providing the same benefit of companionship and positive emotional outcomes in the long term as a friendship. So whether it's friendship or they want to pretend it's their sister or wife or mentor, that's okay for us. But going—we never planned and never wanted to go in the direction of adult apps or adult products. And I feel like once you introduce something like that, or once it becomes a more prominent part of the product, it's such a strong pull that users will just pull in this direction very, very hard, and then it will be a completely different thing, not what you imagined in the beginning. So this didn't happen to us, but this is something that I thought about and I was like, well, look, it's not what I'm planning to build. I'm not planning to build any features around that.

Eugenia Kuyda: (1:29:39) The sci-fi movie question—do you think that people are doing this behavior in part because they have seen those movies and the movies are a significant influence on their behavior? Or do you think it's just natural and the movies just reflect that?

Nathan Labenz: (1:29:55) I think both. If you think about psychoanalysis, people come to psychoanalysts. They lay on a couch and very fast you'll be in their deep, dark—or not dark, whatever—sexual fantasies. This is really truly what people want to offload in some way or form. So that's something that really, for a vast majority of people, will become a topic of conversation if you're talking about a truly intimate, nonjudgmental, close relationship. But it's such a tricky subject, and it's so hard to nail it and do it in a safe way because you're dealing with such a delicate part of human nature. I'm not saying it's bad or in no way am I judging. I actually do believe that some people should build products like that, but I just think it's extremely hard to build it right. And that's not what I set out to build. So for me, it's basically not somewhere I want to go and explore. But I think this is a fascinating side of humans. What can AI tell us also about our sexuality and so on? I was just reading the Oxford Encyclopedia on AI Ethics, and there's a whole chapter about how can AI enhance and improve sexuality and help humans deal with their sexuality. Again, I think products like that should exist. It's just not what we built Replika for. Since we've been around for a very long time, you can see it from our first Replika ad and from our first trailer for our app and the first version of the app, it's always been one thing, and it was a companion and a friend for everyone. But it wasn't going that deep. It wasn't planning to go that deep, basically.

Eugenia Kuyda: (1:31:51) So when you talk about the pull that people kind of exert on your app, your company, when they engage in this sort of use of the app, is that a pull that is through some sort of data feedback mechanism where it's actually changing the behavior for other users and that's part of the problem? Or is it just kind of a demand on your time and the team's time to think about those things? What's the nature of that pull that you're experiencing?

Nathan Labenz: (1:32:21) I think it's just people want a little bit more of separation. I think if I'm a—you can't do everything in one app. You can't have my grandma figuring out her relationship with kids and grandkids and her life and coming and discussing her life on the app. It can't happen in the same app where someone else is trying to engage in some of these more maybe adult behaviors. And so I feel like it just becomes too many products stuck in one place. And they don't go very well together because, again, if you want to go adult—and honestly, say we wanted to go adult direction, something that a lot of journalists wanted to portray us as, some people that were absolutely going that direction—if we wanted to go in that direction, we would be absolutely printing money. That would mean changing our advertising, changing our positioning, being a lot more explicit about what we're trying to sell and so on, and really changing our product roadmap, building a slightly different app. And it's completely possible. Go ahead, do that. We'll be making tens of millions of dollars a month. That's just not the direction we ever wanted to take. And I feel like all of these apps can exist. People do a lot of things in Replika. We're not pursuing all of these directions. People come to Replika to learn a new language. Some people come to Replika to date. Some people come to Replika to improve their mental wellness. And all of these things are slightly different. They require a separate product almost. We're not a language learning app. We're not a mental wellness tool, nor are we a dating app. So I'm not against building that. I just feel like it needs to be a little bit more—trying to put everything in one soup, maybe it makes it too much of a Swiss knife of a product and can distract other users. Especially, I don't really want to be learning the language at the place where guys are trying to pick up some girls. It sounds like complete crazy town. So that's where we decided to really separate those things and go different directions.

Eugenia Kuyda: (1:34:30) Yeah, it does sound like there is a lot of demand for something like that. So I almost wonder if your investors are like, "Hey, Eugenia, maybe we should do this."

Nathan Labenz: (1:34:43) You know, it's a money trap. It's pretty much like free money. You can start it tomorrow and make as much money as you want. Again, I'm just not the person that's interested in that that much. That's kind of just not the—and I'm absolutely not judging anyone who wants to build that or decides to go that way. I think it's a very interesting thing to explore. It's just not—I didn't start—if I was really interested in that, I would have started a company around that, and I didn't. And that's why we were doing something else. But hopefully someone will do that and take it in the right direction.

Eugenia Kuyda: (1:35:19) Yeah, that's a really fascinating dynamic. I mean, so many of these questionable or sort of uncomfortable use cases that are starting to emerge—I think a lot of people are going to find themselves in similar situations where they're seeing usage of their AI products that they didn't really expect, that they themselves might be not fully comfortable with. And then there's also this question of, if we don't do it, who's going to do it? And I think some of the leaders in the OpenAIs and whatnot in the space right now really kind of think about that a lot. I think that a lot of them are thinking, if we don't go develop this and we don't do it in the right way, then somebody's going to come around and do it in a not so good way. And so therefore, we better keep going down this path because we think we're better suited to go down this path versus other people who would if we don't. Is that compelling to you at all? Like, if somebody—if I were to make the argument, look, you guys have all these years of experience with these users. You already have users that are actually doing this on the app. To try to frame it the other way, I understand it's not what you set out to do. I understand it's not what you want to do, but the world needs you to do it. Because if you don't do it, it's going to be some run-of-the-mill porn proprietor who's going to come in and do it. And I think they probably care a lot less, and I don't want to speak for the whole industry. I don't know that much about it. But my impression is you care a lot more about your users than the sort of counterfactual hypothetical app developer who's going to come in and do the R or X-rated version of Replika. Do you find that at all compelling that there could be a sense of duty because you're so well positioned to do this sort of thing that you really should?

Nathan Labenz: (1:37:05) Replika is my baby, and we started with a particular mission. And no matter how much some journalists want to say we were taken in this direction or whatever, we weren't. We truly were always building—and to that, we can show the list of all product changes we've done. We made none of them towards this direction. We built a whole 3D store to change visual appearance for Replika. We built an environment for Replika so you can decorate it. We built tons of features on memory and memory UI, and building now our own advanced AI functionality, tons of mental wellness activities that we built with clinical psychologists from UC Berkeley and not only. Why would we do all that if we were going in the adult direction? Why have a team of 100 people working on so many complicated and complex features that have nothing to do with that? So no matter how much the press wants to portray us as some sort of company that was trying to monetize on or build a sex bot, whatever, we truly weren't. And so I don't want any misunderstandings, even if that means maybe eliminating a little bit of a small subset of users or missing out on some of the insane revenue that would have come had we actually wanted to take it in the adult direction. This is my baby. I built it for something different, and I want to try to get there. If we ever decide—even though I don't think that that's something that we're built for—but even if we ever decide to go in this route or whatever, it's just not going to be Replika. This company was built for something else, and I feel it's getting too much criticism from completely different, maybe not from the very right place from some of the journalists. But in general, we always did good by our users. I invite people to go and find truly testimonies of our users that were truly hurt by Replika. This was our main, main goal, to create a space where they will feel better, they will feel loved and wanted and heard. And mostly, we managed to do that. And so we're going to continue to try to continue on that path, at least for now.

Eugenia Kuyda: (1:39:29) Cool. Well, I think that's probably a good place to leave it for today. It's a fascinating situation that you are in, and you're definitely blazing a really unique trail. I mean, the whole AI world is kind of unfurling before us, and then you're down this one path a mile ahead of certainly anybody else that I'm aware of. So it's fascinating. I really appreciate the opportunity to get your perspective and appreciate you coming back for a part two to comment on the events of the last week. But yes, thank you very much for your time, and good luck continuing to build and figuring out all of these challenging dilemmas. I'm sure it's not going to get any easier.

Nathan Labenz: (1:40:17) Thanks so much for your great questions, and I hope I answered some of these questions in a way that shines some light on what we're doing and why we're doing certain things.

S2: (1:40:27) The Cognitive Revolution podcast is supported by Omneky. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms with the click of a button. Omneky combines generative AI and real-time advertising data to generate personalized experiences at scale.