Dr. Michael Levin on Embodied Minds and Cognitive Agents

Full Transcript

Transcript

Nathan Labenz: (0:00) Everything we do is around this notion of embodied minds and what it means to be a cognitive agent in this physical universe. We think about things like the collective intelligence of cells during embryonic development, during regeneration, and so on. We've had projects in cancer where we can detect and normalize cancer by controlling its bioelectrical connections between the cells and the large scale network. All of these terms—machine, human, robot, alive, emergent—I think what gets us in trouble is assumptions that these are binary categories. Think about the whole spectrum of diverse intelligence: cells, organs, tissues, chimeras, cyborgs, hybrids, and every other kind of combination. That's only going to continue. What are the things that matter to a bacterium? What are the things that matter to a dog? What are the things that can matter to a human? And could we have at some point one being that literally is in the right range, can care about all the living beings on Earth? I think that any improvements that we can make to enlarge our cognitive light cones, to improve our cognition, to have better embodiments—we can certainly do better than we do now.

Michael Levin: (1:07) Hello and welcome to the Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg.

Hello, and welcome back to the Cognitive Revolution. Today's episode is a special one. My guest is Mike Levin, Professor of Biology at Tufts University, and one of the most distinctively accomplished experimental scientists, and one of the most fascinating philosophers of intelligence that I've encountered in my life. We cover a ton of ground over the next hour and 15 minutes, and this is not an easy conversation to summarize. If you're like me and usually listen to podcasts at 2x speed, I might suggest slowing this one down just a bit, because unless you are already familiar with Professor Levin's work, there are simply so many strange results and deeply thought-provoking ideas in this conversation that I found my brain, at least, needed a little extra time to process it all. It might also help to watch some of the short videos that Levin and his team have released over the years, both on his YouTube channel and their academic group website.

After listening back to this conversation myself, a few major themes do stand out. First, AI defies all binaries. I say that all the time, but Levin's work takes that sentiment to the next level by showing that even the most familiar binary distinctions, the ones that we take most for granted—such as that between a living thing and a machine—are in fact rapidly collapsing from both directions as we simultaneously see remarkably sophisticated behavior from even very simple computer systems on the one hand, and at the same time on the other, groups like Professor Levin's devising striking ways to program biology itself.

Second, we have a lot of surprising latent capabilities left to discover. Just as GPT-4 can do far more now than we knew about at the beginning, so it is with biological systems, particularly when we place them way out of distribution, as Levin has done in his famous Xenobot and Anthrobot projects.

Third, and perhaps most importantly, we must always favor epistemic modesty and open-minded experimentation over our philosophical commitments. Many of the capabilities that Levin has discovered in biological systems would have been laughed off as impossible, if not crazy, right up until the moment that he demonstrated them. Reflecting broadly on how little we know about even the simplest systems with which we've coevolved should be enough to keep anyone humble, particularly in the presence of new systems that don't share our evolutionary history.

In Levin's view, the hot-button topic of emergence in AI systems—a subject that I plan to cover in depth with a dedicated episode soon—is inherently a subjective question and ultimately boils down to the element of surprise. I agree with this framing, and I was struck to hear that Levin, a world-class biohacker by any measure, who is obviously willing to conduct experiments that some might consider playing God, still has consciously avoided working on ideas that could make AI systems significantly more lifelike than they are today, at least until he has a better understanding of what that might imply for their subjective experience and our collective future.

As always, if you're enjoying the show, we'd appreciate it if you'd take a moment to share it with your friends. Considering how it brings such critical themes to the fore, I think this episode would be a great introduction to the Cognitive Revolution as a whole. And if I could be so bold, I think it is worth sharing widely. Now, with full credit to Professor Levin, his team, and the many marvelous intelligent systems that they study, I hope you enjoy this paradigm-shifting conversation with Professor of Biology, Mike Levin.

Nathan Labenz: (4:54) Professor Mike Levin, Professor of Biology at Tufts University, welcome to the Cognitive Revolution.

Michael Levin: (5:00) Thanks so much. Happy to be here.

Nathan Labenz: (5:02) I'm really excited about this conversation. You do some of the most fascinating work, some of the most mind-blowing work, really of anyone out there today. I think when people learn more about it—if they've never encountered your work before—when they learn about some of the projects that you have developed, they're going to just think, "Wow, how did I not already know about this?" It's profound stuff on its own terms, but I think it also will hopefully help us develop some new and different perspectives on the future of AI as well. So I'm really excited to get into it.

For starters, because I do suspect that we have a lot of AI enthusiasts and builders in the audience, but I'm not sure how familiar many will be with your work, do you want to just go through a few of your most famous projects? I think if folks know any from you, it would be the Xenobots and the more recent Anthrobots that you have created. So maybe just tee things up by telling us about these strange creatures that you have managed to create.

Michael Levin: (5:58) Sure. Well, my lab does a wide variety of interesting things. I work with some amazing people that span the range from philosophy to biology to computer science to physics and so on. Everything we do is fundamentally all around this notion of embodied minds and what it means to be a cognitive agent in this physical universe. We study mind in a variety of unusual substrates. And so we think about things like the collective intelligence of cells during embryonic development, during regeneration, and so on.

I'll just kind of run down some interesting things we've done in the past. One of the things that we specialize in is that we made the first molecular tools to read and write the electrical memories of non-neural cells. This collective intelligence of cells that builds organisms and so on has to store memories of what it's doing. And we now have ways of reading and writing that information.

We've done things like look at how the different cells of a flatworm know how many heads the flatworm is supposed to have. It turns out there's an electrical memory that stores that information, and you can rewrite it. You can physically rewrite it. And then what you get is a worm that, when you cut it into pieces, the pieces make two-headed worms forever. It's permanent. So that memory, like any good memory, is rewritable, but it's stable. And so you get these two-headed worms, and of course, the genetics are untouched. So this question of where is the number of heads stored is really interesting because it isn't in the hardware. It's software-rewritable.

We've made tadpoles with eyes on their tail, and those animals can see out of those eyes. And so this shows the remarkable plasticity of life because you can produce this animal with a completely different architecture of sensors and processor in the brain, and it will adapt to a new connectivity of that system.

We've had projects in cancer where we can detect and normalize cancer by controlling its bioelectrical connections between the cells and the large-scale network that can remember how to build large complicated organs as opposed to being an amoeba-like cancer cell.

As you said, we've created Xenobots and Anthrobots, which is looking at the plasticity of cells that normally will build very specific things like frog embryos or human bodies. And we've shown that if you put them in new environments and give them an opportunity to sort of reboot their multicellularity, then they can do that in new ways and have new and interesting capabilities that we didn't know about before.

So all of these concepts keep coming up again and again, this idea of the software-hardware distinction in biology. I know a lot of people are very suspicious about computer analogies in biology. We can sort of dig into which part of that is true and which part of that is not. But yeah, those are some of the things we do.

Nathan Labenz: (8:55) These are really incredible, pretty stunning results. I mean, some of the short videos and some of the images of these little living things or quasi-living things are really just—again, how did I not know about this previously? I don't understand how it works. I wonder if you could give me a little bit more sense of how you're doing this control.

So, again, you start with a worm that has this remarkable regenerative capability where you can cut the worm in half and it'll regrow. So that's how the worm starts. Then you do the chop in half, but you apply some change. And instead of growing the missing part, it will grow a second head or a second tail, and you can control that. And again, the tadpoles with the eyes on the tail. Can you give us a little bit more intuition around what are you doing there? I think I've heard you use the term "executing stored procedures," but I'm still kind of at a loss as to the details of the interventions that you're actually making and would love to understand that a little bit better.

Michael Levin: (10:04) Sure. Yeah, there's a variety of techniques, but let's just talk about the bioelectric memory rewriting. So I'll give you a simple story about the workflow that we're doing.

You've got a flatworm, and it has a head and a tail. And these flatworms are actually an amazing model because not only are they incredibly regenerative, but they're also immortal, and they have no obvious lifespan limit. The asexual ones that we work with don't seem to age at all. They're highly cancer-resistant. They can learn. They can form memories. In fact, you can do the old McConnell experiments, which we have done where you train them, cut off their heads, they regrow a new brain, and then they still remember the original information. So this incredible ability to move information around from tissue to tissue in the body and imprint it on the new brain. So you can do all these cool things.

But if you cut them in half—so imagine you've got a worm like this. You've got a head here, you've got a tail here. You cut it in half. Something amazing happens, which is that the cells on this side of the cut—so the head is over here—the cells on this side of the cut end up growing a tail. The cells on this side of the cut end up growing a head. And so now you've got two normal worms. But they were right next to each other, neighbors to each other. Why did they have completely different anatomical outcomes? And how is it that a fragment—you can cut the worm into fragments—how is it that a fragment knows exactly how many heads it's supposed to have?

One of the first things we did was to realize that the algorithm of knowing "should you be a head or a tail?" cannot be local, because these cells were at the same location. So you know right away that from knowing where you were, you cannot tell whether you should be a head or a tail. You have to talk to the rest of the tissue. And so for many reasons, my hypothesis was that that conversation with the rest of the tissues was electrical in nature.

So the very first thing we did was simply prevent it. We said, "What happens if you're cut off from the electrical network and you really can't find out?" So in that case, what you would do is there are drugs, specific pharmacological compounds that you can put on these worms in the water that they're regenerating in that control how well the cells can talk to each other electrically. That's one of the things we started with.

Then we applied something called voltage-sensitive fluorescent dyes. So these are just molecules that are developed by other people—chemists that develop these things. You throw them onto the sample and they will fluoresce. So you use a microscope to detect the fluorescence, and they fluoresce differently depending on the voltage of the cell that they happen to be sitting in. So you get right away—you get this map across the entire tissue. I'm making it sound very easy. It's actually an incredibly time-consuming and difficult process. But eventually, you get a map of where all the voltages in this tissue are.

And what we noticed is that we could interpret this map to say where the head and the tail was going to be in the future. In other words, it's really a kind of pattern memory that tells you where it's going to be. And so once you've seen that, it becomes pretty obvious to then say, "Well, what if I change that pattern? And what if now I can see that it says one head. Well, I would like it to say two heads. What will happen?"

And so now you have an electric circuit, and what you can do is you can make—because you can make a computational model of that electric circuit in the tissue, and so we do a lot of computational modeling. And you can ask that model, "What do I need to do to change the pattern so that it now says two heads instead of one head?" And so by playing around with that model, you can say, "Ah, I need to close some chloride channels over here, or I need to open some potassium channels or something."

Because in all of these things, what we're doing is we're not applying any electric fields. There are no magnets, no waves, no radiation. There's no electromagnetics, none of that. What we're doing is hacking the natural interface that these cells use to control each other's behavior. That interface is a bunch of electrical, electrogenic proteins known as ion channels, which sit on the surface of the cell. They produce a voltage, and the next cell can feel that voltage, and they communicate to each other. It's very much like what happens in the brain, actually.

So what we are doing is using either drugs or optogenetics—not in the worm, but in other cases we've used optogenetics. So that's light-based, opening these channels on and off. And there's some other molecular biology kind of tricks, but the idea is to open and close the existing channels on these cell surfaces. You can think about it as an interface. It's like a keyboard or any other interface that you would have to the programmable layers underneath your machine. That's what this electrical system is.

So guided by—originally just designing these cocktails in our heads, but eventually with a computational model—we can pick drugs that are going to open and close these channels in the right way to give you the pattern that you want. And then you soak your fragment of the planarian in that drug. Typically you do that between a minimum of 3 hours, in some cases 24 or 48 hours. And then you leave it alone. You wash it out, you put it in regular water, and you see what it does. That's a typical workflow. There's some others, but that's typical.

Nathan Labenz: (15:00) Hey, we'll continue our interview in a moment after a word from our sponsors.

It's striking how similar that is in some ways to some of the mechanistic interpretability techniques that people are developing to try to understand how AI systems work. It really sounds a lot like activation patching, which is where—again, it's often done in toy models, so there's a notable similarity. People will take the model and, of course, with an AI, it starts digital, right? So you have a lot easier path to looking inside it than you do in a biological system. But that doesn't mean it's any easier to know what's going on inside because it's still just a lot of numbers flying around, and what does it all mean is not easy.

But doing the simulations, it sounds like you're almost kind of running a lot of forward passes, so to speak, in your simulation. You're kind of saying with this electrical pattern, "What do we expect to happen?" And then if we perturb that pattern, what would happen differently? Is that the sort of counterfactual modeling you're doing?

Michael Levin: (16:02) We have several different levels of models. So some models are exactly what you just described. So it's the model only feeds forward, and all we're able to do is run multiple counterfactual scenarios until we see what we get. In some cases, we can actually run it backwards, and we can say, "Actually, if I want this result, what would I have had to do?" And of course, now we're working to integrate modern AI methods so that you can actually train and have an AI instance that will actually make guesses about what to do. That's the next step.

In all of these cases—so this is very much an interpretability issue because we don't know what the native representation encoding is. So it is not obvious how the different voltage gradients are mapping onto whatever happens next. It's very parallel to neural decoding in the brain where you read brain states from a patient or from an animal model, and then eventually you try to guess, "Well, what was he thinking about when—" You're trying to infer semantic states from the physiology, from the readout of the network. That's what we're doing, and it functions on many levels.

And so you can measure individual cells and get metrics of, "Is that a stem cell? Is it a cancer cell? Is it a mature, differentiated cell?" You can do that. You can take measurements of whole tissues, and you can say, "You going to be an eye, or are you going to be a leg? What's the shape of the face? Where do the eyes go?" And then you can have very high-level information.

We have this project in the frog that—frogs, unlike salamanders, they don't regenerate their legs. So if they lose a leg, what we're able to do is go in with a bioelectric drug that triggers leg regeneration. But what's cool—there's two things that are cool about it, and of course, now we're sort of trying to move that to mice and eventually, hopefully humans someday.

The idea—what's interesting is that, first of all, the intervention in the latest experiments, the intervention is 24 hours. The leg then grows for 18 months. So it isn't a micromanagement kind of thing. We're not sitting there telling every cell what to do, telling all the genes which comes on, which comes off. It's a very early communication to the leg that says, "Take this path in morphospace towards a nice leg, not this path that leads to scarring." And that's the end.

And then the other thing is that the exact same intervention that regenerates legs in adult frogs causes regeneration of tails in tadpoles. What that means is that the information content is not really in the intervention. We don't give it all the information needed to make a complex leg or a complex tail. We say, in that particular application, we say, "Build whatever normally goes here."

So you're offloading a huge amount of information—the task, onto the system itself. All of this works because the system knows what to do, and it is our job to convince it that it should do one thing versus another. So it's very much with all these different layers—we can look at the cell layer, we can look at the tissue, the organ, and then the whole. In fact, in our latest paper, data that came out just yesterday, we showed that even groups of embryos have their own decision-making, collective decision-making that individuals can't do. And so, yeah, it's very much an interpretability question to understand how does the system understand its own states, and then how do we as scientists understand them as well?

Nathan Labenz: (19:24) There's so many connections just listening to your comments there. When you talk about running things backward, that obviously reminds me of backpropagation, but in a very similar way of answering the question, "What would have had to be different in the upstream signals in order to get the desired downstream outcome?" I mean, that's really the kind of tweaking that's going on in the AI models and the backpropagation process.

Also, mode switching is interesting. When people train the current chatbots to refuse your bad requests—for whatever the definition of bad is that they want to exclude from the model behavior—there seems to be this kind of early fork in the road that could be characterized in different ways, but "mode switching" is one label that has really stuck with me, where if you can get the AI to say, "Oh, sure, happy to help with that," then it will continue to do the bad thing against its training. Whereas if it starts with, "I'm sorry, as a large language model trained by OpenAI, I can't do that," then it's never going to do it.

It's striking that there's something similar at the level of limb regeneration where basically, if I'm understanding you correctly, you kind of have this early fork in which mode are we going to go into, and that intervention lasts for basically two orders of magnitude. You said like 1 day to like 100 days essentially that the effect persists. Did I catch that right?

Michael Levin: (20:59) Yeah. Yeah. It's 18 months. 18 months. So, yeah, I really think that there are a lot of similarities here in terms of—if you think of the classic ANN structure where you have the different layers that are progressive abstractions of the input that have come in. So I think the higher you are on that as somebody who works in regenerative medicine or bioengineering, I think that you want to be as high as possible on that level for control because I don't want to have to tell you which genes to turn on. I don't even want to have to tell you which types of tissues go where. I just want to say, "You already know what goes here. Just rebuild it." That's it. I want to have the minimal, the simplest trigger.

And the systems decide that—the decisions are made very early on what it's going to do. And then after that, there's a cascade. It's exactly like you said. Once you're going down a particular road, it becomes much harder to make changes. So you want to do it as high up in the decision hierarchy as you can.

Nathan Labenz: (22:00) Yeah. I mean, that's another just striking similarity—the existence of surprising capabilities. GPT-4, this has been remarked on a ton, right? It was trained—training was complete 18 months ago. It was released like 10 months ago. We're still seeing new state-of-the-art results set with people just prompting it in ever more sophisticated ways and revealing kind of capabilities that nobody quite knew existed.

Here, it's striking to me that, obviously, you haven't demonstrated this in humans yet, but you're working your way up the sort of complexity-of-organisms ladder. I guess this is maybe an ignorant question, but like, why doesn't it happen? Why do we not regenerate our limbs? If we have this capability, why is it seemingly never expressed? Or is it sometimes expressed? Are there examples of people who have done this? I've never heard of anyone regenerating a limb. It's crazy to think that that capability is latent. Nathan Labenz: (23:00) Children, human children, regenerate fingertips. Up until a certain age, somewhere between 7 and 11 years old, kids will regenerate their fingertips. We don't know why humans don't regenerate their limbs. I can tell you a story that might make a little bit of sense, but it's just a story. We don't know for sure.

So here's the story. Imagine that you're a mammalian ancestor. You're this ancient mouse-like creature running around the forest. Somebody bites your leg off. Now here's the problem: you're going to try to put weight on it because you're a tetrapod. You're going to grind that wound into the forest floor. It's going to get infected. You might even bleed out. Unlike a salamander, which has the ability to float buoyantly in peace and quiet for months, you really don't have the luxury of hanging out and regenerating. Your best strategy is to seal the wound, scar, have some inflammation, and hopefully live on to tell the story.

Now there is one example of a mammal that does amazing regeneration, and it's an appendage that you don't put weight on: deer antlers. In deer, they grow these massive structures—bone, vasculature, innervation, velvet. They grow a centimeter and a half per day of these things when they're growing out. I mean, it's crazy. And is it a coincidence that that's the one thing you don't put weight on? I don't know.

I do want to say something about your point about unexpected capabilities. I think this is really critical. I don't know if you've seen this, but a few weeks ago we put out a preprint on this—it's a purely computational study. What I wanted was—and this was done with my student Adam Goldstein—I wanted to really hit this issue of unexpected capabilities. Working in diverse intelligence research, I think it's hugely important and completely, basically, unknown in the AI community, and I think it's a real problem. A lot of the answers to things that people have been debating in AI really have their origins in diverse intelligence research.

One of the fundamental aspects there is that you can find intelligence—problem solving capacities—in very minimal, unconventional systems. Really, the idea that we do not have a good intuition for what to expect. When we build systems, we don't know. Never mind emergent complexity. That's easy. Fractals, Game of Life, cellular automata—complexity is easy. But what also tends to happen is there's emergent agency, the ability to pursue goals and solve problems. And we are terrible at noticing these things when they're in unfamiliar substrates.

So what we did in this paper was I wanted something that was extremely simple and transparent. The thing about biology is that in biology, there's always more mechanism to be discovered. No matter what you show, somebody will say, "Well, there really is a mechanism for that. You just didn't find it yet." So we wanted something super simple, and what we chose were sorting algorithms. These things that computer science students have been studying for many decades—bubble sort, selection sort, that kind of stuff—completely deterministic. Everything is right there. It's completely open. Six lines of code. There's really nowhere to hide. It's all there.

What we were able to show is that if you treat them, if you're a little bit humble about what these things can do and you ask questions about what they can do rather than making assumptions that they only do what the algorithm tells them to do, you actually find some really important capabilities that are nowhere in the algorithm. They're sort of implicit. There's the explicit algorithm that sorts lists of numbers, and that's there. You can't get away from that. They will, in fact, sort lists of numbers. But it turns out that they have some really interesting properties and some novel capabilities that we did not know about.

I think that if that's the case for these really minimal, dumb sorting algorithms, then something as unique and novel as these large deep networks and all the other stuff that is made in AI—I don't think we've even scratched the surface of what's really going on there. The capacity for surprise in even small systems is, I think, massive.

S2: (27:20) So I have to ask, what can the sort—I haven't seen this preprint yet—what can the sort algorithms do that is not obvious?

Nathan Labenz: (27:28) I'll give you two examples. Just to introduce this story: the typical sorting algorithm is you sort of have this central godlike observer who sees the whole string and, under some algorithm, is moving the numbers around. We made two changes to be able to study this.

One is that we said, to make it a little more biological, instead of having a central algorithm, we're going to do it bottom-up in a distributed way. So you've got an array of numbers. Every array element, we're going to call it a cell. That cell has some numerical value from 0 to 100 assigned to it. They start out randomly mixed up. And what we're going to do is every cell is going to follow the algorithm. So for example, every cell doesn't see the whole string, but it sees the neighbors. The 5 wants a 4 on its left and a 6 on its right, and every cell wants that. And we're just going to let every cell be its own agent.

So the first thing you find out is that if you do that, it still works. They actually sort themselves quite well. The next thing we did is we said, okay, now we're going to let go of the assumption of reliable hardware. In other words, if the algorithm says swap the numbers, well, they might be broken. One of the cells might not swap. In the standard algorithm, you never check whether in fact the numbers got swapped the way you wanted them to, nor do you check how you're doing. You don't do any of that. You assume the hardware is reliable, and that's it. We kept all that. We did not introduce any new code for any of the things that I'm about to tell you they do. We didn't put in any new code. So the code is exactly the standard stuff that everybody studies, just being run individually on every cell.

So the first thing that happens is that if you introduce defects in the string—if you introduce broken cells—it still does a really good job sorting. In fact, it will sort other numbers around the broken numbers if it can't move them. Again, there's no code in there to say, "Hey, was this one broken? Did it move?" No code about that.

One of the interesting things about intelligence and trying to estimate intelligence is this—and this was William James who had this example. He said, think about the spectrum between two magnets trying to get together and Romeo and Juliet trying to get together. Imagine you put a piece of wood between the two magnets. The two magnets are just going to stand there pressed up against the wood. They're never going to go around because in order to go around, you have to temporarily get further from your goal. Let's call that capability delayed gratification. This idea that I'm going to reduce the thing that I'm optimizing—I'm actually going to reduce that in order to acquire gains later on. I mean, it's not super high intelligence, but it's an ingredient. Being able to do that is an ingredient. Otherwise, you're just a very simple gradient follower. You're not going to get too far. Unlike Romeo and Juliet, who have all kinds of tools, cognitive tools, to get around social and physical barriers—planning and all this stuff. So, delayed gratification.

It turns out that if you actually track as these algorithms are sorting the numbers, when you introduce these broken cells—which, if you visualize it, one of the things we did was visualize the sorting process as a journey in "sort space." They all have to get to one point where everything is sorted, and they start out at different points. It's like this path, and they all take these different paths. A broken cell is basically a barrier in that path. You're walking along, then you want to move the cell and you can't. It just doesn't move. How are you going to get around this barrier?

It turns out that these algorithms—and some more than others, we looked at I think four different ones—some of them have the capacity for delayed gratification. What they do is they'll go move some other numbers around, and in fact, the sortedness drops for a while. The string gets less sorted for a while, and then they catch the gains later. It becomes better later. Now this is already quite amazing because there is nothing in that algorithm that explicitly—I mean, if you just look at the algorithm, there's nothing in there that explicitly says you have the capacity for delayed gratification. And they do this more when there are more barriers. In other words, they don't just randomly back up and wander around. No, they're extremely linear until it comes time to deal with a barrier, and then they sort of dip down and come back. So that's one kind of capability that we found—that they're actually able to move around barriers like that without any explicit code for it.

The other amazing thing is this: imagine that once you've put the algorithm in the individual cells, you can do a really cool chimeric experiment, meaning that you could have cells that are running different algorithms. So you can mix. Some cells are running selection sort, some of them are running bubble sort, for example. The thing is that none of the algorithms have any code to know which one they are. They don't have any data about what they are, nor do they have any ability to look at my neighbor and see what he's doing. You're just following your algorithm. You have no idea what it is. You're just following your algorithm.

Now imagine at the very beginning, we have 100 cells and we mix it randomly. So every cell has now two properties. It has the number that it's trying to sort, and it has which algorithm it's following. Okay? So there's two types of cells. They're randomly distributed. And all of this, by the way, has developmental biology consequences because the ability to sort out tissues is—animals do this. Embryonic frogs, if we make what we call a "Picasso frog"—we start with a tadpole with all the organs in the wrong place—everything will sort out, and you'll get a very nice frog out the other end. The eye will get back to where it needs to be. The jaws will come. They know how to sort themselves out. And we make chimeric animals too. We make what we call "frogolotls." It's got a bunch of frog cells and a bunch of axolotl cells, and you can actually ask: this different hardware that normally makes different things, what's it going to make?

So imagine we have this chimeric string, and then we're going to ask a simple question: what is the degree of clustering at any point in time? In other words, what's the probability that when you look to the cell next to you, it's the same type as you are? And Adam actually invented this term, which I like a lot, called "algotype." So there's genotype, which is the algorithm that you're following. There's the phenotype, which is what actually happens in biology. And then the algotype is what algorithm are you actually running. What's the probability as I look at my neighbor that he's the same algotype as I am?

Now, initially, at the beginning, it's 50% because we assigned algotypes to numbers randomly. At the very end, it's also 50% because at the very end, everybody's going to get sorted in order, and the assignment of algotypes to numbers was random. So of course it's going to be 50% again. Because you've now reshuffled everything in order, but there was no pattern. So there's still not going to be a pattern. 50%. So if you imagine this graph—50% here, 50% here—but during the sorting period, it actually goes like this. And what it means is that in the middle of the sorting period, they sort together. Common algotypes like to hang out together.

Now this is kind of a weird way to think about it, but to me, it's almost like a minimal model of the human condition. It's like, eventually, the physics of your world are going to pull you apart because the actual sorting algorithm is inexorable. You can't get away from it. So eventually, you're going to get yanked apart. But up until then, you have this life that allows you to do some cool things that are compatible—they're not directly forced by the laws of physics, but they're compatible with them. And you get to do this thing where you hang out with your buddies for a while until you get sort of yanked apart into what the physics is trying to do.

And so that's something that's completely non-obvious from the algorithm. You'll never know looking at the algorithm that that's what it was going to do. I have a gut feeling that this can be harnessed in various ways. The fact that it's also clustering means that you can get multiple work out of the same algorithm. It might be doing other things. And again, this idea that even something as simple as the sorting algorithm has this other property that we wouldn't have guessed until we checked. Until we actually ask, "What are you actually doing besides the thing we asked you to do? What else is baked in?" And that kind of emergent ability to maximize other outcomes besides the ones that are explicitly programmed in, I think, is probably all over the place, both in biology and in AI.

S2: (36:01) Hey, we'll continue our interview in a moment after a word from our sponsors.

There are so many different follow-up questions I want to ask about all that. At least kind of three big themes, though. One is—and I know you've developed this in different papers and different venues—but definitely very relevant, I think, in the AI context is this sense that the once supposedly very clear distinction between life and machine...

Nathan Labenz: (36:29) Maybe it was never really properly so clear in the first place.

S2: (36:32) Or at a minimum, now seems to be kind of blurring. I'd love to hear you talk about that, especially as it relates to this notion of emergence, which is one that I—I don't know how closely you're following the AI discourse—but intense debate right now about emergence. How to conceive of it, what should count, is it a mirage? Actually, a best paper award at NeurIPS, the recent major AI conference, was given to a paper arguing that emergence is a mirage, which I think is, honestly, put my card on the table, kind of missing the point on a few key levels.

But you could talk about this, I'm sure, for 10 hours. But especially with this kind of eye toward: people are like, "Oh, well, it's AI. It's just computer code. It'll never do anything unexpected. We can control it." And you are really, even with some pretty simple systems, calling that into question. I guess you could react to that however you want, but I'm very interested in how you project what you're learning with these small systems and these biological systems onto this notion of possible emergence in AI.

Nathan Labenz: (37:35) I'll be kind of philosophical for a moment, and then let's talk about definitions a little bit, and then I'll give some practical examples.

All of these terms—machine, human, robot, alive, emergent—what are all these terms for? What are they supposed to do for us? I take all of these terms as engineering protocol claims. I think they're all mirages in an important sense. All in an important sense. I think all of these terms are not objective truths. I think they are claims about the utility of a particular worldview from the point of view of some other agent, including the system itself, by the way. Cognitive systems have to have models of themselves. All of these things are different models of what's going on.

I think that emergence is basically a kind of expression of surprise in an observer. If you knew something was going to happen, you don't think it's emergent. If you were smart enough to predict that in advance from knowing the rules about the parts, then to you, it's not emergent. To somebody else who couldn't predict it, it's absolutely emergent. And so I don't think these are binary categories. I don't think there's an objective truth as to whether something is emergent or not. I think everything is from the point of view of some observer.

Now all of this business about machines and living things and so on. Look, you do not want an orthopedic surgeon who doesn't believe that your body is a simple machine. If you watch what orthopedic surgeons do—they've got hammers, and they've got chisels and they've got nails and screws—they absolutely treat your body as a machine. Okay? And you want them to. That is the right frame for what they're trying to do. Do you want a psychotherapist that thinks you're a machine? You do not.

And so there are different levels of this framing, and this is—I've got this thing called the TAME framework, T-A-M-E, stands for Technological Approach to Mind Everywhere, which begins by setting out a spectrum all the way from mechanical, clocks and things like that, all the way up to humans, all kinds of things in between, where what's different between them is not what they're made of, and not how they got here. It's not whether you were engineered or you were evolved, or if you're squishy and alive. I really, as a biologist, I have very little interest in whether something's alive. I'm not even sure that's a useful category, really. Now, the level of cognition, that's super interesting. But I don't think it necessarily tracks with being alive at all.

And what happens when you move across that spectrum is you change tools as far as how some other observer is going to interact with you. The way you interact with mechanical clocks is very different than the optimal way to interact with cybernetic devices like thermostats, versus learning agents like animals and some robotics, versus humans and so on. So are we machines? Yes. Are we amazing agential creatures that do things that simple machines don't do? Absolutely. Are aspects of our psychology robotic? Sure. Are there aspects that are not? Yep. All of it. It's—I think what gets us in trouble is assumptions that these are binary categories. I think there is no such thing. I think these binary categories don't exist.

And assumptions that all of this has an objective answer that we should just discover what it is, and then we're done. And even worse, some people try to make decisions on this from a philosophical armchair. In other words, they look at something and go, "Oh, well, that's just—I see what that is. That's just physics. That's not cognitive." Well, you have to do experiments. You can't just have feelings about where something is on that spectrum. You have to do experiments.

For example, we took gene regulatory network models, which are extremely simple, either Boolean or ordinary differential equation networks. And again, deterministic, very simple, nowhere to hide. We showed that they can do six different kinds of learning, including Pavlovian conditioning, just out of the box. And you wouldn't know that. If you have this commitment that something that looks like that has to be stupid, you wouldn't know that. And who knows what else they can do?

So I feel very strongly that these are all empirical questions. And if you can find a frame that you've taken from behavioral science or from cybernetics or something, and if you can usefully apply it to that system, then there it is. Then you found a good way to deal with it. So I don't believe that this machine-life distinction is valuable at all. I've never seen anything really useful come from trying to enforce a binary distinction like that.

S2: (42:19) One of the things I think is really striking about some of your findings is just how small-scale intelligence can be. Whether it's a clump of cells that constitutes a xenobot or an anthrobot, or it's even just virtual cells in a running sorting algorithm. But it starts to get confusing, I guess, when it's like, okay, how small can we get? Is there some—a lot of these systems, maybe not all, but a lot of them do seem to have multiple scales. So I'm kind of wondering: is that multiple scale question critical? Could you have something that just works at a single scale? What would that even look like?

And also you said this, you know, this framework for mind everywhere. Do you have any intuition for how this relates to—you kind of said, you know, everything is subjective. Does that also imply that there is some subjective experience on behalf of these systems? Do you think these little sorting cells have any sense of experience? I think these questions are also very under-theorized in AI. People are today just like, "Oh, well, it's an AI. Of course it doesn't have any experience or have any moral value." I'm not rushing to say that they do, but I'm also like, you seem very quick. Everybody seems so confident in that, and I'm kind of not so confident. Nathan Labenz: (43:41) Yeah. Though, well, the one thing I could say for sure is that we absolutely cannot be confident because we do not know. I mean, I hear people all the time making these pronouncements that it definitely is or it definitely isn't. No. We do not have a principled way of answering these questions for the biological world, and that means we are completely out to sea when we're faced with unconventional embodiments. And we know this. Science fiction has been at this for well over 100 years. This idea that when something shows up, lands on your front yard and sort of trundles out, and it's kind of shiny, but also it's giving you a poem about how happy it is to meet you, and it's kind of got wheels, and you're not quite sure where it came from, but also you're having a great conversation with it. Like, what are you going to use as a criterion for how you're going to treat it and so on. So all of the old categories that we used to have in terms of, well, did you come from a factory, or did you come from the process of random mutation and selection? Those kinds of things. These are all terrible categories, and they're not good ways of making that distinction.

Let's talk about the first question about how low down does it go. People, even in minimal active matter research where people can make very simple systems out of like three inorganic chemicals, then they can solve mazes and they can do interesting things. I think that all of this really becomes disturbing when you insist on a binary category, when you want to know, okay, is it cognitive or isn't it? Then you've got a real problem because look, each of us starts out life as a single unfertilized oocyte. It's a little blob of chemistry. We look at it and we say, okay, very clearly, this does not have the kinds of cognitive capacities that humans have. Nine months and maybe a couple years later, you've got something that obviously does. Developmental biology offers no sharp line during this process where a lightning bolt says, okay, now you've gone from physics to mind. Okay? There is nothing like that. The whole process is smooth and continuous. So there is no escape.

All of this, these continue. First of all, there's the continuum of development, then there's the continuum of evolution. If you think that you as a human have some very specific competencies, just start walking backwards. Which of your hominid ancestors and which of the mammals before you and before that, and eventually you get to something that's basically much less complex than a current single cell organism. Where did those things peter out? There is no good story about that. It is imperative that we understand that it's not are we cognitive or aren't we, it's how much and what kind. It cannot be binary. There's no way to support. Not only that, now with bioengineering and robotics, we can make combinations. So if you think machines are not whatever and humans are, fine. So right now, we have cyborgs walking around. We have people with microchips in their heads and various other things, insulin pumps and various other things. That's only going to continue. Now right now, you're dealing with, I mean, talk about non-neurotypical. Like, right now, you've got somebody that's, you know, 98% human, maybe 2% digital. Eventually, you're going to get to 50-50 and every other kind of combination. What are you going to do then? Are you going to do it by weight? Like, what percentage of you is original parts and what percentage of you is aftermarket? It would be ridiculous. Right? So there is absolutely no way to maintain this hard distinction.

An interesting question might be, is there anything in this universe that is zero on the cognitive scale? Okay? Because I think very simple things already are not zero. Now the question is, is there a zero? This is a hard question. I'll tell you what I think about it right now. Let's ask this. What would the most simple, the most basal, the most basic versions of agency look like? What do you need for that? Now it's obviously not going to, you know, people say, well, you think that the rocks have hopes and dreams like us. No. You have to scale down. The point isn't that it's going to have our level of cognition. What does the most minimal level look like? Right? The smallest possible.

Well, I think you need two things for that. You need some degree of goal directedness, and that in William James' definition is the ability to reach the same state by using different means. Okay? So same goal by different means. So you need some degree of goal directedness, and you need to be, not completely, your actions need to be not completely explainable by current local conditions. So if what you're going to do is completely determined by all the physical forces acting on you right now, then you're probably some sort of billiard ball, and that's it.

So those two things, I think even particles have those because least action laws in physics tell you that there's goal directedness baked into the bottom levels of the universe, and quantum indeterminacy gives you a really dumb version of not being predictable by local conditions. It's not great cognition because it's random. That's not really what we like, but it's something. And so here's what I would say. If there is any kind of a definition to life, I think life, we call life those things that are really good at scaling those up. So if you got a rock, it has no more capabilities than the parts that went into it. It didn't scale. It's sort of lateral, and it's bigger, but that's it. If you've got something that's alive, it's cranking up the agency, the indeterminacy, and the goal directedness across every scale of organization, and its cognitive light cone is increasing, you know, as a function of time.

So my friend and colleague, Chris Fields, who's a brilliant physicist among other things, I asked him, can you have a universe in which there is no least action in which there would be zero? He said that only in a universe where nothing ever happened. In a completely static universe where nothing ever happened, it would be zero. But as soon as you get interactions already, you've got the basics of least action. So I believe in this universe, I don't think there is a zero. I think everything has some capacity, but some things scale it up in an interesting way and some don't. And that means that for some things, the tools of behavioral science are going to be applicable, and for some things, they're not. Right? And that's the empirical, that's why this is not the same thing as ancient animism where you saw a spirit in every rock because it actually does not pay off to treat rocks with tools from behavioral science, but it does pay off to treat gene regulatory networks that way, and it absolutely pays off to treat various animals that way because that's how humans train dogs and horses knowing zero neuroscience. It's because there's this amazing interface that some animals expose that you can train them.

So yeah. And then I guess the last thing you talked about is about inner perspective. I try not to say too much yet about consciousness per se, but we can say something about inner perspective and which systems have it and which systems don't. Here's what I think. Imagine a landscape, you know, kind of a hilly up and down landscape, and you got a bowling ball on this landscape. In order to know what this bowling ball is going to do and to make it do whatever it is that you want it to do, all you have to do as an external observer is to pay attention to your view of the landscape. Your third person view of that landscape tells the whole story. You know where the hills and valleys are. That's all you have to know. You don't need to take into account anything else.

Now imagine a mouse on that landscape. When you get a mouse on that landscape, your view of that landscape is not really that relevant. What's really relevant is the mouse's view of that landscape because he might have been rewarded and punished at different times in different locations. He's seeing that landscape in a completely different way. There's a valence map sort of superimposed on it that he's got. He's got his own opinions about where he's going to go. And so for all these different systems, you can sort of ask to what degree, and again, it's not going to be a binary thing. It's never going to be a binary thing. But you can ask to what degree do I need to take the perspective of that system and ask what does it see and what does it think about what it's going to do? To the extent that you have to do that a lot, you're dealing with a high agency system that has an inner perspective. To the degree that you can get everything done by your own model of what's going on, then probably it doesn't. Right? And so that I think is a heuristic by which we can start to say, how much inner perspective can I expect from these things?

And I really, you know, as far as being sure that, I mean, look, let's be clear. I am not claiming that today's AI architectures are mimicking human brains. Okay? I don't think they are. I don't think they're mimicking human cognition, although they do have some interesting things in common that people don't realize. But I don't think they have to. The point about AI is not that you have to be a human for us to have to be kind to you. There are many living beings that are nowhere near humans. And the same thing for danger. Right? You don't need to be human level or above to be dangerous. Many really dumb things are extremely dangerous.

So the thing we have to understand is that you cannot pin your hopes for being able to distinguish, like, moral worth and things like that. You cannot pin that on what are you made of because there's nothing magical about protoplasm. You cannot pin it on being an evolved biological because there's nothing magical about the random meanderings of the evolutionary process, which just doesn't optimize for anything that we care about. It doesn't optimize for intelligence or meaning or anything like that. It optimizes for survival, for copy number. And so none of those things, none of those things are reliable guides. You can't say just by knowing what something is made of or how it got here. We have to have principled frameworks for making that decision, and we don't have them yet in biology really very much. So I guess the beginning of maybe a framework for starting to make sense of those, kind of touched on a little bit in those comments, but

S2: (53:34) just to kind of pull out a couple of definitions, I'll try to do it briefly. You tell me if I'm missing anything. I've heard you define intelligence as the ability to solve problems in some space. I think that kind of notion of in some space is interesting. Then you said also, there's the question of not are we or aren't we, but how much? You've used the term the cognitive light cone there, I think. In other words, what is your domain of concern and possible reach, you know, into, or how far can you project your influence into the world? And that is sort of a way of thinking about how much intelligence you might have. Does that extend all the way up to sort of nature as a whole? Are you thinking of, like, the Gaia hypothesis? That's a far out one in many people's minds, but

Nathan Labenz: (54:23) it seems to follow pretty naturally from what you're saying. Yeah. So the key to all of this, so first, let me define. The cognitive light cone is the size of the largest goals that you can pursue. Okay? So you can think about what are the things that matter to a bacterium? What are the things that matter to a dog? What are the things that can matter to a human? And could we have at some point one being that, like, literally is in the linear range, can care about all the living beings on earth? Like, we can't do it as humans, but maybe our next, you know, some next stage of evolution can. So it's this idea of what are the size of goals that you can actually maintain?

The important thing about this is that you can't decide that just by thinking about it. You have to do experiments because the way you find goals is by perturbing the system. You cannot infer intelligence or goal directedness by pure observations. You have to do perturbative experiments. So what that means is you put barriers between you, you make a hypothesis. What problem spaces is it operating in? What are the goals? These are all hypotheses, and multiple observers could have different hypotheses. What are the goals? What competencies does it have to reach those goals? And now you get to test that hypothesis by putting in different barriers and seeing what it does. Right? Does it have long term planning? Does it have learning? What can it do?

So with respect to the Gaia hypothesis, some people say absolutely not. Some people say, well, I like to think it does and therefore it does. Neither of those is any good because you can't do any of that without doing experiments. Now, can you do experiments? So for example, people have said to me, that sounds like you're going to say the weather has some degree of cognitive ability. Well, I'm not going to say that because we don't know yet. But one thing that this framework does for you is it doesn't close off the possibility of testing it to find out. Do I know that if you had the appropriate ways to change the temperature, the pressure of air, and whatever, you couldn't use some kind of a habituation or sensitization assay or an associative learning to show those things in, I don't know, in a hurricane or something? We don't know. I mean, that's an empirical question. Right? The thing about this framework is that it does not let you just sort of pick answers out of thin air. It requires you to do experiments.

So, Gaia, you know, can we show that ecosystems learn? Well, ecosystems have stress. Would an ecosystem be able to show different kinds of learning, anticipation? We don't know, but that can be tested. We're actually testing this right now. I have a student who's testing this in predator prey kinds of models to see if they can actually learn from experience. So we're like, we don't know. These are all testable questions. So is it possible that, you know, on the scale of, I mean, people ask, you know, on the scale of galaxies or something that we're part of a giant mind. I think it's absolutely possible. I don't think we know, and I don't think you can say yes or no just, you know, based on philosophical commitments. You have to do experiments.

S2: (57:20) That's maybe a great place to kind of segue to some questions that I have about maybe recommendations or sort of inspiration that you could give to people working to understand AI systems better. And I think, you know, you said there, you can't just sit and guess. Right? You have to do a perturbative, I believe is the word, experiment to see what is going on. In the AI world broadly, there's a lot of debate around the degree to which AI systems are generalizing beyond the distribution that they've seen in training. And

Nathan Labenz: (58:01) it strikes me that some of

S2: (58:02) the experiments that you've done, particularly, as you said, rearranging the face of a tadpole or whatever and watching as they sort of reconstruct their faces. One way to talk about what you're doing there is you're taking them way outside of the distribution of what presumably they had ever seen in evolutionary history. I mean, maybe. Right? But that seems pretty, like, you've found a dark corner that, you know, presumably has not been previously explored. So with AI, we're trying to kind of begin to wrap our heads around that. Do you have any sort of, you know, advice, like habits of mind, food for thought, you know, any kind of suggestions from what you've learned in your work that people could take inspiration from on the AI side?

Nathan Labenz: (58:45) Yeah. Well, there's two kinds of things. There are some very specific biological principles that I think would be interesting. And then there are kind of general ways to think about this in terms of the whole debate about AI. I mean, I want to say a couple things. One thing about embodiment. There's a lot of talk, people say, well, if it's just a software agent, if it's not embodied, if it's not integrated into the real world and, you know, kind of grappling with some sort of embodied physical existence, it doesn't, you know, it doesn't have real, it doesn't know what it's talking about. It's shuffling symbols. Right? This is the old, like, Dreyfus argument and all that kind of stuff.

So I want to say something interesting about, I hope, about embodiment. Embodiment is absolutely critical, but embodiment isn't what we think it is. People think embodiment is a physical robot that hangs out in three-dimensional space. And that's because most of our sense organs are pointed outwards, and they're optimized for tracking medium scale objects moving at medium speeds. Imagine if we had, imagine if we had a sense organ for our own body chemistry. Let's say inside our blood vessels, we had something like a tongue that could, like, feel, I don't know, 20 parameters of our physiology. I think, cognitively, we would be living in a 23-dimensional space, and we would immediately recognize our liver and our kidneys as intelligent beings that navigate that space. They have goals. They have certain competencies to reach those goals. Every day, we throw, you know, wacky stuff at them, and they know how to navigate that space. And they have certain competencies, and they live and they strive and they suffer in that space.

So I think biology tells us that there are many spaces. There are spaces of gene expression. There are anatomical morph spaces where the anatomical collective intelligence operates. There are metabolic spaces. There are, then, of course, there's the familiar three-dimensional space of behavior, and then there are linguistic spaces and so on. Embodiment can take place in any of those. Intelligence could take place in any of those. So the first thing I would say is to take very seriously this idea that, you know, is popular in science fiction or whatever, that this physical space that you're in is not privileged in some way, and everything else is virtual. There are many other spaces, and they are just as real. There are other beings that live in these other spaces. Many of them are in our bodies right now, and those spaces are no less real than this one. It's just that, you know, our left hemisphere, like, this is the kind of sense data that it mostly gets, and that's why we all feel like we live in this space and everything.

So that's the first thing. So that's what I'd say about embodiment. Something else I would say is about this issue of symbol binding and grounding. Right? This idea that, well, you know, it shuffles little letters and it shuffles words and so on, but they're not grounded to anything. It doesn't, you know, it doesn't know what they mean. If you have kids and you watch a little human develop, what you see very quickly is that they start out as kind of happy-go-lucky pattern matchers. They will just sort of repeat stuff they hear. They see what sticks, and they sort of talk about all kinds of stuff that they have actually no actual grounding on. And then eventually, they get to the point where you say, oh, wow, yeah, he really knows what he's talking about when he's saying that. This is a smooth process. It's not a categorical, yes, you are real or no, you're not.

In fact, I don't know about you, but I thought, at one point, I tried to sort of estimate what, like, what percentage of the things that I talk about are actually grounded in the sense that I've had actual experience with them. You know? We can talk about all kinds of stuff, you know, places we've never been and, you know, all kinds of things we've never seen. A ton of our cognitive fodder is not grounded to anything. It's just tied to other stuff. So it's important to be clear that a lot of these supposed distinctions are really spurious.

You know, this issue of confabulation. Right? I mean, humans confabulate constantly. There is, you know, there's a good theory that basically says that part of your language ability is basically just to tell stories about what your brain is already doing, right, after the fact. To concoct good stories that you can share them with others and increase cooperation. So there's some amazing, you know, some amazing data on, you know, on split brain patients and other cases where it's very clear that we're very comfortable confabulating.

There are these distinctions. Again, not saying that the current architectures capture what's essential about life because I don't think they do. A lot of these things are not, the biology isn't what people think it is. And I would encourage workers in AI to think about the whole spectrum of diverse intelligence, cells, organs, tissues, chimeras, cyborgs, hybrids, not just like the standard adult human that, you know, that we think about as a counterpoint.

This idea of ethics and the kind of the impact of highly competent AIs, and people say, this is really scary. You know, we're creating these intelligent entities and we're going to let them loose into the world and we don't know what they're going to do. We've been doing that for millions of years. It's called having children. We already do that. All of us, we create these guaranteed high intelligent agents. We do our best or not during the critical kind of training phases, and then we let them into the world. And sometimes they do, you know, miraculous things, and sometimes they do horrible things. And so I think a lot of the issues that we have around AI right now, I mean, obviously, there's some unique ones, but most of the issues around AI and what are we going to do to stay relevant and, you know, what happens to collaboration and to AI art and all this kind of stuff is really just reflections of existential issues that humans have been struggling with for a really long time. You know, these are all fears about what happens, what, you know, what do the next generations think of us as they mature and, you know, and fears about humans and what does it mean to stay human. Yeah. I mean, we can talk about that too, you know, this issue of what do we want to persist, you know, as far as people being worried that we're beginning to get taken over and so on. It's a whole thing we could talk about there.

S2: (1:05:05) This is extremely thought provoking on a lot of different levels. One quick follow-up on the perturbative experiments. It seems like there is a decent analogy I can already start to see between some of the things that you're doing and, you know, there is a lot of weird stuff that is kind of on the edges of modern AI research, but is, like, remarkably, it kind of works. Stuff that I think probably would get a lot more energy if the mainline stuff weren't working so well, then people would be kind of forced to go out and explore more of, you know, these kind of weird frontiers. But right now, the vein of just progress in the mainline research is so rich that, you know, there's not a lot of incentive in many cases to go outside of it. But you do see these things, which kind of remind me of some of your work where, you know, you can, for example, take a lot of weights in a lot of models and just drop a ton of the weights to zero and find that, oh, hey, it still kind of works. You know? And you can, you know, take two pretrained models and put a little connector between the two of them, freeze the pretrained ones, and just train the connector and get kind of novel behavior that, you know, neither model could come up with on its own.

I guess, you know, I'm riffing there. I think that kind of work feels really important. It also to me, it does feel a little bit, I mean, the whole AI enterprise to me, not like any individual modern day experiment, the whole AI enterprise does feel to me a little bit dangerous, to be totally honest, because unlike our children who I think we have a pretty good handle on kind of, you know, just how big their, you know, individual light cones may be, I think we may very well create something in the AI realm that just has, like, way bigger scope of action than anything we're used to, and then, you know, that could go badly for us. It sounded to me like you weren't too worried about that. Maybe you're just not that worried about it, but I wouldn't, based on everything we've talked about, I find it hard to imagine you would rule that out or see some reason that, like, shouldn't be an issue that people would worry about at all. Right?

Nathan Labenz: (1:07:09) No. It's definitely an issue, but I also think that that issue is here long before we make anything with a huge cognitive light cone. So there are many things that are extremely dangerous that do not have a large cognitive light cone. Some of that is because of how we relate or misrelate to them. So I can imagine AIs that are going to be that would be extremely deleterious for society, but not because they're so smart. It's because we're not that smart. Right? And I think that we, you know, one of the big things that I think is an existential risk scale thing for humanity is to develop a principled science of where do goals of novel systems come from and to interact with, ethically interact with systems that are radically different from us, minds that are radically different from us.

We make systems, you know, Internet of Things, social and financial structures, AI, obviously. We make all these different things. We do not have a science of where novel goals come from in systems, not just emergent complexity. That's bad enough. Emergent complexity is bad enough. But also what is to me clear is that what can emerge is emergent goals. So systems that have some degree of pursuing various goals and some competency to get those goals met. And we do not have a good science of knowing when that's going to happen and what those goals are going to be, what the competencies are going to be. And so I'm not saying that it isn't possible that we make a hugely, you know, super intelligent AI that's going to be a problem. Absolutely possible. But I think we need to realize that that problem can occur long before we get to that point.

Our physical infrastructure and also our kind of the general mental frameworks that people are using are extremely brittle. You know, most of the terminology that we rely on today in the legal system, in, you know, in interpersonal relationships, that stuff isn't going to last the next decade or two. Like, all those terms are going to crumble. And it's not because we're dealing with something yet that's super intelligent. It's because we haven't got the right framework for dealing with other minds that are different from our own. That's where a lot of effort has to go. Nathan Labenz: (1:09:27) Yeah, that question of emergent goals is definitely a very salient one in the AI space, and I think it probably deserves to be even more focal still. But some challenges that we have in AI today: One is figuring out what the goal should be. The pretraining goal is just predict the next text, then we've got reinforcement learning from human feedback, which is satisfy the human. But then we look at ourselves, and as you said, we're kind of brittle. What satisfies the human is not necessarily good for humanity or the ecosystem broadly or whatever. So lots of possible problems there. People start to think, well, maybe we could have multiple goals and we could kind of—this is sort of what humans seem to have, this kind of multidimensional value. Maybe we could have multiple goals that we could optimize the systems for. Yikes. Okay, a couple more and then I'll just let you go off on all of them.

Memory is a huge challenge right now for AI. We have, at least with the kind of systems that people are most familiar with, the context window. It can process that information densely at runtime, but there's not really another level of memory in today's frontier systems. So people are kind of rigging up all sorts of scaffolding and finding things that work and saving those to a database and recalling them later. They're not very integrated. You've got such incredibly remarkable findings in terms of memory and bioelectrical space in organisms. Really interested in that.

Robustness is another big challenge. These AIs are superhuman across passing all the graduate school exams, but then we can trick them. They're so gullible. It's like, well, what's up with that? There is something definitely remarkable about our robustness given our finite capability that the AIs are not that close to yet. Even a superhuman Go player—I don't know if you saw this result, but a relatively simple attack was able to beat the superhuman Go player. So I'll stop there. The big ones: goal design, memory, robustness. AI has a long way to go to catch up to biology, and I wonder if you have any suggested directions for the AI developers.

Michael Levin: (1:11:35) The thing about memory, I'll just give you a quick biological example: caterpillar to butterfly. So caterpillar, soft-bodied robot, lives in a two-dimensional world, eats leaves. Butterfly is a hard-bodied robot, flies around a 3D world, drinks nectar. In order to get from here to there, you have to build a completely new brain. So what it does is during the metamorphosis process, it basically dissolves most of the brain. Most of the cells die, most of the connections are broken, and then you have a new brain that's suitable for a completely different kind of creature. The memories that—and this has been shown—the memories that a caterpillar forms are still present in the butterfly. In fact, mapped onto its new kind of lifestyle. We don't have any, to my knowledge, any memory in the computer science world that can survive such refactoring. You just tear the whole thing up and reuse the pieces, and now you can still retain the original information. Biology does it quite differently.

Josh Bongard and I have a paper on polycomputing and this ability of biological systems and subsystems to reinterpret the same physical events from their own perspective as multiple different computations. I think that's actually pretty key. Also key is the fact that evolution doesn't make specific solutions to specific environments. It makes problem-solving machines. This is why—and I could tell you all kinds of crazy capabilities that living things have to deal with things that they've never seen before in evolution. And it's because evolution doesn't overtrain on prior experience. It produces multiscale agents that are able to solve problems in new ways.

Fundamentally, for the use of AI, we have to decide: do we want tools that we use for specific purposes, in which case pleasing the human is probably a great goal to shoot for? Or do we want true agents with the kind of open-ended intelligence that we have and the same moral worth that we see among animals and among humans? Those are very different things.

Honestly, I had started about six months ago writing a paper on specifically what parts of biology the current AI architectures are missing. Which I think would elevate the whole thing majorly because I think they're just missing some of the really exciting new biology. And I stopped, and I'm not going to write that paper. Not that somebody else isn't going to crack it, because they will. So this is not going to solve this issue. Because I think to whatever extent I'm right and that those things are actually what's critical about agency, they can easily be implemented in other media. There's nothing magic about either evolution or biological substrates. You can implement those kinds of things in other media. But if you do, to whatever extent I'm right, that those are the ingredients to an actual mind, that means that we are going to then generate, I don't know, trillions of new agents with moral worth. And I'm not interested in being responsible for that. But it's going to happen, and I don't think we need it. I think people are going to do it just because to see if it can be done. And I think it's the equivalent of having—if you do it correctly, which somebody will do it correctly—it's like the equivalent of having trillions of new children.

Now you might think that your job is to populate the cosmos with as many minds as possible. I mean, that's one view. But to me, if you're not able to vouch for the kind of upbringing and life they're going to have, then I think that's a mistake. So we can stick to the kind of AI that's actually really helpful. It's a tool. It's a thinking tool for humans. It doesn't have the kind of open-endedness and the kind of agency where you have to worry about it or you don't know what it's going to decide to do. I think it would be much better to stick to that kind.

And it's not because I think humans are—just to sort of end on this, because a lot of people are really worried that we're going to get supplanted. I think we will and I think we should. If you think forward, 10,000 years from now, what do you want to be living on this earth? Do you really want it to be the exact same humans that we have now that are subject to lower back pain and stupid infections and they have about 80 years of productive lifespan? Why is that something that we're attached to? I'm not interested in that at all. I think that any improvements that we can make to enlarge our cognitive light cones, to improve on our cognition, to have better embodiments—whether those are engineered or bioengineered, I don't care—we can certainly do better than we do now. If anybody has been to a hospital knows that this is not a great embodiment that we're in right now. It's amazing, but the future should be way better than this.

So I'm not talking about limiting AI because I think humanity should be frozen in the version that we are now. I don't think there's any reason to lock it down that way. But I also don't think that there's any particular reason, and I think there are good reasons not to try really hard to make new agents that have moral worth. And I absolutely think we could in other substrates, including digital. So yeah, I'm not really advertising those methods that I think actually could be done. But there's plenty of other stuff that I think is perfectly reasonable and could help create better AI tools for us.

Nathan Labenz: (1:17:18) I've been starting to think about that same question from a different angle. What are the pieces that are missing from the current AI systems, and what would it maybe look like to solve them, and what would the consequences of that be? And I do think you raise a good sanity check on the wisdom of the direction of that research.

When you look at the AIs today, one thing that is kind of striking is we've got these language models. Right? And they just predict the next token, quote unquote. They're very much just happy to be a chatbot for you, but they—you said multiple times, there's no binaries. I always say AI defies all binaries we try to put on it. This one also seems to be kind of defied to a limited extent at least where you cast the language model as an agent. You basically say, you have a goal, and now I'm going to put you in a loop with some tool in the outside world. And they're not very effective agents, but they start to look something like proto-agents. They certainly try to pursue the goal and, yeah, again, usually fail. But how do you interpret that, or how do you kind of make sense of that sort of non-agent versus agent mode of the current systems?

Michael Levin: (1:18:28) A lot of problems in thinking about this arise from, A, comparing it to humans and, B, comparing it to very high-performance humans. So I sometimes hear really brilliant scientists or AI workers say, ah, this thing can't even do—how many people can do what you're talking about? Like, barely any of them can. Your friends, because it's a very selective elite population of human minds, but most humans can't do that. So there's that. And then again, look at the diversity of life on Earth. I mean, we have all kinds of other creatures that are not humans that are minimal or sort of meso-goal-seeking agents. You've got insects. You've got birds and fish, and you've got all these things. For AI to be a real agent, it doesn't have to be like a human. You might be developing some kind of weird insect-like thing or something like that.

And let's remember, we are not very nice to all kinds of creatures that are for sure agential creatures. So adult mammals in our food chain, whom we eat in factory farming—factory farming is an incredible moral lapse, and it's not because we don't understand that pigs are intelligent and that cows are intelligent and that they deserve certain moral protections and so on. It's not because we don't know. It's that we're just not very good as a species. We're not very good at extending our cone of compassion to things that are in any way different from us. Right? The stupidest, slightest difference is enough to trigger this self versus other thing, and then, you know what happens then.

It is absolutely erroneous to assume that these systems do not have some degree of goal-directedness. I think that it's possible that we haven't figured out how to measure it properly yet, or it's possible that we have and it's just low, like fish or insects or something like that. But again, I'm not claiming that these architectures share the important things that—share a lot of the important things with life. One thing that's super confusing nowadays is that in the past, you were guaranteed that anything that talks also shares your agential key properties. That's no longer true. Now we have these things that talk, but they actually—I don't think they share them. But that doesn't mean the fact that they don't share them with elite adult modern humans doesn't mean that they don't have any. They may have some, and we are not good at recognizing it.

And that's why I'm saying that. I'm not saying it because I'm fooled by the chatbot action. I'm not saying it because I just sort of assume that robots and AIs are going to be that way. I'm saying we do not have a great way to detect it on the biology end. And we certainly do not have a great way to detect it on an exobiology end. So like, if we had alien life and so on, we'd be really up a creek, actually. We have no clue. And so I don't see any way to be certain about anything with these complex creations.

Nathan Labenz: (1:21:19) Well, that is, I think, an appropriately sober note to end on. I have really enjoyed reading your work in preparation for this. It is fascinating, and this conversation has been super thought-provoking. I'm sure people are going to take a lot from it, and I hope we can do it again at some point in the not-too-distant future. But for now, Professor Mike Levin, professor of biology at Tufts University, thank you for being part of the Cognitive Revolution.

Michael Levin: (1:21:53) Yeah, thank you so much. Yeah, it was great to talk to you. I'm happy to come back. You should also try to get Richard Watson at Southampton if you know his work. He's got a lot of really interesting things to say about this. So that's something else you might want.

Nathan Labenz: (1:22:08) Cool. That's a great tip.

Michael Levin: (1:22:09) It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.