SPEAKER_00: I want to just make this very clear that my perspective on AI and automation is that there's an upward spiral when you have more energy, you have more intelligence, you have more capability. These drive all the metrics of human flourishing up. They don't take. So when we think about the answer, when will you get lights out manufacturing? I think the answer is never, because people will always find new things to do with the tools that we've built, even very powerful tools that can think and maybe are even self-aware. These will only increase the number of jobs, the increased wages, but there'll be different kinds of jobs. There'll be the sorts of things that maybe we can't even imagine now.

SPEAKER_02: This Week in Startups is brought to you by InTouchCX. Looking for ways to make your startup more efficient? InTouchCX has a groundbreaking suite of AI-powered tools for end-to-end optimization to give your business the edge it needs to thrive. Get started with your free consultation at InTouchCX.com slash twist. Fount. Do you want access to the performance protocols that pro athletes and special ops use? With Fount, an elite military operator supercharges your focus, sleep, recovery, and longevity, all powered by your unique data. Want a true edge in work and life? Go to Fount.bio slash twist for $500 off. And .techdomains has a new program called Startups.tech, where you can get your company featured on This Week in Startups. Go to Startups.tech slash Jason to find out how.

David Friedberg: Hey, everybody. Welcome to This Week in Startups.

SPEAKER_08: We've been focused a ton on AI this past year. Of course, we talked about it over the last decade on the show. But things have heated up with language models and, you know, the very forgotten category of startups is, of course, robotics. We see once in a while on the internet a trending video. And the trending video tends to be one of Boston Dynamics robots doing a backflip. Uh, or we see maybe, um, some surgery being done, uh, on a grape. You've seen all these viral videos.

SPEAKER_12: But, uh, the idea of humans leaving the factory floor and going and doing things in the real

SPEAKER_08: world, but we don't see many startups doing that. We have one in our portfolio. Cafe-X making, uh, coffees at SFO's airport right now. Uh, and of course, uh, our friends over at Tesla are making Optimus, uh, and there's another startup called Figure. They're working on a humanoid robot. Sanctuary AI is today's guest. They're another startup working on this problem. And they're specifically, uh, focused on building robots with general intelligence. What does this mean? Well, it's not verticalized, uh, and they're not just making a cup of coffee. But what if these robots could solve problems in the same way we do as biological creatures, as human beings? And, uh, if that works, well, that's gonna have the economic impact of humanity and it's gonna go well beyond just the steam engine. Uh, and we have, uh, the founder or I should say the co-founder and CEO of Sanctuary AI on the program. His name is Jordy Rose. Welcome to the program, Jordy. Thanks for having me. Uh, great name. Uh, I, I, I am reminded of the Mark Knopfler lyric from, uh, the amazing song Selling to Philadelphia, where he says, I am Jeremiah Dixon. I am a Jordy boy. Uh, do you understand the, the reference Jordy boy?

SPEAKER_19: I do. I do. Yeah.

SPEAKER_08: Yeah. Uh, uh, so let's talk a little bit, uh, about the company and, uh, I know you were founded in 2018, so you've been working on this for a while. We've raised close to a hundred million bucks. Um, where are you at with building this humanoid robot?

SPEAKER_22: And I would love to see the latest.

SPEAKER_23: I should start by saying that our approach to the problem and the reasons for us working

SPEAKER_00: on it are slightly different than most people who work in robotics. Um, for us, the motivation for doing it was a belief that human-like intelligence and more generally the intelligence of animals, which is kind of our model for what intelligence means, um, is very intimately tied to our presence in the world. Um, we have a body, we are a thing, uh, we experience the world through our senses, we develop understanding of it through interacting with it, and then, um, we act on it to achieve our goals. Um, all of those things are very difficult to do if you're not actually physically present in the world. So this, the starting point of this, which actually goes back more than a decade now through two different companies, um, was to explore this idea that intelligence would, by which I mean, general intelligence emerges as a consequence of having to deal with the real world, the real world. You never see the same thing twice. You have to be able to generalize from your, your previous experiences to new experiences. You have to be able to understand the common sense, uh, ways that the world is, uh, so we've been building software, which you could call general intelligence or, or AI, but it's also control systems for robots. And we've always viewed the problem of artificial general intelligence through that lens is that for us, a true general intelligence can be thought of as a control system for a robot that converts what it sees, hears, touches, feels about the world into actions that are intended to reach goals. So for us, the robot is some of a means to an end, um, and because of that, um, that thesis, we focused almost exclusively on a, on a very hard, but very fundamental problem, which is the, the building and use of hands. So much of the humanoid robotics videos are performative. They show robots doing things, but they're not valuable things. And for us, I think that the key, the key value of doing this is to understand how an entity and robot or a person understands the world well enough to be able to manipulate it with its five fingered opposable thumb hands. You know, I believe that the, the hand was played a big part in our technological evolution and also in the development of language, which are related things. Um, so that's, that's how, how did that happen?

SPEAKER_27: How did the hand, uh, play a role in language?

SPEAKER_28: I'm curious, uh, was it writing, uh, or the ability to hold a pen speaking?

SPEAKER_29: Ah, so how does a hand help you speak? Yeah.

SPEAKER_31: So there's, uh, although this is speculative, there is a lot of evidence that the, um, the

SPEAKER_00: earliest spoken language was very strongly connected to the things that our hands do, like point, touch, grasp, and so on. And the, some of the evidence for that is in neuroscience, where the part of your brain that controls your, the grasping and the use of the hands overlaps with your language center. These things are not disconnected. And when you actually try to build a system that, that touches, feels the world and can interact with it in the way we do, you, you see this explicitly is that the, the cognition, you think of it as the, the, the domains of intelligence, many of them, and maybe all of them are required in order to do something with your hands. It's a remarkable thing that, uh, planning, um, reasoning, logic, all of these things are

SPEAKER_33: connected to the way that we interact with the world through our hands.

SPEAKER_08: That's fascinating. Like when you were saying that I was thinking, so I put my hand on my chin and then if I were to, if you and I were navigating the world, uh, we were, you know, uh, early settler somewhere, we might point towards the direction we want to go, or I might put my hand on my chest to refer to myself, or I might put my hand out, uh, and, uh, my palm up to refer to you in some sort of gracious way.

SPEAKER_36: Uh, is that what we're referring to this sort of instinctual thing that happens with our hands as we're talking?

SPEAKER_00: Yes, um, our, our view in the, the position that we've taken is that the hands and their use and the mind are interwoven in an inseparable way in people. So if you want to understand human-like intelligence, the kind of intelligence that we have, um, the hands are the appropriate starting point. And that's why we focus so much on them. Now you asked to see some things I can actually show you one of the hands.

SPEAKER_27: All right. So I see on the screen here, you've got, uh, yeah. Uh, very interesting looking hand, uh, five digits, yeah, four fingers and a thumb and a

SPEAKER_42: palm, uh, and it looks like something out of Terminator, um, but a little more elegant. In fact.

SPEAKER_45: Well, I think that I would not characterize it that way. I think that the, the way that we imagine this hand is that it's the best that the technologies that the global community knows how to build. It's the best that we can get to a human hand. There's a lot of things in this hand that, that aren't immediately obvious just by looking at it. And those are mostly about the sensors. Our, our, our sense of touch is a very important thing for our intelligence and how we are in the world. Um, the, we, we tend to take it for granted because it's always there. Um, and when we look at screens and things, you know, people are very visual and they think about the world in terms of seeing, which is fine, but there's a, there's an interesting observation that seeing is about, about the future.

SPEAKER_00: It's about planning because the things that you see are away from you. So for example, if you look at a cup and you want to pick it up, the, um, the part of your

SPEAKER_45: brain that plans, um, thinks about the future, but touch is a little different touches an immediate

SPEAKER_00: thing. That's in the now touch is not, doesn't have foresight. It's all about the present moment. And when you make contact with the world, say you're seating in a chair or you're picking something up, or you're turning a Rubik's cube in your hand, the sense of now is, is intimately connected with the, with the touch sense. And without it, you can't be who you are. So this is an important thing for building robots is that touch is not a second-class citizen. If you're trying to build a system that, that behaves and thinks like we do.

SPEAKER_34: And so these hands are covered in, in very sophisticated touch sensors that allow them to feel the world, something like we do.

SPEAKER_28: Yeah. So when we're looking at each digit, um, I guess we have, uh, you know, a couple of

SPEAKER_08: knuckles, uh, and so the tip of your finger has one pad. Then that middle knuckle, um, I guess has another pad and then there's a longer pad in that third spot. So if you were looking at your own hand, you have those sort of three segments of a finger. They each have a pad on this robot and the thumb obviously has the same configuration. And I guess when they touch each other, that's telling it something's hitting that pad.

SPEAKER_49: Is that correct? It's more general than that.

SPEAKER_51: Yeah. So the, the, uh, the sensors, uh, in your hand are not just about a yes or no question

SPEAKER_00: about you're touching something. They're very rich. You can feel temperature. You can feel, uh, if something's sliding past your fingers, which is very important when you're trying to hold something or, or turn it in your hand, um, imagine trying to put a key in a lock and turn it a lot is going on in that, that, that thing in your brain. And a lot of it is driven by touch.

SPEAKER_52: If you didn't have the, the sense of touch, it would be very difficult to insert a key into a lock and turn it even something as simple as that.

SPEAKER_27: So these resistance, right? You have a certain resistance on either side and the top or the bottom of your finger where the, where it's touching the key.

SPEAKER_00: I'm imagining this as you're saying it, yeah, you're, you're the way that we do things in the world is, is not, we take it for granted. There's a thing called more, more of X paradox where it, the things that we take for granted and are easy, are actually some of the hardest problems that there are. And the reason they're easy is, is that we've had a billion years of evolution to create a system that is fine tuned to be able to deal with things like, you know, picking things up and putting things in things and things like this. But the reason why, um, we have, uh, AI systems that can write at the level of GPT four or create images, you know, that from, from scratch that are as beautiful as any of the human artists could draw, um, the wonders of the digital age, but we don't have a robot that can do laundry

SPEAKER_44: is that doing laundry is a fundamentally much more difficult problem than any of the ones that modern AI has, uh, has managed to master.

SPEAKER_42: Yeah. It's fascinating when you think about how complex this is and that paradox you mentioned, uh,

SPEAKER_08: that seems like, um, you know, like a, a fascinating, a fascinating evolutionary, a moment. These systems are so complex that they must be automated because to actually with cognition,

SPEAKER_62: to try to think, okay, I'm going to have to give some resistance as this key goes into it and I'm going to have to fear a click a couple of times and then I'm gonna have to twist it left, but I'm gonna need to put more pressure on the inside of my index finger versus my thumb. And if I put too much pressure, I'm going to break the key off in the locks. I mean, it's incredible when you think of all of that occurring and it's occurring in just an, an automated fashion. It's just a chunk of a task. Open the, open the door. It's not even one test either.

SPEAKER_08: It's probably open the door, which is the key getting taken from your pocket, being put into the lock, twisting, open the door, closing at the whole, the whole shebang. It's just abstracted into one instruction set, huh?

SPEAKER_45: Well, that's an interesting phrase and I'm glad you mentioned it because that's the way

SPEAKER_00: that most serious embodied cognition efforts work is they have an, an idea of an instruction set, which is very similar to the way that processes work by, I worked my first half of my career in building computer systems and the, um, the, uh, the marvels that we've built and the computing side are fundamentally based on a very non-trivial fact that every program that you could write boils down to the execution of roughly order, 100 different tiny little programs that just happen in different orders. So every program that you can write on a computer is basically composed of only about a hundred building blocks and modern processors, and the reason you can do that is that processes have a natural way to turn the analog nature of the world into a digital character that allows error correction, which is the fundamental reason why you can do all the things that we do today that in the computer, there's this thing called a transistor, which is the basis of this digitization going from things that are just any number at all. When you measure them like voltages and currents to something that's only a zero one in motion, that is the taking of actions in the physical world, you need to be able to find a way to do that same thing. And so what we've done is created this type of instruction set, which is a very small number of building blocks that you can compose in different orders to create massive complexity of tasks and potentially all of them. So if you think of a, a robot moving through the world or a person as a program, then you can imagine that any program could be written in, in say, maybe just a hundred things in different orders. And that's what we're trying to do here is to figure out what those hundred one things are, and then use a technique called task planning, which is the idea that given a goal, like, say, I asked the robot in natural language to do something, the robot can figure out how to

SPEAKER_44: sequence the things that knows how to do in order to achieve the goal and therefore, and thereby achieve general intelligence.

SPEAKER_00: Because if I can ask the robot to do anything at all, like in the human sphere, and the robot can actually perform the task, then it would be fair, I think, to think of these things as, as being, as, as reaching the goal of having general intelligence. By the way, I should mention that there's a concept that David Chalmers, who's a philosopher, introduced, called a philosophical zombie, where a system can have the appearance of being like us in the sense that it can do things like we do, but it doesn't have the first person conscious experience that we have. So there's a, there's a lot of mysteries about the relationship between being able to actually achieve goals, do things, and whether that is related or different than the experience

SPEAKER_44: we have of being people, this, this thing that it feels like to be a thing. That's a deep, deep mystery.

SPEAKER_66: All right, listen, efficiency needs to be top of mind for every founder in 2023. Fundraising is drying up. So you need to extend your runway. And one great way to do that is automation. But it's hard to apply automation in your day to day operations, isn't it? So here's an amazing solution. In-Touch CX provides easily integrated automation tools for customer support. You're wondering how it works? Well, let me tell you. In-Touch CX provides automated and live chat, email, and voice support. This eliminates unnecessary process and cost, and it will make you faster. In-Touch CX will streamline your customer support process, cut back on repetitive and time-consuming tasks, and increase productivity by 30%. And it's going to simplify your business. So revamp your workflows with In-Touch CX. In-Touch CX partners are experiencing 45% average cost savings in customer support ops so far. Find out how In-Touch CX can improve your startup's efficiency. Get a free consultation with their automation experts and get started at In-Touch CX dot com slash twist. That's In-Touch CX dot com slash twist.

SPEAKER_27: Yeah, consciousness, the big C is for philosophers and religious people.

SPEAKER_08: What is consciousness? Is it just some illusion that we're having in this brain of ours, which is a collection of a bunch of subroutines, as you're sort of alluding to here? Or is it, uh, you know, this God molecule in our brains, making us, uh, sentient and, and driving us to, to do things? I guess this is one of the exciting things about AI is that we're in some way, or in your case, quite literally, trying to deconstruct and then reconstruct what is happening in cognition.

SPEAKER_62: But it has to start with, Hey, pour me a glass of milk, so it has to know what milk is easy enough to do now, uh, with visual computing, but poor. Okay. We know what that word means.

SPEAKER_08: It's just moving, uh, some liquid from one place to another, and then we have to, uh, then of course, make it do that accurately. So if we were breaking down a task like that, and you said, there's about a hundred things you're teaching it. What are those little subroutines or those micro behaviors, or we used a term for it? Um, what was the term you used?

David Friedberg: Uh, we call them micro policies.

SPEAKER_00: So in, in the world of policies, interesting. Yeah. In, in the world of reinforcement learning, which a lot of this is grounded in, uh, our, our, our, we came up through the reinforcement learning school of thinking about cognition. Um, the, uh, a policy is, is, is an, is an action that you take from a current state. So it's a, it's a prescription for how you act, given the observation of the world that you have. Um, so these micro policies are, um, a collection of very specific types of behaviors. Like say, for example, turning a key in the lock that we train individually in, in, um, isolation from any other use.

SPEAKER_44: So the way that works is that we take the robot and we, we, a person who is teleoperating the robot, which is a process of, of a person controlling and being kind of immersed in the robot, receiving the senses of the robot and moving the robot through a, an, a rig, which is another type of robot that the person is strapped to.

SPEAKER_00: Um, so the person moves the robot to accomplish the task. So the person knows what it means to pick up a key and put it in a lock and turn it. And we collect, uh, order hundreds of, uh, episodes, which are instances of solving that problem. And then we use that to seed, um, uh, a thing we call large behavior model. So large behavior model is much like a large language model, except the fundamental data is, is the data of experience. It's, it's vision, audio proprioception, which is the information from the servos and the robot where it is and so on, how fast it's moving and, um, and touch haptics. So if you can, you can use the same idea where you take a bunch of data and instead of predicting the next word or token in a text prompt to response, you take the, the past, which is the things that have happened to the robot. And you predict the future, which are the sort of analog to the large language model, predicting the next tokens. But in, in, uh, uh, an interesting twist, the predictions of proprioception are predictions of where you will be, how you will move. And you can then send those predictions to the actual motors and the motors can move. So that one of the most, um, uh, fundamental change turning points in my professional career happened about, uh, 10, 12 years ago when, uh, I, and some colleagues read, uh, Jeff Hawkins book called On Intelligence, which was really the first thing that I read that made sense about a potential model of human cognition. And central to that story was the idea that our brains are predictors is that we, we imagine the future and then we, we implement the imagining. Uh, so if I decide to pick up a cup, my brain is predicting how my motor's signals will fire. And then it, it sends those predictions to my muscles and then I perform the task. So these large behavior models that we and others are working on now are of this sort is that they predict the future based on the statistical properties of the data that they've looked at.

SPEAKER_33: And then they execute the tasks and they work quite well.

David Friedberg: So for a human being, we're gonna perform a task.

SPEAKER_08: We're gonna pour this glass of milk. We will in our minds and we, we do this, uh, either consciously or maybe even subconsciously. Okay. I'm going to pick up the glass. I'm going to pour the milk. I'm not going to spill it. It's going to force in some kind of an arc. I'm going to watch it, Phil. I don't want to splash.

SPEAKER_62: I'm going to stop pouring at a certain point. And you kind of visualize this movie in your head, this potential future behavior.

SPEAKER_81: And so our minds are so, um, powerful, they can actually essentially play a scenario, like, almost like a screenplay, like a, a little vignette, and then our muscles actually go play that routine is, is that the concept here in terms of intelligence of what happens in our brains?

SPEAKER_00: Yeah. So an analogy would be, um, uh, let's say you take a, a piano keyboard and each of these little micro policies that we're talking about are one of the keys on the keyboard, your brain, your mind. Cause I, I want to make a distinction here is that I've, I've come to believe very strongly recently that the, the, the mind is a creator of stories about the future. You, the awareness that you are, your conscious presence is not your mind. It's a different thing. You touched on it briefly. I don't have any proof of this, but I am much more of the mind that the, there's a big mystery there about what it is to be the thing that is you, but it's not, it's not your mind. Your mind is a machine, just like your heart. And the job of your mind is to, is to produce stories. So in the analogy to the piano, think of the mind is creating sheet music when the, and then the sheet music is automatically put on the piano and the keys are played and you hear a melody or, or a song in the analogy, the song is the behavior. Like for example, picking up a glass of milk and drinking it, all of the behaviors that we exhibit in this model are all different songs that are generated by pressing the keys in different orders. And with, you know, different, different styles. So the mind is, is the creator of the sheet music that it does always, this is, brains are always doing this. Um, and then it's played on your body, um, like a song and, and our conscious perspective is sometimes not aware that these are separate things. You know, we have, um, difficulty introspecting our minds and our behaviors for a variety of reasons, but I think that after working on this problem for a long time and seeing the, the synthetic analogs of us, how it actually works in robots.

SPEAKER_44: I think this is a good model is that your mind is a machine for creating sheet music that is immediately played on the, the instrument, which is you and your awareness is a separate thing that kind of watches this. And sometimes we get confused and we think we are the mind, but I don't think we are.

SPEAKER_79: I think we're something separate.

SPEAKER_62: So there's the mind and then there's the machine and this machine is going out, conceiving of these tasks, executing on them, playing the sheet music, running through the script.

SPEAKER_08: I use this, the, the analogy of a film using the analogy of piano, but the script gets played, the sheet music gets played. It happens, but our awareness that I am a human, I am Jason, you're a Geordie. We are on a podcast. We're having a conversation. I'm trying to understand what you're doing. Uh, you're trying to understand my questions. And then there's going to be a hundred or 200,000 people who listen to this.

SPEAKER_62: And they're also going to try to, that's consciousness, the awareness of each other and ourselves and our place in the universe. And that there is even a universe.

SPEAKER_81: These are two different things.

SPEAKER_62: But for some reason, we perceive these two different things that are occurring, the mechanical execution of tasks through this very, uh, interesting project, uh, process, which you're now recreating is different than consciousness and consciousness.

SPEAKER_81: Who knows when we're going to ever figure that out? Or if we, if we can figure it out, this idea that we are aware that we are a living being, but we can figure out, uh, at least at this point in time, it feels to you. Like we're going to figure out, and we're close to figuring out how our brains break down complex tasks and do them so elegantly.

David Friedberg: Am I re reframe? Am I, um, repeating that back to you correctly?

SPEAKER_31: Yes, that's right.

SPEAKER_00: Uh, there's a, the, a spiritual, uh, leader, I suppose you could say called Eckhart Tolle, who refers to the first thing, which is the not knowing that you are different than the plans that your mind makes as being unconscious is the phrase he uses. Um, and I think that it's a natural state of people is to not be aware that the mechanical following of scripts, which is most of our behavior. And that's the sort of thing that you can, there's a shot to, to doing in a robot. So I'm, I'm fairly sure that we can build machines that can do all work, like all of it, at least as it's currently, you know, uh, understood things like automotive manufacturing, uh, logistics, bringing parcels to your home. Um, I think that all of those things are within scope to do within say a decade, at least have the capability. So this idea of building a thing that appears as though it's intelligent and does all the things that you'd want that's within reach. But the thing that I'm really kind of taken with is this other notion. You know, I used to be a theoretical physicist a long time ago, and I worked on foundational problems and quantum mechanics and general relativity. And I've always had, um, you know, at the base of who I am, I've always been interested in understanding how things work at some fundamental level. And it's always bothered me that all we ever experienced of the world is this first person thing, the feeling of being you in the moment, but we don't understand that at all. And, uh, I think that this, this neglect of what is probably the most central direct experience we have of the world means that there is a discovery waiting to be made about the relationship between our experience and notions of space and time. And, um, and, um, and I think that the, the, the, the, this project is somehow in some ways aimed at that is that, uh, you know, it's, it's sort of starts from a weird spot because you see these robots and there's mechanical hands. And then I'm talking about, you know, some fundamental relationship between the emergence of space and time and how we perceive it through our conscious perspective.

SPEAKER_33: And as he seems to be not related, but I actually think that they are very tightly related.

SPEAKER_91: You see these blue light glasses I'm wearing. I, I, I, I'm not wearing for style, although they are very stylish. They've totally changed my life. Why? I started having headaches, right? And had eye strain. So I got these blue light blocking glasses that do a little magnification because I need readers. Yeah. I look nuts, but my eye strains gone down. My headaches have gone away and I'm sleeping better. Do you know how I got on this? I got on it because I now have a health coach. Who's my health coach? It's found F O U N T. It's a health company that's created custom health and performance programs that are tailored to your body, obviously. Also your goals and they take into account your lifestyle. My coach is incredible. I text with them all the time. They did a blood work for me. They check out my wearable data and we do weekly calls to see if I'm on track and getting results I want. They also told me about some supplements I should be taking based on the blood work. And they do it at a fraction of the cost. We upgraded my diet. I'm doing a little more protein. We've optimized my sleep. That's great. I got the supplement packs. I feel great. I feel like I'm in control of my destiny. If you want to be like me and you're concerned about your health and you want to just try to do better, have some experts on your team. Build your own program. Go to found dot bio slash twist. That's F O U N T dot B I O slash twist. Get your free consultation. Mention twist. You get $500 off your first month and get your own personal health coach. Health is well, and if you're running a startup, if you're a CEO, if you're a capital allocator, take it seriously. I love this service. Found dot bio slash twist.

David Friedberg: Well, if we, if we think about the experience of being human and our place in this universe that we're trying to figure out.

SPEAKER_81: Performing the tasks, as you said earlier in our conversation, is how we navigate the world. And it's, it's how we are actually doing this act of trying to figure out what it is to be human. And this all then starts to open up all kinds of possibilities free. Well, when we pour that glass of milk, when we play that sheet music, where is our decision to do that occurring?

SPEAKER_08: You know, what, what parts of it are automated, which parts of it are just wrote and, and, and just get executed on. And so it does open up, uh, and I agree with you, this is the, the question that we will, uh, always try to figure out. And this is why science fiction, you know, uh, always winds up here, which is what is it, what does it mean to be human?

SPEAKER_62: Whether we're talking about Blade Runner or Prometheus and the, and the alien series and really Scott's take on it.

SPEAKER_81: So let's get back to reality here when you're training the robot, you are not saying, Hey, we're going to pick up a tennis ball here. Uh, if we're going to pick up a tennis ball, it has this size, therefore we're going to program it to pick it up. That's what people did with robots before they very explicitly had to do some very narrow verticalized tasks.

SPEAKER_62: You're having a human being, uh, like the guy who played Gollum, I guess, in, uh, Lord of the Rings, uh, use gloves or something to send the instructions to the robot's actual physical actuator hands. And they're incredibly, uh, sensitive and have those pads on them. So we're teaching it, Hey, I'm going to just pick up, uh, Andy circus.

SPEAKER_08: Yeah, he was a guy who played, uh, uh, Gollum.

SPEAKER_62: We're going to actually just pick up the tennis ball and then the AI that we train is going to know what happened and, and, and that's what's going on here.

SPEAKER_44: Yeah. So I have got, uh, uh, uh, this might be helpful. Can you see the, here we go? Yeah. Yeah. We got a video of a robot.

SPEAKER_45: Yeah. Yeah.

SPEAKER_00: So this is the, this is Phoenix and what you're seeing is the, um, uh, this process that we're talking about where there's a person in a, in a suit, they have haptic gloves with force feedback. So if they can feel the world, they see through a heads up display. So they can, if they feel like they're looking through the eyes of the robot and they're connected to their own robot, that when they move, moves the robot in an analogous fashion. So the, um, uh, uh, yeah, so this is what, this is the, what it looks like when you watch the robot side of tell operation, you can see that these machines are, um, uh, capable of doing lots of different things. I mean, that might not be obvious from watching this, but the, the systems are nearly capable of doing anything that a person can do under this type of control. Uh, as long as they don't have to move around the world, this is focused on the upper body stuff.

SPEAKER_44: And the, the problem that I mentioned, which is the dexterous manipulation of the world.

SPEAKER_08: And you're seeing it there, you know, basically pick up, uh, an object, uh, and then scan it with a barcode as if you were, uh, working in an Amazon, let's say factory and shipping and packing boxes, or even doing something as, uh, delicate as using a Ziploc bag, uh, which we do unconsciously. Unconsciously, we feel it, it feels like the Ziploc bag, yellow and blue may green, and you just, you, you have that. Color system, but you also have the feeling of it.

SPEAKER_62: So humans, uh, do the tasks and then take me to what the software then does with the human having done the task. What does it, what does it do next? Uh, in terms of building a model to then go do the next thing in the world.

SPEAKER_44: Yeah. So the, uh, um, so imagine you have a, uh, uh, a Reddit post, which is some sequence of words that, uh, someone says, I, I really love Diablo too, because, you know, Amazon is my favorite character.

SPEAKER_00: So somebody has written something that sentence is the expression of a thought that a person has had into words. Now, when you train something like a GPT large language model, that sentence is used to help figure out the statistical likelihood of each word. Let's just keep it simple, following the preceding ones. And if you give this model enough words that people have written the expression of their thoughts, then if I was to say, type in a prompt, which is my favorite game is. Then of, or all of the words that have ever been written to, to, to some approximation, there's a probability of what the next token will be given that prompt.

SPEAKER_44: And then the thing can unroll, which means I put the next word in. And now I ask with all those four words, what's the fifth word. Okay. Put the fifth word in what's a six.

SPEAKER_00: And each time it's a probabilistic thing. So you roll a random number and you pick the thing that the random number says it should be. So with this type of model, these large behavior models, the data is a little different. It's the data of the sequence of, of, of successive nows. It's the time data from the person performing the task. So if I asked the robot to open a Ziploc bag, let's say that's the micro policy that we're going to train. So a person picks up the bag from the table through the robot, opens the bag and, you know, maybe pulls it open a little bit. So that then becomes the analog of a sentence. It's a piece of data, which we're now going to use to train a model where instead of predicting the next word, we're going to predict the next, um, sequence of actions. And we unroll the same way we would a sentence. So every successive, uh, prediction becomes a movement pattern for the system. And in this case of the kinds of things we build while it's, it's, uh, it's similar in, in some sense, there are some very big technical difficulties in actually doing this that require the synthesis of many different kinds of artificial intelligence advances. Uh, for example, you could send the pixels from the camera in at every step to one of these models, but the pixels are not, they're not the thing that you really care about. What you really care about is where are the things and what are they, which is a much lower dimensional thing. So machine learning, computer vision techniques have been developed that will take the camera feed and extract what you could think of as the semantic or important information about the scene. And those are typically the things that you put into these types of models.

SPEAKER_44: Uh, and that that's not just true for vision. It's true for haptics and audio and proprioception.

SPEAKER_00: So on the audio side, the obvious thing is if a person speaking, you could use the actual audio waveform, but you could also use the text and text as a much more compressed and high quality version of the, the data than the actual audio itself.

SPEAKER_44: So we, we tend to do text extraction from speech before we send them into these types of models as well.

SPEAKER_08: Uh, that's fascinating. So you can know, uh, with machine learning, Hey, there's a bag in the scene and the bag is open and, but the bag's upside. Down. So it needs to be, uh, we, we should flip it around. So these things don't fall out of the bag, et cetera. And so where are you at? Let's let's, I think I understand what's happening here, uh, in terms of the language model, uh, analogy, uh, and then just translating that into predicting the next best thing to do. And so where are you at in terms of training this in the real world? I assume that factories and the example you give looks like, uh, you know, I'm packing, uh, and shipping probably one of the most boring, monotonous, uh, soul crushing jobs a human being could have. So why not give that to a robot and sure, you could do it 24 hours a day or whatever the robots are going to be capable of.

SPEAKER_62: So where are you at in terms of taking this and actually having it at a, a fulfillment center, packing boxes and making sure that it scans them and puts the right.

SPEAKER_104: Objects into the box and then ships them onto the next person and being on, uh, you know, this, uh, distribution center floor.

SPEAKER_44: To be clear, the initial go to market is in automotive manufacturing.

SPEAKER_00: It's not in logistics and retail. We focus almost exclusively on that with, uh, with one exception, uh, the, in automotive manufacturing. If you, if you take a look at a video that say Toyota makes of their, their factory floor, automotive manufacturers, one of the most automated, uh, systems there are in any industry. But if you watch what actually happens, there are hundreds of thousands of people in automotive manufacturing facilities all the time. So the question is why, why aren't they being automated? So when you look at what they're doing, there's kind of two categories of answer. One is it may be beyond the bounds of science. We may not know how to do the thing that they're doing, but there's another answer, which is that often people are used to connect machines. So let's say I have a machine for stamping a part and I have a machine for, uh, putting, uh, for making the part in the first place. So moving the part from the one machine to the other machine is a very difficult process that involves all of these things that we talked about.

SPEAKER_44: You need to be able to know what a thing is, where it is, localize it, use your hands to pick it up, sometimes out of a cluttered mess, put it somewhere, which often requires putting something on a jig, which is a difficult thing.

SPEAKER_00: You need to be able to move around and so on. So the, a lot of the work that's done in automotive manufacturing specifically is a combination of different solve problems that have never been put together in a way that you could make economic. And the, one of the key factors of these general purpose machines that we and others are building is that this is exactly the kind of thing that's required in order to, to actually do this for real. Is that if I, if I was to spend all my time and energy building a machine that did one of the things that somebody in this factory floor does, it would be very difficult to build a business. But if you could build a machine that could do say 50 of the kinds of things, now we're talking. So our, uh, our initial, uh, use cases are nearly all of this sort. They're automotive manufacturing. They're, um, they're the connector problems where you're moving between machines with parts or things of the material. Uh, uh, and even the things that aren't automotive manufacturing that we've looked at all share the same feature. For example, in warehouses, which was my first, like my last business built robots for, uh, for e-commerce distribution centers, there's a problem called induction. So induction is, is the problem of taking things that usually come off trucks, you know, big pallets and, uh, just stuff, you know, imagine all of the things that are, you could buy on Amazon coming into a warehouse and then, and then, and then taking. Taking them from their, their point of delivery and then putting them, getting them wherever they should be in the system, on a shelf, in a box, whatever. So induction is, is another kind of problem that's related to this, where you're dealing with system, things that you need to manipulate with your hands, opening boxes, closing boxes, putting things in boxes, taking things out of boxes and so on. Um, and so that that's another category of things that's related, but nearly everything we're doing now is, is, is helping automotive manufacturers dramatically improve the, uh, efficiency and productivity of their workforce.

SPEAKER_117: We're back with another pitch it to J Cal. This is the segment brought to you by our friends at dot tech domains. Dot tech domains are giving twist listeners the chance to show off their startup on this week in startup. So go to startups.tech slash Jason. That's startups dot T E C H slash Jason to apply. There's only one rule. You need to have a dot tech domain name to get featured. This week I received a great pitch from label drive, which you can find at label drive dot tech label drive helps other companies manage their AI data. And they've built a tool for collecting and labeling data. That's especially focused on identifying and categorizing objects to save your time, save your money and build better products. And as we all know, that's crucial for AI training. So I want you to go right now to label drive dot tech. And if you're interested in getting featured on this week in startup with your new dot tech domain name, I want you to go to startups dot tech slash Jason and apply today.

SPEAKER_119: That's startups dot tech slash Jason and fill out the form to apply.

SPEAKER_08: You know, if this works, uh, when do you think you'll have the ability to have the robot, you know, find those 50 different things to do. And let's say, you know, that it feels like you're, you're well on your way, um, the first question, when do we, when do you get that solved and in factories and just doing it day in and day out?

SPEAKER_00: The plan that we've got, uh, takes us from where we are now to the full automation of important tasks, by which I mean, there are markets for say billion dollars of annually recurring revenue for us. So say, let's say, let's say that's a kind of a thing that we want to target. We enter into agreements with our customers where the first step is that we mock up their, their situation in our facility here in Vancouver.

SPEAKER_122: Think of it as a digital twin or not a digital twin, a real world twin.

SPEAKER_00: And there's also a digital twin, by the way, but the real world twin, and then the processes that they, they pay us to ought to show that we can automate using this type of thing, the kinds of tasks that they want as a first step. So there's a, there's a period of a roughly two and a half years that we see where we go from where we are today to being able to really do something for real in the lab of the sort that you could then scale. So that's the first step when we start scaling is likely around the middle of 2026, where you're going to start to see, uh, the increasing number of these types of machines actually deployed inside automotive manufacturing plants, contributing to the productivity of the plant. Uh, so this is the plan of record. Now I've done, I've, you know, I've done quantum computing and, uh, all sorts of things where it's very difficult to predict how things will go. So, so the, the, the, the, in something like this, you have a plan, um, things could go faster, it could go slower, it's unclear, but that's what we're aiming at. I think you'll start to see the beginnings of large scale deployments of these machines somewhere in 2026.

SPEAKER_08: And so 2026, you start seeing the deployment of these, and then when do you think factories start to remove humans? I guess they call that the lights out moment.

SPEAKER_42: You don't need to have the light. You don't even have to install lights in our, you know, space.

SPEAKER_08: Uh, it's, I know it's funny, but when do you think you have that lights out moment? Uh, and, and factories don't need to have humans in it.

SPEAKER_45: So I want to make a point about this. There is a myth that AI and automation reduces, uh, labor.

SPEAKER_00: It's not true throughout history. There have been a series of, of moral panics where the next big technology thing is believed to do something terrible to employment. It's never happened.

SPEAKER_128: Every single time there's been a new thing introduced. And, and I think that the, the, the, the central reason for us thinking this is that it's the lump of labor fallacy.

SPEAKER_00: The idea that there's a fixed amount of work. And if you, if you give the work to the robots, there's nothing left, that's simply wrong. Um, the way that, the way that it actually works in practice is that when you, when you give, say, like you give a bunch of labor, like I want to build 80 million cars. So that's a fixed amount, let's say we could do that all with robots, the amount of work that's available for the general human population expands as a consequence of that. It doesn't shrink. So the, I want to just make this very clear that my perspective on AI and automation is that there's an upward spiral when you have more energy, you have more intelligence, you have more capability. These drive all the metrics of human flourishing up. They don't take. So when we think about the answer, when will you get lights out manufacturing? I think the answer is never because people will always find new things to do with the tools that we've built, even very powerful tools that, that can think. And maybe are even self-aware, uh, these will only increase the number of jobs, the increased wages, but there'll be different kinds of jobs.

SPEAKER_122: There'll be the sorts of things that maybe we can't even imagine now that are made possible by these things.

SPEAKER_120: Like, look at the internet 20 years ago or 30 years ago when I, this is a great example.

SPEAKER_122: Yeah. Yeah.

SPEAKER_42: Now we have an entire podcasting industry.

SPEAKER_08: We have people who take pictures or, uh, you know, there's an incredible song, a company called song Finch. What they do is you go there and you tell it, you want to make a song for your mom or your dad. They pair you with an artist and you pay them $200 and they'll write a song about your mom for her birthday. That's very cool.

SPEAKER_81: Well, I mean, just there's, there's humans out there and I guess these used to be bards or, you know, court jesters or whatever, who would, who would do these kinds of tasks as well.

SPEAKER_08: But we find things and you're just thinking about your robot and oh, well, we have this new problem, forest fires. Uh, well, how are we gonna def, you know, clean up the, how are we gonna rake up as our, uh, former president, you know, joked of, you know, how are we gonna, how are we gonna, uh, rake up all that debris under, uh, the trees there, uh, you know, in the mountains in California. You know, if somebody had 10 of these robots, uh, able to do a test, they might say, oh, you know, I have an interesting idea. Maybe we could clean up the, and do some deforestation with them. And they will eventually, uh, in your mind, a decade from now or two decades from now, uh, not just be in factories. They'll, they'll be in our lives. They'll be side by side with us solving problems in the real world. Yeah, that's the ultimate vision here is that they could leave the factories.

SPEAKER_00: Yeah, I, I think of them as being, uh, a kind of thing like the automotive industry, where at some point, they'll be ubiquitous and parts of, and our entire civilization will, will, will be built in synergy with this new thing, like we did with cars, you know, roads and so on. Um, by the way, I wanted to mention that this happened, this, this business of the job upgrade happened to me when I was, uh, starting school. There were no quantum computers, um, at all, except maybe theoretically. Um, and we started a company to try to build one. This is an example where the, we probably hired about over time, I don't know, maybe 300, 400 people who had PhDs in physics in that company. Uh, that, um, and this is D wave. Yeah, this is D wave. Yeah, that, uh, that that was a new kind of job that was created as a consequence of a, of a revolutionary new idea. So this is the sort of thing that always happens with, with, uh, innovation is that. And, and I, I, I'm kind of, uh, emphasizing this a little bit because we're at a very weird time right now where there's an attempt to do regulatory capture and artificial intelligence. It's a very dangerous idea. This idea of D acceleration or stagnation or, or holding back, um, which are connected to ideas of, of the old ideas that were rooted in communism. Uh, these are very dangerous social ideas that I think it's, it's important that, that we don't stay silent about. And people like me who have very strongly held beliefs that technology is the solution to maybe all of our problems, not only the ones we create, but also the ones that might emerge as a consequence of our natural habitat, you know, global warming or meteors or whatever. The idea that we, the better we can get at creating new things, the better we are all is a very important policy idea that I don't think is being communicated effectively by the community of people who build technologies.

SPEAKER_08: You know, there are a group of people who want to, uh, have the government and they've specifically gone to Washington and said, Hey, please regulate us. And they happen to be the people who are at the, maybe at the forefront or some amongst the people at the forefront and, you know, building a bunch of regulation into this would benefit the people who have the lead today. As opposed to say open source people or, or, uh, folks who are coming up is that the thinking of what their motivation is because this is a group of technologists who are on the cutting edge. Why would they go to Washington and want to in have a bunch of, you know, uh, non-technical politicians, uh, slow things down? What, what do you think their motivation is?

SPEAKER_44: I think the best answer to this is, is somebody that you had on, on, uh, Bill Gurley. Yeah.

SPEAKER_00: So his, his take on this, I really resonated with it was one of the most, uh, you know, sometimes you watch something and you're like, I'm disagreeing with everything this guy's saying right now. I think I, I would, if, if, if people are interested in this subject and they haven't seen that, I would most definitely recommend it.

SPEAKER_42: Bill Gurley all in talk, yeah, we'll put on the show notes for everybody. Yeah, but the regulatory capture is what they're going for. It, it, it calcifies the winners as the winners, it, it builds up a mode for them and this could be just cataclysmic for humans, right?

SPEAKER_08: Uh, we need this technology to solve problems.

SPEAKER_151: Yeah.

SPEAKER_00: And it's, and it's, um, I think that's the point is that the, the solving of problems comes from, from innovation and, and, and growth, uh, and the, the, the, the forces of stagnation. The people who are pushing for not that, um, are very strong right now. And I think it's, it's dangerous because I think that the, the, the, my view of this is that civilizations metrics, like how well people are doing are very strongly correlated with growth. And there's an idea that we have to slow everything down, which is, is I think a dangerous idea. I think that what will that, what would happen if we were to implement policies that were restrictive is the same thing that happened. I'll use an example with nuclear power. A lot of the problems that we face today in the global warming sense and catastrophic, you know, potential futures that we might be looking at are, are connected very strongly to a, uh, the precautionary principle, which the, the, in the nuclear industry was, well, we, we don't want to build nuclear power plants because we're afraid of nuclear bombs. Which is ridiculous by the way, because they're not the same thing at all. Not the same thing. Uh, yeah. And, you know, maybe a, a, a reactor melted down once or twice. Yeah. Three mile island. Uh, yeah. You don't count all the deaths that happened in all these other industries, which were massively higher. If we had not done that with nuclear instead embraced it, we would not be where we are today. And so there's, there's examples of this, uh, fear, which is a rational fear that can become policy that, um, it could be very dangerous here because I think that this, these technologies we're talking about, which is the AI, uh, robotics to a certain extent. Um, but not just those things more generally, we should take the attitude that, that the spot upward spiral is the objective. We want more, we want more energy. We want everybody on the planet to come to the energy consumption of us. We don't want to reduce energy consumption. We want to increase it. Sure. And then we want to increase everybody by another thousand times. And we need to be able to find ways that technology can enable that and then enable solving the problems that might come of it, uh, from second order effects like global warming. These things are all solved by innovation and technology that it's innovation and change is not the enemy. It's the, it's the, it's our friend. It's a necessary part. Yeah. And it's connected to who we are as, as, as people, you know, people are explorers. We're adventurers. We want novelty. We want to go to the, you know, to places that no one's ever gone either literally or figuratively. Uh, and that is the essence of the human spirit to me. And we want to be advocating for that as technologists and, and, and leaders in, in our fields. Yeah.

SPEAKER_42: And it's so paradoxical. I remember when I was a kid, uh, all these great musicians who I loved, Bob Dylan and et cetera, did the no nukes concerts.

SPEAKER_08: And we really were indoctrinated into this fear of nuclear.

SPEAKER_62: And the second order effect is that we burned more coal and we burned more oil and we heated up the planet. And now we're trying to solve the problem. And the solution was there in the seventies. And then sometime in the eighties, we decided, Hey, let's stop doing this. And now eighties, nineties, two thousands, two thousand tens. We're sitting here four decades later.

SPEAKER_81: And finally people are starting to realize 40 years later. Oh, you know what? Maybe that was a mistake. Should we start building these again? And now we've got to reconvince everybody that we went on a 50 year side quest that made no sense. Uh, and it's incredibly frustrating.

SPEAKER_98: And, you know, it's, uh, yeah, to some of our friends, people have been on the pod, Sam Altman, Reid Hoffman, Mustafa.

SPEAKER_08: Like they, I think they are misguided here. We, we can have conversations about this, right? I mean, there's nothing wrong with having a conversation. Hey, how do you make nuclear safer? Hey, could these robots, I mean, it's, it sounds farcical, but could the robots escape and, and then do bad things in the world? Sure, we could have this conversation, but that doesn't mean that we need to have a bunch of regulators come in and say, Oh, somebody in Washington's gonna approve your language model and your code. And that doesn't make much sense to me. That seems like they're doing regulatory capture. I agree a hundred percent. Um, and you know, another thing I realized about what you're saying, Jordy, is, um, there's something about solving problems.

SPEAKER_62: I realized in this conversation that when, and when we were talking about jobs and then there's a sensitivity to that with good reason. We, you know, we automate things and a large amount of jobs could go away quickly and there could be displacement, of course.

SPEAKER_160: But when the mind and consciousness is left alone, our minds are designed in a very interesting way to think and find the next problem to solve.

SPEAKER_62: There's something fundamental about human consciousness and this brain and, you know, Darwin and evolution that our species survive dominated and evolved with something inherent in our code, which is understand the world and find the next problem to solve.

SPEAKER_163: Is that any of that resonate with you? Oh yeah.

SPEAKER_00: I mean, everybody who's light awake at night and they can't get their, their, their mind to stop spinning through all the negative scenarios that could happen. Everybody I think experiences this. You're exactly right. Is that this tool that we've got this beautiful mind that does all these wonderful things. It creates the worst nightmares possible about what will happen as a consequence of it working well. And so with technology, our mind spins up all of these, uh, horror science, fantasy ideas, we turn them into movies like Terminator or black mirror. None of that is real. I think there's a, there's a very important, powerful message here is that the, the, the terrible stories our minds tell us when you lie awake at night about your personal life is the same process that generates fear about the, the outcomes of change. Hmm. So when we do something new, we innovate, we discover something about the world. There's a natural tendency that all of us have to. Imagine what might go wrong and. Yeah. Yeah. So my, my, I would advocate for being aware of that is that it's a story your mind is telling you. The Terminator thing is not true. It's not real. It's not never going to happen. It's just a story that somebody made up that resonates with our, with our basal base nature fears.

SPEAKER_122: And concerns about the future and so on, but it's not real. What's real is very different. Yeah.

SPEAKER_22: And in our minds, there was a reason this obviously existed or the, the person who worried.

SPEAKER_08: Hey, I wonder if these berries are poisonous or not, or I wonder if there's something dangerous in that body of water. Maybe I should be cautious. Yeah. A little bit of caution thoughtfulness probably extended life and people who were reckless probably had shorter lives. And so, yeah, the gene pool probably evolved this way, but you, you must be aware of how catastrophizing it is. I mean, people can get really wound up.

SPEAKER_62: We see this with social media presenting us with so much bad news in the world. Our brains are not designed to process that. Are they? No.

SPEAKER_00: No. And, and the, this is a, an example of how technology can have unintended consequences that are negative. It's social media hijacks this propensity that we have to, to tribalize, to fear, to other, um, to see people, other people as being different. You know, what part of this, um, idea of, uh, thinking of this conscious perspective that you have is separate from your brain carries with it. Another idea that we're all connected. You know, we all have this thing. We all share in it. Uh, the, the analogy that Eckhart Tolle uses is that there's an ocean and we're ripples on the ocean, but this ocean is the same for all of us. Uh, this, this idea is, is a powerful one when you're trying to think about why you're reacting in a certain way to certain things. You know, like the social media stuff is an amplifier of the negative aspects of how our, how we function as, as people. But that doesn't mean that we shouldn't have done it. I think this is a point, is it, like you said before, we want to talk about it. We want to have a frank discussion about it, but the solution to these things doesn't come from shutting things down. It comes from having this discussion and making, and making good, good, good, clear-minded decisions about how to, how to, how to build, not, not how to break. Yeah.

SPEAKER_08: And one of the great paradoxes of all of this might be, we build up this AI and we, and we get to some general intelligence. It might tell us, uh, it's a non-zero chance. It might explain things to us about our own consciousness, why we're here and what consciousness is that we ourselves could not come to the, to the answer. So we may unlock some mysteries, uh, that explain our own existence, uh, in a way, uh, and, and, and that is just to me, uh, would be a wonderful gift of accelerating this. You know, is what, what if this machine, what if this artificial intelligence can be more objective about us and can teach us something, right? That would be a pretty mind blowing outcome. Sure would. Yeah. All right, listen, continue success with this, uh, from, yeah, just, uh, working on, uh, quantum computing and now to robotics and figuring out how to make, uh, you know, these sequences play. Uh, it's gonna be very interesting to watch your progress and listen, accelerate it all. Let's go. Uh, I I'm assuming you're hiring and you, and this must be most, one of the most fascinating places to work in the world. Uh, if people are interested in learning more or, or maybe applying for a position to, to build this out and accelerate, uh, human, uh, intelligence and, and, and augment it, uh, so beautifully. Now, where can they find out more?

SPEAKER_00: Uh, so I, and one of the other founders of the company, Dr. Suzanne Gildert have a podcast, uh, called the sanctuary ground truth podcast. That's a place that you could, uh, you could look. Uh, we also at our website sanctuary.ai, there is a careers page. We are hiring and, uh, growing quite quickly, and there are positions for all sorts of different kinds of people. We mostly hire technical people, of course, but there are some other things. And if anybody's interested, please watch the, uh, the ground truth podcast and, um, uh, go to the website and check us out.

SPEAKER_41: Amazing.