SPEAKER_00: If you go back in time, I built a technology called LLVM, which is this fairly obscure compiler technology that then is probably on your phone today and on many of your laptops and in your consoles and things like this. That technology helped unify a generation of compute around CPUs

SPEAKER_01: in particular. And so LLVM was great for hardware people because they could integrate with LLVM

SPEAKER_00: and then they got all the C++ and all the Swift and all the other languages and Rust and Julian and things like this for free. But machine learning doesn't have that. And so what modular is building

SPEAKER_03: is it's building that thing that once you plug into it, you have a full AI stack for heart for

SPEAKER_04: hardware maker. That's a very powerful thing. This Week in Startups is brought to you by Roots. Invest in the only real estate investment trust that creates wealth for you and its residents at investwithroots.com slash twist. SuperGut is the only nutrition brand clinically proven to improve digestion, balance blood sugar, sustain energy, and manage weight. Save 25% on the delicious shakes, bars, and prebiotic mix at supergut.com with code twist. And LinkedIn marketing. To redeem a free $100 LinkedIn ad credit and launch your first campaign, go to linkedin.com slash this week

SPEAKER_08: in startups. All right, everybody. Welcome back to This Week in Startups. Really excited for today's

SPEAKER_10: guest because he's worked at some of the biggest technology companies in the world and working on AI. His name is Chris Lattner. His company is modular. He's worked at Apple. He's worked at Tesla.

SPEAKER_11: He's worked at Google. And now he's got his own startup, as I just said, modular. So as we all

SPEAKER_13: know, NVIDIA is dominant right now in the AI space. $16 billion in revenue in Q3. That's 2x year

SPEAKER_07: over year. They're wildly profitable. Stocks doubled since 2023. But as we've said on this pod and all

SPEAKER_08: in, and there's going to be competitors coming, right? Of course. And some startups are going at NVIDIA on the hardware front. We had light matter on recently, episode 1787. And they're trying to use optics, photonics based chips, basically, to move data around. It's going to make things cooler in data centers and help with these large AI jobs. Well, Chris is taking a different approach at modular. They're going to make it easier for developers to run AI modules on non-NVIDIA hardware. And they just raised $100 million, as AI companies are apt to do in 2023. Chris, welcome to the show.

SPEAKER_15: Wow. Quite the introduction, Jason. Thank you for having me. It's great to be here.

SPEAKER_08: Yeah. Great to have you. And you are in the thick of it. One of the things I hear over and over again from people deep in the AI space. I had a conversation with Elon about this not recently. Um, and we see it at open AI and other places is only a small amount of the hardware that's being purchased is being used at any given point in time when AI jobs are running. So for people who are technical, but maybe not working in the specific field, why is it that when we push a job, you know, we're doing chat GPT five or Claude 7.0, whatever people are doing, they're doing a Lambda or a Lama. I mean, there's just so many different things on hugging face right now. Why is it that so the the hardware is not optimized to these jobs? Why are we find ourselves in this? And then what is the actual

SPEAKER_10: percentage of the hardware being used, whether it's an H 100 a 100, or my M two on my MacBook Pro?

SPEAKER_16: Yeah. So it's super interesting. If you zoom into what is AI these days, right? So many people focus

SPEAKER_01: on training. You have to start with the research. You have to start with models. Models are changing all the time. I mean, just follow what's happening. It's hard to keep up with the pace of innovation in the model architectures, but then there's also the inference side of things and the deployment side of things. And so these two markets, these two problems are actually completely different. So what you're talking about is you're actually referring to the training side of this and modern training jobs,

SPEAKER_00: as many people know, have gotten huge, right? You get tens of thousands of nodes, thousands of GPUs. These are monstrous jobs. And so because of that, what you get is these timesharing systems. And so it's super funny. Like we went from personalized computers all the way back to the mainframe or the job sharing, like I'm going to put in my punch cards, right? That was Perot

SPEAKER_23: systems. Yeah. Yeah. Yeah. And so we're back on somebody's mainframe. Well, yeah,

SPEAKER_01: so we're back in those days. And so the actually better analogy, if I'm not joking about it is HPC systems. And so if you, if you go back 10 years ago or something, you'd get one of these massive supercomputer systems that a national lab would install. And then researchers would have to like walk up and allocate time against it. Right. And so the big question then is how do you amortize the spend for the hardware across a lot of work that happens on any one of these massive

SPEAKER_22: supercomputers and training systems today? They're massive supercomputers in every way, shape and form. The program malls are very different. The workloads end up being a bit

SPEAKER_01: different. And so there's some differences, of course, but the way they get managed is very similar. Now, what I've seen is different groups that own these things manage them sometimes better, sometimes worse. Right. And one of the challenges you'll see is that, for example, the big research teams may allocate, you know, 20,000 GPUs or something. But then the question is, how do you fully utilize it? This is, this is one of the cases where timesharing like clouds are actually really great because often you're not training models all the time, right? Your model training is actually proportional to the research cycle that you've got going on. And so if you're,

SPEAKER_00: you know, one of the massive companies like Google, where you have thousands and thousands of researchers, what you'll do is you'll have this big hardware pool and then you'll have the

SPEAKER_22: researchers that are all like effectively putting in their slot so they can use the machines when they come up and then they run their batch job for perhaps hours, perhaps days, perhaps months, right? And they get allocation for it. But if you get these smaller groups where sometimes they're

SPEAKER_01: on cloud, and so they're just renting by the hour, sometimes they build their own data centers. And then what they, the problem they have is, okay, cool. You have all this hardware. How are you utilizing it? Is it being productively used? And so these are major questions. I think that the entire industry is struggling with, but if you go just adjacent to that, that's training, that's where, where the models come from. If you go to production, the character is completely different. And so here you're not talking about supercomputers here. You're talking about the fact that, you know, you may have tens of researchers that train a model and they use a

SPEAKER_22: massive amount of hardware to do so, but then you need to deploy that model. Let me deploy the model.

SPEAKER_29: The problems are completely different right here. The problem is you have a billion users

SPEAKER_31: and a lot of queries and then a lot of follow-up queries and people want to, I guess, I'm not sure

SPEAKER_07: what it's called when you, well, there's prompt engineering and the prompts are getting more

SPEAKER_35: sophisticated. So all that creates load on the system. Yep. And the load on that system is really

SPEAKER_00: different. Instead of it being one massive computer that is then batch scheduled, what you need is you need scale out. And so any one of those systems is actually a single node often, but now you need thousands and thousands and thousands of these nodes. And those are fully utilized because you

SPEAKER_01: got users in 24 time and all the time zones. Right. And so that's actually a very different problem. And it's super interesting. And so if you look at AI today, it's super fascinating to me, how much energy has been put into the training side. Everybody's always talking about the research models and the training and the training and the training. Few people talk about what it takes to get that thing into production. Yeah. And one of the big challenges that we as an industry are facing today is that, you know, these systems that people build with like TensorFlow and PyTorch and these kinds of things were always built by the research team for training. And so getting that model in

SPEAKER_22: production is super difficult. And this is almost an unsolved problem these days. And one of the challenges there in particular is it's not just about cloud, right? Often you want to train a model and then put it on a phone. Right. And so that's a very different problem space. And it's much harder than done. Um, some, I mean, it's very, both of these problems are really cool, but it's

SPEAKER_08: explain to folks after all the training has been done, and then you have this language model, um, and, uh, you, you then want to load it onto a phone. How does that all work? What is the output? And how would you explain it to, you know, a lay person of, Hey, we built the model, but now we want to distribute the model to a bunch of different places and then let you play with it. But what is required

SPEAKER_00: there? So, um, I don't think that it would be in good taste to talk about how we do this because it is so complicated and nasty and, and horrible that we cannot go into all the details, but I'll give you a sense because that's, that's how I am. Right. So, so if you take a traditional enterprise that's building ML into their products, right? Often they're not building one model into one product, right? So they have some, they have many different kinds of models, some recommender models for like, Hey, maybe you should look at this in your shopping cart next. You have classification models. So you're looking at, okay, well, you, you like that shirt. Like maybe you should pick this shirt, some, you know, there's many different kinds of products. They then get matrixed into many different kinds of things that they're deploying into. So often cloud is a big deal, but then you have mobile apps and a lot of other things. And so what has ended up happening is that deploying ML today involves building this entire matrix of all these point solutions, because there's no one thing that allows you to span across all of these things. And so what you end up using is like this catastrophic array of like 15 different tools. And all these tools have different problems. Like, so I, I'm, I'm, I'm a Apple sort of an Apple

SPEAKER_51: alumni. I, uh, have a ton of friends. Yeah. The, uh, easy to use programming language for building apps.

SPEAKER_00: And so, so I love Apple and I love the Apple folks, but, uh, to deploy ML onto an Apple platform, you have to use their point solution called core ML and core ML is not compatible with all the models.

SPEAKER_22: And so there's all this friction just to get onto an Apple device. Right. And so Apple devices are pretty common out there. And if, if that's hard, you just think about what it means for this wide spectrum of different things. And one of the challenges here, the, the fundamental, the incentive structure problem is that hardware makers like Apple, like, like many other hardware makers always want to build a solution for their hardware. And nobody's trying to build something that scales across everything. And so this is what we're focused on.

Chamath Palihapitiya: Hey everybody. Today I'm joined by Roots CEO, Dan Dorfman. Dan, welcome to the show. Thanks for having me, Jason.

SPEAKER_57: Tell everybody here in the audience, what is Roots and what makes it different than the other real estate investing platforms? I'm a complete neophyte.

SPEAKER_58: Roots is a REIT with a little twist. Sorry, I had to do it. We are the first real estate portfolio that we know of that builds wealth for both our investors and our residents. And we've created a unique win-win model that creates partners and not tenants. Am I as an investor, if I wanted to put

SPEAKER_61: money into this getting dividends, or am I just getting the growth of it? How does all that work?

SPEAKER_58: When you invest with us, you get to participate in two ways. One is through the distributions of profits generated at the company. And we pay those out quarterly over the last 12 months, that's equated to about a 6% cash on cash return to our investors, just in distributions. And then the other way everybody participates is each quarter, we reevaluate what's called our net asset value. And as that ticks up our unit price or our share price of our portfolio goes up as well. And that's how you would basically be able to sell your share at any point and liquidate your investment and move on to your next piece or leave it in and keep growing with us.

SPEAKER_61: Head to invest with roots.com slash twist to sign up and start investing today.

SPEAKER_64: That's invest with roots, no spaces, no dashes.com slash twist to sign up today.

SPEAKER_66: Because Nvidia has CUDA, right? That's their software for writing their machine learning apps. Apple has theirs. And these two things are good. Google has theirs. Tesla has theirs,

SPEAKER_00: like everybody builds their own thing. So if you go back in time,

SPEAKER_70: Um, why does everybody build their own things? Is it just because it didn't exist before or because

SPEAKER_08: it's customization is necessary to get the, you know, end result they want?

SPEAKER_00: Well, because they don't have a choice functionally. Right. And so it's super interesting. I mean, AI is so important to what we do, right? Nobody takes a step back and says, if AI is so important for the industry, why is all the AI software so bad? Right. And so you look at that.

SPEAKER_14: Is it a function of time? We just were so young in the game?

SPEAKER_00: Yeah, that's, that's, that's a big aspect of it. So I, the, the analogy I give to people is that AI is like an adolescent, like it's like a teenager, right? It's, it's, uh, it has some, it's very exciting. It's overconfident. It's got some wins under his belt. It sometimes rolls over his parents' car and causes a mess. Right. But what's happening right now is everybody just wants AI to grow up. Like people want to build AI into their products. They want to not mess with the AI infrastructure. They want to actually be able to deploy things and build AI enabled products. Right. And right now,

SPEAKER_01: if you, if you're one of the fan companies, for example, you can take a team of 50 people and brute force it. But if you're many other people that should be using AI in their applications, it's so much more difficult. And to your question, like, why is there build their stack? They don't have a choice. Like all of the technologies that exist today are built for particular piece of hardware, or they're built by a research team. The stuff is not production quality. And if you go,

SPEAKER_00: if you go back in time, I built a technology called LLVM, which is this fairly obscure compiler technology that then is probably on your phone today and on many of your laptops and in your consoles and things like this, that technology helped unify a generation of compute around CPUs in

SPEAKER_01: particular. And so the LLVM was great for hardware people because they could integrate with LLVM and

SPEAKER_00: then they got all the C++ and all the Swift and all the other languages and Rust and Julian and things like this for free. But machine learning doesn't have that. And so what modular is building is,

SPEAKER_03: it's building that thing that once you plug into it, you have a full AI stack for heart for hardware maker. That's a very powerful thing.

SPEAKER_66: Yeah, what is what's Nvidia's take on what you're doing? Are they supportive of what you're doing?

SPEAKER_08: Or do they feel like what you're doing, they're not supportive of because it's going to help, you know, people maybe port to other hardware platforms and maybe take away their dominance? Or

SPEAKER_79: do you get the sense that they care about their dominance at this point? I mean, they seem to have run away with it right now.

SPEAKER_01: Yeah, well, great question. So I mean, there's this narrative in the industry that we're here to hurt Nvidia or something. Nvidia is one of our most important partners. Right? And, and, and one of

SPEAKER_00: the things that I think people forget about is Nvidia is really invested in building some really crazy, exotic next generation products. Yeah. Right. And so what we're interested in doing is we're interested in expanding the developer ecosystem that can use those products. So we're on a very complimentary set of missions here, right? And so what we're doing is we're looking at saying, okay, well, this whole AI thing, it evolved rapidly. Again, it, it's very high potential,

SPEAKER_01: but it's all a mess. Like the people who do it, as you know, are wicked smart, some of the most brilliant people in the industry, but there's other good people too, that have good ideas. Right. And so if we expand out the developer community,

SPEAKER_84: if we 10 X, the number of people that can participate, think about the amount of innovation that can happen there. Think about the new use cases and applications.

SPEAKER_07: Yeah. Right now people don't actually know this, but a lot of what's happening in AI is limited to people who can code in, um, CUDA, CUDA. What is it? Uh, CUDA. Yep. And then I guess some people write in C sharp or C plus plus, what would the other ways people generally get AI code, you know, down the hardware stack? Cause you're, you're building Mojo I know, which is, you know, more Python like, I think. Yeah. Well, we'll talk about that.

SPEAKER_00: Um, so it really, it really varies. And again, AI is not one thing. This is another thing that I think people get sometimes distracted by, but it's not like transformers are one thing, for example. And so if you look at a lot of model or, uh, like stable diffusion, which is a unit model,

SPEAKER_01: which is a very different architecture, what you get is a lot of Python on the outside.

SPEAKER_00: The Python hat handles what's called tokenization of converting input text into something model can understand. You then get something like PyTorch or TensorFlow involved, which is itself a a gigantic complicated thing that is awesome in some ways, but also challenging in other ways. You get custom CUDA kernels, as you're saying, so you want to get high performance out of one

SPEAKER_01: accelerator. And so when you get C plus plus, because sometimes Python is really slow. And so what ends up happening is, is a developer building one of these next generation models,

SPEAKER_00: you have to know all of these different things. And so practically speaking, no, no, no sane humans actually can do that. And so this is why you need teams of experts. And these teams are super experts

SPEAKER_01: in every single different one of these parts of the problem where somebody knows model architecture and differential equations. Somebody knows Kudo, somebody knows C++, somebody knows all these things. And so only that is what's able to bring these things together, which we've seen this movie

SPEAKER_07: before in the early days of the web, setting up a web server itself, getting a sun microsystems, you know, server, you know, it wasn't like today, obviously. And remember when we had apps come out, even pre iPhone, if you were trying to build something for Nokia, or Docomo, or any of these other platforms around the world, it was really hard. And there was a limited number of people who could do it, which meant, you just didn't see a lot of apps, they would come very slowly, a couple of apps a year. They were super interesting. But then once

SPEAKER_32: and they're expensive too, right? Because the development costs are so high.

SPEAKER_08: Yeah, which means something that's fun or interesting, like the idea that there would be an app for skiers, like I have an app on my phone for skiers called slopes. There's like, probably a half dozen of the fact that there's a solo developer or two person development team on their weekend hustle building an app. It's just a crazy thought. I mean, you were at Apple

SPEAKER_07: when this happened, the concept that an app could be made by one person in their spare time and get to a million dollars in revenue or even 100,000 revenue 10,000. It was just, there were so many hurdles to that you had to actually do deals with the carriers, you had to put up servers yourself, you had to figure out how to get distribution on. Yeah, getting the app, the distribution on people's phone was a roadblock. You just think about the genius of Steve Jobs, the app server distribution, that the payment rails for people buying it, and then the you know, there's really lightweight, easy app discovery and the ability to write them. So you're working on Mojo, this is a programming

SPEAKER_00: language. Well, just just before we move on from Apple, right? So my job at Apple was to lead the developer tools. Right? I mean, I had many hats. But by the time I left, I was running the developer tools team with Xcode, the whole iOS app development ecosystem, built the Swift programming language,

SPEAKER_01: also supported all of the internal hardware, which Apple has very fancy, very exotic, and next-gen hardware that they're building. And a major part of the job is to make people more productive,

SPEAKER_00: make it so more people can participate, exactly as you're saying, because so many people have good ideas for apps. Right? And so if you get more people involved, like the move from Objective-C to Swift, massively simplified things, made it much easier to learn. That was a huge movement that then enabled entirely new categories. And so many people today tell me, you know, I was able to become a programmer because of Swift, right? And so ML, I believe, has got exactly the same thing going on, right? Where it's absolutely possible for the most advanced teams to achieve things, right?

SPEAKER_01: But first of all, like complexity, which is really our enemy here, complexity, like if you fill your head with accidental complexity, you don't have space for other stuff. And so by relieving the accidental complexity, you make the teams of experts even more productive. But then you're also more

SPEAKER_109: inclusive to other people that have good ideas, but either are, you know, repelled by the complexity.

SPEAKER_08: What are the strategies for getting rid of complexity? I mean, I'm just thinking about playing chess, you kind of learn some heuristics, you know, some basic sets of moods, chunks of moves, that you can apply in different places. Or, you know, we have co pilots, which, you know, and we have

SPEAKER_07: open source, we have a lot of different ways to help people with complexity. But when you look at complexity in the world, what do you think of do you have a playbook for reducing complexity?

SPEAKER_00: Yeah, absolutely. So, so, and this is one way that modular is very different than pretty much everybody else in space, but complexity comes through abstraction, or reduction of complexity

SPEAKER_01: comes through abstraction, and through getting people to be able to work together. Okay, and so

SPEAKER_00: the idea here is that you look at all the domains of people that are involved, including all the people putting together the transistors on the chip, right? There's so many different specialities, the details can't fit in any one head. So success comes from teams of people, right? And then composing

SPEAKER_01: on other people's work. And so a lot of what I think software has been successful, I mean, you've built some pretty epic systems, right? Yep. It comes from being able to take things that other people built that you don't have to understand, and then build new things on top of it, right? And so what a lot of folks are doing today, and ml systems, and ml ops, and a lot of these things, they say, Okay, well, there's so much complexity out here. What are we going to do? Well, we're going to throw a layer of Python on top of the stack. And then you'll deal with our layer and look, look how simple it is. Therefore, you don't need to know about any of this complexity. Now, there have been dozens or hundreds of attempts at this, I mean, there's a lot of stuff out there, some of it's really good. But, but the challenge with that is, if you're building on top of something like TensorFlow, or PyTorch, or, you know, you're trying to get onto novel kinds of hardware,

SPEAKER_00: or like a TPU or something like that. Well, you actually get exposed to all this accidental complexity, because it all leaks. And so yeah, you get this cool demo. But you can't fix performance,

SPEAKER_01: or scalability, or programmability, or security, or, like, these core problems that people struggle with, by adding a layer of Python on top of systems that are fundamentally broken.

SPEAKER_07: Yeah, the facade doesn't work. And in a way, what we've seen happen in the modern web, over time, you have cloud computing abstracting away, putting up servers, and that been then storage got abstracted, I mean, GPS got abstracted away, there's a software development kit and SDK for anything, there's an API for anything. And then even building glue between

SPEAKER_79: systems has gotten easier used to call it middleware, I guess, back in the day, I don't know if

SPEAKER_124: there's still a term for that, but enterprise Java beans. Yeah, it was like weird stuff to try to

SPEAKER_79: get you to move data from one system to another, it seems like comical. Now, maybe just talk about

SPEAKER_07: the complexity in the world, writ large, and the technology stack, because you've been at this for

SPEAKER_08: a couple of decades, it is pretty amazing. When somebody's coming in now, a 20 year old developer in school, who is like building stuff, how much do they know about what's actually going on beneath? You know, you see the little tip of the iceberg, what did they are they even aware of like, the complexity underneath?

SPEAKER_00: Yeah, well, so I mean, again, it's hard to make generalizations about all 20 year olds. Yeah, because there's some variance there. But on the average 20 year old, they know Python. Yeah, they know. If you go into computer science, you know how to train a neural network, for example, but you don't know how to deploy it, right? You get exposed to some other programming, maybe you'll get a little bit of C++ or something like that. But most of most people coming out of a computer science degree, no Python, and pretty much everybody that is not designed to be a computer scientist. So there's a lot of other fields out

SPEAKER_01: there. No Python. Right. And so Python is great, because it's super high abstractions, like the ultimate duct tape language, where you can bolt together these very powerful libraries. But Python also has certain challenges when it comes to performance or dealing with hardware or a lot of the things that inhabit the AI space. And so running Python on a service with a billion users is not always great, right? And so there are challenges there. And so when you come back to what what is modular doing about this, or tackling instead of adding layers of layers of Python on top of existing systems, we're saying, let's go explode those systems. Let's do the hard thing,

SPEAKER_109: let's go build the system from the bottom up. And this starts at the hardware, right, the hardware, there's a lot of really good hardware out there. To your point, nobody knows how it works. I mean, the people that built it do but, but most application developers don't know how it works. And what has

SPEAKER_01: happened is that right on top of the hardware, there's all these different layers of effectively middleware, just like you said, right, but each piece of hardware has a different layer of middleware. And so that means that when you get to the top layer, the part that anybody actually wants to work on is super fragmented. And it makes sense. It's the inside structure, the people building hardware, they want to build a thing for themselves. But the losers are all of us trying to get our jobs done. And many people in ML don't want to care about the hardware, they're made to care

SPEAKER_136: about them. You've heard me talk about Supergut a bunch. This has been a key part of my health journey. It's an awesome nutrition company that my bestie, David Friedberg from the All On Podcast started. I love their bars. I love their shakes, especially the gut balancing chocolate brownie bar. It is delicious. They also have an unflavored prebiotic mix you can add to anything. I like to put

SPEAKER_57: it in my coffee, you can put it in your own mail. Their products are super helpful for weight loss. Why? Well, Supergut's products mimic the effects of Ozempic by boosting your GLP-1 hormone. This

SPEAKER_136: helps quell hunger and boost your metabolism, which is a great, great combination, obviously. And Supergut's prebiotic fiber, that actually alleviates digestive issues. And obviously, the products all taste great. The best part, the team at Supergut actually put the work in and scientifically prove their products work. They conducted a placebo-controlled clinical trial with Stanford last year. That's been published in the medical journal, Diabetes, Obesity and Metabolism. The results were amazing. The participants in the study, they lost weight, they lowered their blood sugar, they improved their metabolic health, and they had improved digestion and so much more. Whether you want to improve your gut health, maybe drop a few pounds like I did, or just feel better throughout the day. And listen, you're busy, you're traveling, I like to bring Supergut with me. Go to Supergut.com and use the code TWIST, you get 25% off. Go to Supergut.com and use the code TWIST to get 25% off. I've been on this health journey, I've lost 40 pounds. A big part of that, sincerely, was me using Supergut. So, go to Supergut.com and use the code TWIST for 25% off. What is this hardware going to look like in

SPEAKER_07: five or 10 years? Because we're at this point in time where what OpenAI did with, I think, 3.5 really kind of captured people's imagination and, you know, being able to actually play with it, inspired a lot of developers to maybe get in there. Um, and so here we are, everybody buying up sovereign wealth funds, you know, governments, countries, uh, you know, individuals, companies, startups, everybody buying up all this hardware, racking it, data centers. And, um, it seems to me, um, having watched this happen with fiber, uh, you know, we overbuilt fiber massively, and then all the fiber companies, WorldCom, et cetera, uh, there were a ton of these, uh, went bankrupt. They became worth literally 98, 99% less than they were when they went public and all that wound up getting bought by Google and other people at auctions. Are we in a similar moment right now where we're building up massive capacity? Or do you think there's enough jobs here to actually use this hardware? And then

SPEAKER_13: the second part of the question, so there's something about like this moment in time, uh, where does this all wind up? If we're sitting here five years from today, are we looking and going, Hey, wow, there's somebody just leapfrogged Nvidia, or there's three choices. You can go just like you

SPEAKER_91: do Android or you can do an iPhone, or you can, you know, pick AWS, Azure, or Google, or Rackspace,

SPEAKER_01: or right on down the line. Yeah. Well, so great question. So there are really two different questions. Two different questions. One question there is the today problem, right? And today everybody's talking about Nvidia and the stockouts of Nvidia and wouldn't be great if there are other options. It's super funny because the majority of spend by many, by many metrics is actually on the inference side, which is still very dominated by CPUs. Yeah. And again, like we, we talk about

SPEAKER_00: the pain point. Well, the pain point is people try to build these massive systems and there are not enough GPUs to go around. But meanwhile, so much AI is in our life. That's all being served in cloud.

SPEAKER_84: A lot of that's happened. I mean, some of us on GPUs and cloud, but a lot of that's on CPUs,

SPEAKER_126: right? And it works totally fine. It works totally fine. If you're on Amazon and it's showing you

SPEAKER_07: some additional products, like the one you're looking at in all likelihood, that is in machine learning job that's being done on a CPU that was written five years ago or 10 years ago.

SPEAKER_163: Or if you do Google search query, there's dozens of models all talking and doing weird things. And

SPEAKER_00: there's this an intricate dance, right? And so, and so it's really interesting. If you look at that, like the, your question about, is there going to be an oversupply and overabundance? I have no way to know, right? My goal is increased consumption by creating new categories, right? And so, and it has

SPEAKER_01: nothing to do with the H100 or NVIDIA. It's just about AI and the applications of it are like a good thing. It makes people's worlds better. And so if we can increase the number of cool things and make our lives better, that, that seems good to me. Now your question about where do we go from here, right? So forget about, forget about cloud for a second. Like, so I, I've been working in the hardware software boundary for, for decades now. And the thing that when I zoom out and I look at, look at this time, it's been super interesting. You know, people talk about Moore's law ended, you know, whatever, and what, what is Moore's law? Well, different nerds will argue pedantically what

SPEAKER_00: that means, but it really means, you know, back in the day, we'd give a new laptop and every year

SPEAKER_167: it would be, you know, 18 months to be twice as fast. Your Pentium chip was twice as fast.

SPEAKER_00: Absolutely. On the same code. Right. And so, and so what ended up happening, I don't know,

SPEAKER_01: 10 years ago ish is we had multi core CPUs. Ah, we have more than one of these to deal with. And then we had GPUs come on the scene. Yep. Right. You, you look to now we have massive GPUs. We have really dedicated AI chips, like the Google TPU and Gaudi from Intel and like all these things, there's tons of these things. Um, and we still have CPUs, but these days CPUs have like a hundred cores on them. Right. And so to me, again, many people are laser focused on the today problem. Yeah. But what happens when you look out five years or 10 years? Yep. Right. And to me, I look at,

SPEAKER_109: this is driven by physics. This is not a question about software or things like this. Physics is

SPEAKER_01: forcing hardware to get weird and more importantly, specialized in the rise of wearables, the rise of personal computing, the rise of all like AR VR, like all these things are a straight line towards very customized chips. And so that's very interesting. Yeah. Yeah. And so we're going to have all, I mean, we're going to have even more crazy hardware in five years than we do today. And this is where you start to say, like, how can we scale the software? Right. Nobody's going to be able to

SPEAKER_96: rewrite everything for every new generation of hardware that, that doesn't work. And this is, this is why we're focused on solving this problem. What do you think of the open source,

SPEAKER_07: um, risk five and, you know, AMD licensing models and then hardware being built by other folks, obviously Nvidia outsources their hardware in terms of how it's being, and they're a designer as well, but it's proprietary and it's closed. Um, so is what happened with Python and other open source and, you know, everything we've seen in the open source community is, is that likely to happen with

SPEAKER_08: hardware? Um, or is that, um, you know, great question. So immediately before module, I worked

SPEAKER_01: at a company called sci five and they are the inventors of risk five. Yeah. Risk five is an open source instruction set. And so what risk five allows you to do is it allows any hardware maker to create a member of the risk five family. And what that means most importantly is you get software. And, and so that is huge. Traditionally, you'd have, for example, arm owns the arm instruction set and only arm and its licensees can build arm compatible chips. Yeah. Or x86, you can have Intel and AMD and they're the only ones allowed to build x86 chips. And so with risk five, it allows you to go build our arbitrary people can invent new things and play there. And I think that this, this is causing an explosion of innovation. And again, the challenge with that, and the good thing about that is you get an explosion of innovation, the challenge of that is you get all this crazy hardware, right? And so there's no software. And so you need software that can scale on to all this

SPEAKER_00: innovation. And so that's really where kind of the, the industry's little loggerheads in. Yeah. And so

SPEAKER_66: AMD and these folks, they, they have blueprints, but they own those blueprints. They're, they're,

SPEAKER_07: they're patents. You can't just take them and build a house with them if we're just using an analogy here, but if you take the risk, uh, five, uh, do they call it risk five or risk V?

SPEAKER_109: It's risk five. The nerdy on that is that there's four things before it. So yeah, yeah,

SPEAKER_07: no, I, I kind of got that. I would, I've heard somebody say risk V and I'm like, are you sure it's risk V or it sounds like risk five? It's definitely five. It's, it's basically caught up to arm,

SPEAKER_08: I think in terms of throughput or it's close enough. Um, well, so, so, so with any of these things,

SPEAKER_01: it completely depends on what you measure, right? There's advantages to arm. There's advantages to risk five. It's all super nuanced. And a lot of people want to make overly simplified. Does this, is this thing better than this? And in tech, it's never really that simple. And so arm has got a very strong position. They certainly have some challenges. Um, they, they, they've got to stay on their toes. Um, but, uh, but really the innovation is the piece that I care about. Right. And, and I want to make it so that once these people invent really cool risk five based silicon or arm based silicon or whatever, right, that they can actually do something about that, right?

SPEAKER_109: Because having cool hardware that nobody uses is really kind of a problem right now.

SPEAKER_57: All right. Listen, when you're selling to business to business buyers, you really want to get your pitch in front of decision makers. Why? Because upper level execs are usually the ones making purchasing decisions. Duh. The problem is high level folks can be really hard to find and target on most social media platforms. But on LinkedIn, oh my God, they know all of the CTOs, all of the CFOs, all of the VPs of finance, engineering, HR, recruiting, all those titles are sitting there waiting for you. And now let's just talk about the funnel. LinkedIn is about to hit a billion members. Did you know that 950 million members at this point in time, there are 180 million of those 950 who are senior level execs, there are 10 million C level executives in that 180 million senior level execs, which are part of the 950 million members, I am a C level executive, I am on LinkedIn all day long, because LinkedIn equals business business equals LinkedIn, and LinkedIn ads are built specifically for B2B marketers. LinkedIn generates two to five times higher return on ad spend than other social media platforms. LinkedIn equals business business equals LinkedIn, when people are on LinkedIn, they're ready to do business. It's that simple. So make business to business marketing everything it can be and get $100 credit on your next campaign. For me, your boy J cow, I'm sending you the hundy linkedin.com slash this week in startups to claim your credit. That's linkedin.com slash this week in startups terms and conditions apply because they've given you the hundy. Tell me how Nvidia got here to a certain extent.

SPEAKER_07: Yeah. Um, because I think we watched this happen where nerds were playing call of duty and they wanted their frame rates to, you know, it doesn't even matter. It's, it's beyond the just noticeable

SPEAKER_79: perception in biology. You, you can't even tell the difference between 120 frames or 100, 240. It doesn't even matter, but these lunatics wanted the best. And I guess Nvidia just kept giving them better and better

SPEAKER_07: hardware. And then you had this crazy crypto moment where everybody started buying all this hardware from Nvidia to, to run jobs. And now AI is kind of circuitous route. I think, um, maybe you could explain why that's brilliant and then what the limitations of it are. Cause again, it's not always one thing, but

SPEAKER_13: I think the history of how they got here is kind of important or is it not?

SPEAKER_131: It's totally important. Um, and I mean, to your audience of people who care about startups,

SPEAKER_01: it's super illustrative, right? Because Nvidia didn't magically step onto success. It was earned, right? It wasn't, it wasn't an accident. And so if you go back, I'm not a super expert in video

SPEAKER_00: history, but my understanding is it's a combination of two really important things. So Nvidia, like some of the other companies you're a fan of, uh, goes, went through several phases where they made bet the farm bets, had near death experiences, and then we're right. And so one of those bets was on

SPEAKER_01: programmability. And so a lot of people were building the call of duty accelerator and there's a bunch of competition and just make games go faster, just make games go faster, just make games go faster. And Jensen and team, but I think it was the Geforce three on saying, okay, well, hard coding for graphics is not enough. Let's make it so you could do more general compute on this hardware. And so it's not going to be like a CPU. It's a different thing. It's a different category they created,

SPEAKER_00: but let's do this. And that was a huge bet and a non-obvious bet. Nobody else made that bet back then. Um, almost drove them out of business through the complexity of executing on that. Um, but what it meant is that new kinds of things could run on the graphics card

SPEAKER_01: and that created new markets. And so one of the things you're pointing out is crypto, right? Well, they didn't design a crypto accelerator. Crypto wandered up and said, I need tremendous amounts of compute. And they were there and ready to serve it. And because they had programmability, they're able to scale into the opportunity. You know, they talk about luck, right? Well, how do you, how do you get lucky? Well, part of it is being ready to take advantage of the luck that presents itself. And I think that is really what, what happened to them. Um, if you, if you look at, um, machine learning, right, a lot of people go back to the seminal

SPEAKER_00: moment in machine learning called the Alex net moment. And it's fine. And Alex was when Fei-Fei Li's team at Stanford created this big dataset called image net, and they created a

SPEAKER_01: competition around it. And that competition was to go find the most accurate predictor and identifier for what was in an image. And so for, for a few years, people were working on this using traditional machine learning techniques. And then these folks invented this deep drone network called Alex net, but then solved image net. I mean, not solved it, but Matt made massively forward in terms of prediction. Now, um, the way that story is usually told is that it's a combination of two different things. It's a combination of having a huge amount of data, but then also having GPU compute. And so we need both data and compute to be able to solve that problem and make that massively forward, which then catalyzed so much of deep learning today. But the thing they forget is that nobody had the convolution kernels, the algorithms to implement resonance that didn't exist on a GPU back. So the reason Alex net happened is a combination of three things. Actually, it's a combination of data, the amount of compute that was available, and then the bet that Jensen and his team made on programmability to allow some researchers to go invent some new algorithms and then do it on their platform. And then fast forward a few years, it turns out, yeah, they were lucky that deep learning caught on and turned out to be pretty economically important, but that's what put them in the position that caused all these things like TensorFlow and PyTorch and things like that to get built on their platform. And that's how CUDA got entrenched into so much of machine learning today. Yeah, so this is the journey of Nvidia. I mean, you can you can like play this back across so many startups, right? Are you creating a new category? Are you are you leaning into the obvious thing everybody's talking about today? Are you seeing around the corner and betting on where technology is going, right? There's so many of these questions that I think that, you know, there's no one right answer, but it really plays into a lot of the journey. Well, I'm getting to your point about what you're

SPEAKER_79: doing. Mojo. If you enable more people, the street finds its own use for technology.

SPEAKER_07: Exactly. And Gibson quote, like you say, Hey, listen, you want to do something that you want to try to identify an image and figure out if it's a hot dog or not. Sure. Use our GPUs. You don't need our permission because it's permissionless. I mean, not crypto permissionless, but you know, it's your hardware, you own it, do what you want with it. And it is one of the great, great things about whether it's open source or just open platforms in general, people building platforms. Yep. Um, and so when

SPEAKER_08: you look at this, uh, from a playing field, having been an apple, uh, and watched what happened with open platforms and apps, where do you fall on the call it the AI rapper debate of 2023? Oh, this company,

SPEAKER_13: we have a great company roam around. They let you type, they're, they're building a vertical itinerary, travel itinerary piece of software. And so you can go to chat GPT and say, Hey, where should I go in San Diego with my kids or, you know, roam around building it and they've got very narrow data set

SPEAKER_219: and they're, they're really tweaking it around travel. So you, you have all these verticalized

SPEAKER_13: ones. I have, uh, we invested in a verticalized screenplay writing software. So we're writer. It is, it's kind of like final draft just for that. Yep. And my belief is like, yeah, there'll be a lot of these vertical things because you have the interface and you have all the kind of features that will go around it. And sure, chat GPT could do a version of it, but it's not going to do like a polished version of it. So the AI rapper derogatory statement towards startups building verticalized AI apps versus one giant language model quad or magically solves all the problems magically solves every problem on the planet. Is that even possible? Or where do you think this

SPEAKER_81: all winds up? Well, so, I mean, I think that there's many different angles in terms of what is the better

SPEAKER_01: product, what captures the most value in terms of investment hypothesis, like what is the ROI on these things, right? So when I look at this as saying, um, I, I'm not a believer in a one size

SPEAKER_00: fits all solution. I mean, maybe theoretically AGI someday will come. And until then I will hold, hold onto that thought. But, um, in the absence of AGI, which magically solves all problems, I look at AI as being a solution to certain kinds of problems, right? And some people, some of my friends even, uh, want to say that AI is better than software, you know, and it's just

SPEAKER_228: like a straight replacement, but that's in my opinion, objectively false.

SPEAKER_65: What they mean by that is just having a chat interface with an AI agent and talking to them,

SPEAKER_07: you'll solve more problems than having to write software. It'll just do whatever the task is. Or you're saying in terms of writing software. Well, I mean, you, you know, Jason, like, you know,

SPEAKER_01: that building a product is way more than like having an algorithm, right? It's about building a relationship with the customer. It's about having user interface. It's about having a revenue model. It's about having a brand. It's having like all of these things, right? And so when I look at that,

SPEAKER_109: when I look at one of these verticals, so you talk about the copywriting thing or, or these things,

SPEAKER_01: these are clearly valuable products. AI is clearly a valuable way to implement these products, and it can be differentiation within that category. I don't think that makes that product magical. I think that that makes it comparable to other things in that vertical. And so AI is a much more efficient and smart and, uh, product focused way of building out that

SPEAKER_00: technology. But I would look at that as saying as an implementation detail of building into that vertical. And I think that has a huge amount of value. And so like, if you're looking as an investment hypothesis, I would not value that as an AI company per se, I would value it as

SPEAKER_01: a vertical consumer vertical, whatever it is, company, and, and now they're doing it in a smart

SPEAKER_79: way using the best tech they have available. Yeah, just like, there's going to be some, you know, the Yelp out the Yelp app is so much better than using the website, right? And they just use

SPEAKER_07: that new, uh, technology to, to make it a better experience. Everybody's also looking at the David

SPEAKER_01: versus Goliath thing, right? And so everybody wants the little guys to take down the big guys,

SPEAKER_22: but the big guys have all these other things going for them, including distribution, many of these

SPEAKER_10: other things. Well, listen, you're on the inside of all this. I gotta ask you, what is the inside

SPEAKER_07: track amongst people of your peers who are deep in the AI game and have been in it for a long time? What's your take on what's what open AI did this open source, you know, or, you know, open it's in the name, uh, and that, Hey, we're gonna, we're gonna, this is too important. This technology is way too important for any major company to have a wrap on it. It really, the world needs us to go out there and really make sure that it's not just deep mind inside of buried in some Google, you know, uh, corridor and some building on a campus. Sure. We're gonna build this. And then they got to 3.5 and they're like, whatever, 3. And they're like, you know what? We were wrong. Uh, I don't know if they ever said that, but this is way too powerful. We're gonna be closed AI. Well, do people look at that as just a money grab as cynicism or as sincere, but how does the industry, and I'm not

SPEAKER_13: saying necessarily you, but do people look at that and go, it's a money grab. They went from a nonprofit to a for-profit. That's all it is. You know, the, the people there wanna make money, which is fine. We all do. You're racing venture capital. It's that doesn't come without expectations. So what,

SPEAKER_11: what's the, what's the take on that crazy move to go from a nonprofit to a for-profit from a open

SPEAKER_37: system to a closed system? Honestly, this isn't my area of specialization. I mean,

SPEAKER_81: I'd much rather. I'm just curious. What do you think about this weirdness? My, my opinion is, what do you expect? They took VC money to get return. Yeah. The end. Yeah. Well, I mean, it's and, and so, I mean, I think that things that appear too good to be true sometimes are,

SPEAKER_01: right. And so if you're expecting, if you're expecting somebody out of the goodness of their heart to dump billions of dollars of compute into building a free product, then well, you're paying for it somehow. Maybe it's with your data. Maybe it's some other way. I mean, I think this is generally true in the world and I think people are getting smarter about that. And so, I mean, I, I don't know. I mean, I think the surprise is surprising, but, um, I don't, I don't know too much

SPEAKER_212: about the details on how they decided to do that or what it means. Yeah. There's trade-offs

SPEAKER_07: everywhere. Uh, you look at the impact on society. I am, you know, I'm an investor in a lot of companies and what I'm seeing on the frontline of startups and inside really nimble organizations that are the tip of the spear in terms of using technology, not just to build the product,

SPEAKER_110: but to build their businesses. They're building 12 person businesses with four people. Yeah. They are

SPEAKER_08: getting a lot done with less and it happened boom in one year. This is year one. I mean,

SPEAKER_07: I, people still forget that it was last fall that 3.5 came out and kind of blew people's minds, let alone 4.0 and whatever else coming next. So when you look at the impact on the world,

SPEAKER_11: knowing what, you know, from the seat you're in, um, is what we saw this year, which is to say, I think people got 30 or 40% more efficient at their jobs. If they know how to use this technology easily, is that going to compound or is it going to be the same and then impact on society?

SPEAKER_234: Yeah. Well, so I don't know, I don't know the math on that, but, um, the, the impact is going to be

SPEAKER_01: huge, right? But the huge impact is also going to be, um, spread out over time, right? The impact, as you say, you have seen it, but you're zeroed into a very specific part of the problem. We still can't hire programmers. There's not enough programmers out there to implement all the stuff that needs to be implemented. And so while it is true, it's impacting part of the ecosystem, it turns out that, um, there's a big part that it isn't. One of my questions is that when you have disruptive technology, how do you think about technology diffusion? How long does it take something that should be disruptive and everybody knows is a 10 X improvement or whatever? How, how long does it take to like actually get out into the ecosystem? Because sure, the neural network algorithms change every week, but we humans don't like we, it takes a long time for us to learn new habits and it takes time for all the planning cycle and things like this to change. Um, one of the things I think people forget is that as a coder, people focus on, okay, I'm going to study up. I'm going to put the semicolons in the right place. And you know, I've worked on programming languages forever. Um, but so much of coding is working as part of a team, right? And so the way I look at this is I look at it as saying, okay, imagine you had the amazingly awesome coder robot, right? And we're not amazingly awesome yet. We're promising, but we're not amazingly awesome yet. You still, that's like adding a member to your team, right? And so adding one member to a four person team is huge, particularly if they're really good, but you still need to review the code. You still need to integrate in the product. You still need to decide your product strategy. You have to understand the relationship with the customer. You have to, so you're, you're, you're improving one really important part of the problem. You still have to do all the other work, right? Yeah. Now chat GPT and things like this can help with some of that. They can help with graphic design and like AI is good, good at many different pieces. Right. But I think that it will take time for us all to figure out how

SPEAKER_109: best to utilize this. And is it human accretive or is it disruptive or how does that work out over time?

SPEAKER_164: But how much faster are developers getting in your estimation, like with these co-pilots and it feels like they're getting 10, 20, 30% faster year over year. Um, I, I don't, I don't know if it's,

SPEAKER_265: if it's cumulative is the problem, right? So, because what I've seen is sort of what I was getting at

SPEAKER_01: is like, I've seen a lot of boilerplate get automated. I haven't seen a lot of the actually

SPEAKER_270: interesting part of product design get automated. Huh? Fascinating. Yeah. So that's where the human

SPEAKER_00: creativity will be. Yep. Yeah. And so this, this is where like, yeah, if you take, I don't go back in the day, XML or something like, if you take something super boilerplate, then AI animation's

SPEAKER_109: amazing. Right. But there are also other better ways to do that, you know? So that's, that's a

SPEAKER_271: different way to look at the question. We also have a little bit of a corollary for this. So, you know,

SPEAKER_07: I look back on my career and it's like, it was two decades before everybody got a PC on their desk

SPEAKER_70: and in their home. It was literally from like 1980 to 2000. By the time you got to 2000, the idea that

SPEAKER_13: somebody didn't have a computer at work was like, really? I mean, you'd have to look really hard in an organization in 2000 to find somebody with a desk without a desktop computer on it or, or cell

SPEAKER_01: phones. Right. And then you look at cell phones, two decades disruptive technology. Diffusion takes time right now. I think this may go much faster than hardware transitions did because the inherent

SPEAKER_96: time delays and manufacturing and stuff like that is much lower, but it'll be similar. Well, I mean,

SPEAKER_79: now we, I think that's, you just nailed the point, which is then you look at something like Google,

SPEAKER_13: Uber, or, you know, some other software based platforms that don't require, you know, hardware that are built on top of them. Those things all took 10 years. So I think maybe this next group is,

SPEAKER_79: you know, maybe we go from 20 years to deploy 10 years to deploy and hit the masses. And maybe now

SPEAKER_81: it's three, four, five. Well, as you look at startups, right? I mean, I think that I've seen so many

SPEAKER_01: of these, I'm sure you've seen probably a hundred X more, but so many of these folks are like, look, I built a thing. It's, it's a thin layer on top of chat GPT. I hacked it together a month. I'm going to make mass amounts of money and it's going to be amazing. Right. Yeah. In my experience,

SPEAKER_163: which is obviously small selection size, but, um, if you can build something in a month,

SPEAKER_283: so can everybody else. Yeah. There's no moat by the way. Exactly. And so if it works,

SPEAKER_163: then everybody's going to be after you. Right. And so, and so that's one of the challenges. And for me,

SPEAKER_01: this is where I, the, the things I work on can take years. Right. And so what I do is I say, okay, well, this is going to be a 10 or 15 or 20 year journey. How do I break it down into milestones? How do I have usefully viable things that are maybe not the big win? Because everybody wants to jump to the end, but how do I make sure we're making progress in delivering useful value and in learning and iterating and cycling, building up to something that's really quite huge. And to me,

SPEAKER_287: that's, that's a lot more interesting. What's your next one? What's the next milestone? What's the way point that you're working towards? Yeah. So why don't we go back to modular?

SPEAKER_01: Because I don't think we've, we've talked much about products and where we are. Um, yeah. So,

SPEAKER_00: so modular, what we're doing is we're tackling all this complexity, right? This industry is a mess. We have all these people, all these companies, all this stuff happening. And it's, you know, just keeping track of it as a mess, but also you have all these infighting groups, like none of the LLM companies get along. No, the hardware people get along. No, the cloud people get along.

SPEAKER_01: Nobody gets along in the space. Right. And so as a consequence of that, all that complexity is being forced on us. And so modular is rebuilding this from the bottom up and providing a unified thing that simplifies this way for people. Mojo, which you brought up, is one of the major pieces of this. What Mojo is, is it's a programming language. Well, who in the right mind invents a new programming language? Well, I've been there,

SPEAKER_291: done that. I've, I've, I've built OpenCL. I've built the, the, one of the most widely used

SPEAKER_00: implementations of C++. I built the Swift programming language from scratch. Right. And so why do you do that? Well, you do that because you want to build and help and solve a problem that you can't solve

SPEAKER_01: any other way. Like building programming languages should never be in anybody's right mind. The first thing you jump to, but here's the problem we faced, which is that everybody in machine learning uses Python. People generally love it. Right. Python is, I mean, my, my kids know Python, right? Yeah. It's ubiquitous everything and people don't consider it to be broken, but then you run into AI where now you have high performance GPUs and you have crazy accelerators and you have all this kind of stuff going on and you have C++ and you realize that Python is really great at composing opaque things that other people made, but it doesn't give you the hack ability to actually go customize and change things. And so what Mojo does is Mojo says, okay, well, let's take this problem and let's do a very hard tech

SPEAKER_00: project of building a new programming language, inventing all new compilers and runtimes and very

SPEAKER_01: low level system stuff that allows Python to scale. Let's embrace Python and its entire ecosystem. Because what I've learned in my experience with this kind of stuff is that generally humans love to learn things. We all love to grow. We have like learning new techniques. We want to put new things in our toolbox. It's all great, but we hate resetting to zero so that we can then learn. Right. And so what Mojo allows you to do is if you know Python, you can walk right in the things you already know, continue to work. But now if you want to write some high performance code, you can do so. And not everything needs to be high performance. You can choose where you care about applying the time and that allows you to scale. And so a big part about what modular does is our number one mandate is meet the consumer where they are. Right. And guess what? A lot of developers are on Python. We love Python. We want to make it better. We're not trying to go like make a completely different system that has nothing to do with Python and hope it ends up being better. It's a different approach. Um, AI is what you're talking about. Huge mess, like all these different fighting systems. There's no thing to plug into. None of the stuff is compatible. So what modular provides is this thing called the AI engine. And the AI engine is a drop in compatible replacement for

SPEAKER_84: TensorFlow and PyTorch. And so if you're using PyTorch, if you're using TensorFlow, you do not have to

SPEAKER_22: rewrite your code. It turns out who wants to rewrite their code. Nobody stands up. Right. And so what we

SPEAKER_01: can do is we can be a drop in replacement that then provides a ton of value. And so for a lot of enterprises, it has value in terms of consolidating, eliminating all the point solutions. And so many people have a little bit of TensorFlow, a little bit of PyTorch. And so that's, that's huge for now they have a little bit of CPU, a little bit of GPU, they have a little bit of this, a little bit of that. They have different kinds of models and different kinds of specialized things. And we can consolidate that into one simple thing that turns out is commercially supported. Who wants to run their own mail server these days? Right? Like, do you want to build and run your own cobbled together storage thing? Right? Yeah, exactly. It doesn't make any sense. I mean, again, AI needs to grow up. It's programmable and extensible. Do you want to give up your product strategy to somebody else? Well, no, it turns out that people want to take models and then customize it. You want to make it work right for what you're doing. And so having the ability to hack the system is actually super important, right? It's extensible via hardware, right? And all these different pieces, the mojo and engine. How hard is it to make it compatible with each different

SPEAKER_191: hardware platform? How long does that take? And it's super hard, right? So, so I mean,

SPEAKER_01: I mean, if you want me to talk about my, my backstory, like I've been working on these super exotic, esoteric compilers and systems and GPUs and accelerators and things for decades, right? And so a lot of what brought modular to exist is this realization that if we keep building one-off solutions to each of these things, we as a software industry will never scale. Yeah. And so a lot of the core tech, a lot of the core invention at modular. And the reason that what we have is interesting is we enable people to bring up hardware much faster. And so, for example, we have just on CPU front, as an example, lots of people use Intel CPUs. They're really great. They're pervasive, pervasively available in the cloud, right? Turns out that PyTorch, for example, super optimized by Intel for Intel

SPEAKER_29: CPUs, right? Also turns out that you can get AMD CPUs in cloud. Turns out their instance types are

SPEAKER_01: usually much cheaper for the same amount of performance horsepower. But guess what? For some reason, it doesn't run super effectively on AMD CPUs. Oh wow, go figure. Go figure, right? And, and so turns out modular has massive performance uplifts on Intel, even bigger uplifts on AMD. But then you can also go to these other instance types like Graviton, which are ARM-based cloud servers, and they're even less expensive. And our performance uplifts are even bigger, right? And so what we can do is we can provide the ability to move your workload to the place that makes sense for, for your thing.

SPEAKER_29: And for us, bringing up Graviton, just in terms of bringing up an entire machine learning stack, took us four hours. Wow.

SPEAKER_01: For a completely new architecture. And that's one of the things that nobody in the industry, in the AI infrared industry has, is the ability to bring up the entire stack quickly and then do performance. Most of the time, the problem you have is that you have to do all this incremental work to get new kinds of models to run. And so that's one of the reasons why you get all this

SPEAKER_164: fragmentation. There's always being a translator. You have Apple decided they would get off Intel.

SPEAKER_79: They never are on AMD, but Windows was on both. And they started doing these M1 M2 chips. They're pretty extraordinary in terms of running a laptop or a desktop in terms of performance video. And of course, you know, battery life, they're optimized for what, you know, a very consumer bent,

SPEAKER_22: let's say, um, and that's, and that's the world I lived for years at Apple, right? Right, right.

SPEAKER_00: They're helping with hardware transitions, helping the watch get to 32 bit arm to 64 bit arm to all, all the complexity that goes into that, that Apple makes magic for developers. So nobody has to know

SPEAKER_316: about it. Yeah. And are they, do you think they're going to play a role here? Do you think their chips

SPEAKER_08: are so high performance that they're, they've got a shot at taking on some machine learning and,

SPEAKER_07: you know, AI jobs and sincerity, or is it just because I was just watching somebody, you know,

SPEAKER_79: putting Lambda, they were, you know, trying to build some models on their M2 and they were just like, wow, that's pretty extraordinary. Yeah. So, so, so what I've seen out there is that, um,

SPEAKER_00: so I've been out of Apple for a long time, so I don't speak Apple, I know nothing about the roadmap,

SPEAKER_319: and et cetera, et cetera, et cetera, et cetera. Disclaimer, disclaimer, disclaimer. You get, um,

SPEAKER_01: I don't think they're interested in the training market. Their hardware is completely irrelevant there in my opinion, and they're not even trying because they don't think it's an interesting market. It's not consumer aligned. It's very low margin compared to this. I mean, NVIDIA is accepted, I guess, but, um, but that's not their strong point. What they're really focusing on is the client. And so you look at it, there's all these llama.cpp and things like this, where people are running LLMs on their laptop, Apple's all over that. They're super into that. And it turns out that again, you look at the shift that we started from, there's this training part of the problem. What we've seen is this rise of pre-trained models. And so training a model is actually becoming actually less important over time, maybe at least the number of people that participate in that can go down. And, you know, if, if meta keeps launching like amazing models that they train themselves, right. That are good enough or great enough that you were on the

SPEAKER_79: inference side and yeah, running it on your desktop becomes super interesting. Right. And inference is

SPEAKER_84: the part that you integrate into your product. Right. And so that becomes the interesting thing

SPEAKER_311: is you want to run chat GPT on your phone. You don't want to train chat GPT unless you're crazy.

SPEAKER_79: Yeah. Right. Amazing. Well, listen, uh, great start. Uh, really excited to see where you take it. I know you're on a hiring binge right now. Uh, and you're really want to bring talent on board. Uh, yes, pitch to developers of why to come work on this problem and what, what are you looking for?

SPEAKER_01: And what's the culture like at module? Yeah. So, so what we're doing is we're taking on a really hard technology problem, right? So this is, this is a part of the problem and a layer of the stack that very few people understand. And honestly, it's things that people want to build on top of, instead of having to understand. Right. But now for the specific kinds of hardware, software, cloud folks that care about super scale, it turns out there's a lot of money being spent in the space. It turns out there's a great set of opportunities in front of us. It's, it's a really exciting time in, in, in the domain. One of the things that's really unusual about modular is that we don't run from demo to demo to demo to demo. We actually build high quality production stuff and we care about building things. Right. And what I found is that if you build things right and

SPEAKER_84: deliberately strategically, and you put down the bricks one after the other, you can build some pretty epic things. And you look at Mojo, for example, like we're building potentially the successor to Python. Amazing. Right. We love Python. Python's never going to go away, but this thing can take Python and give it superpowers. And as it does that, right, the opportunity to impact hundreds of millions of developers is profound. Right. And you look at AI, how many developers is AI can impact? Uncountable. All. 100%. I mean, and the fact that we might have,

SPEAKER_07: you know, a, a larger aperture of, uh, people who could participate in developing, right? Like

SPEAKER_79: exactly. It wasn't open to as many people. And now with these tools. Yeah, exactly. And so, and so

SPEAKER_01: modular, right. What we're doing is we're focusing on this layer of the stack that we think we contribute to. So we're not building the LLM. We want to help those people do that. We're not building the cloud. We're not building the hard work. We are helping solve this problem that we think is really useful for people and it will allow other people to build on the platform. And it's building this thing out. Our platform is opening as an open platform. We think we're going to be able to help lots and lots and lots of people, which is super fun. And you want to build it, right?

SPEAKER_79: So I, uh, was just looking at your careers page, go to modular.com slash careers. If you want to build important things and you want to build them, right? And enable a lot more people to participate in the future. Listen, you've been a great guest. Please come on again. Um, and continue success with

SPEAKER_340: it. Yeah, I'm a huge fan of your, Jason. So thank you for having me. Thank you. I appreciate that.

SPEAKER_341: All right, everybody. We'll see you next time on this week in startups.