SPEAKER_00: Hey everybody, welcome to This Week in Startups. This is Alex and today on the show we have a couple of amazing Twist 500 interviews for you and then at the end Jason is going to jump on and do a couple of founder questions. So first up, Gabe from Harvey AI. You have heard of this company. It's worth a lot of money. It's growing very quickly. I wanted to learn more about why the law is such a ripe place to apply AI today. Then we were talking to Alex from Open Router, an amazing startup. We have used their data on the show a number of times. I was really curious

SPEAKER_01: though, what's the business behind the hype? We learned that and more. And then as I said, at the end, Jason's going to pop on and answer a couple of questions. All right, let's go.

SPEAKER_04: This Week in Startups is brought to you by Lemon.io. Hire pre-vetted remote developers and get 15% off your first four weeks of developer time at Lemon.io slash Twist. Uber, bad data equals bad AI. Your AI is only as good as the data it learns from. Uber. That's right. Uber AI solutions now works with enterprises around the world to source, label, evaluate, and scale real world high quality data for every industry everywhere so that you can focus on building the next big thing. High quality data equals smarter and faster AI. Uber.com slash Twist. And Interpret. Interpret turns feedback noise into customer intelligence so your team knows exactly what to fix and build next. Head to interprets.com slash Twist to book a demo and see it in action. Hey, everybody. Welcome back to This Week in

SPEAKER_00: Startups. This is Alex, and we are joined today by an entrepreneur and a founder that you have heard of. The legal profession has become a hotspot for AI innovation. Lots of startups are raising money and growing very quickly. But at the absolute top of the heap, there's a firm called Harvey. You've heard of them. I've heard of them. I've long wanted to learn more. So please welcome to the show. It's Gabe Pereira, the co-founder and president of the company. Gabe, how you doing? Good. How are you? I am. I'm actually quite good. I'm a little bit shocked though that you're not suited up. I thought anyone who interacted with the legal field had to pretend like they just stepped off an episode of

SPEAKER_08: suits. This is what Winston and I thought when we started the company and really early on, we had a

SPEAKER_10: customer event and we bought suits and showed up and they were like, can you guys please just wear hoodies? And so that's why I'm in a hoodie. Before we jumped on, we were joking. We were going through

SPEAKER_13: the fundraising history of the company and you guys most recently raised $150 million from Andreessen, a billion dollar valuation. And I joked, you know, did you buy a boat? I didn't think you actually had. But you let me know that you have a unique sleeping arrangement right now.

SPEAKER_10: Yeah, my mattress is still on the floor. So we got to fix that at some point.

SPEAKER_13: You really don't actually. I mean, you seem fine. Company's doing great. So I don't see any issues there. All right. For folks out there who don't know what Harvey does, I think we should start with a little bit of kind of just brass tacks. So the way that I understand the company today is that it has two major components. One is a thing called Vault that allows law firms to bring their information, their data, case history, just all of their stuff to the platform. And then you have a tool called Assistant that uses that information and does a lot of work for people doing drafts,

SPEAKER_17: that sort of thing. Is that a fair summation of Harvey today?

SPEAKER_20: Yeah, I would think of where we're at today is I think kind of how Cursor built this IDE or didn't build the IDE, but kind of added all this AI. I think the big challenge for legal was there didn't

SPEAKER_21: exist this kind of single workspace where you could work on these. In legal, they're called client matters. And so a lot of what we've been building in the past three years is how do we build a single workspace where either a lawyer or an agent could perform a client matter, whether it's a litigation, an M&A, end to end. And I think the challenge with that is lawyers use kind of a wide variety of tools. You need case law, you need data rooms, you need e-discovery platforms. And so we're starting to partner with a bunch of these folks and then we've built some of this functionality. But I think the more or equally important piece is it's not just building the tools for individual lawyers. One of the massive challenge that these large law firms have is how do you manage all of these different client projects? And so if you're one of these large law firms, you're doing tens of thousands of client matters. You have millions of historical client matters and you need to make sure all that data is separate. And so there's a massive part of the product that I think doesn't get seen as much is all of the governance and admin controls you need. And then another thing we're working a lot on is a lot of this work is not just how do law firm lawyers collaborate with each other, but how do they collaborate with their clients? And so now a large number of our customers are clients of law firms and we're building a lot of functionality for these two parties to work

SPEAKER_13: together on client matters. So if I'm a law firm and my client is also a Harvey customer, I presume there's a kind of a bridge between the two that lets Harvey kind of talk across the sides, but let's say that I'm, I don't know, a private equity firm and my law firm doesn't use Harvey and I do, does that create added friction or does that still help the overall legal process? Yeah. I mean, I think what

SPEAKER_20: we're seeing, so when we sell to enterprise, I would think of there's two large things that our

SPEAKER_21: enterprise customers are using Harvey for. There is a bunch of internal legal work that these enterprises do that they typically don't collaborate with outside counsel. And so things like contracting and other internal legal processes. And so we're building products there. And then what we're seeing increasingly is when these large private equity firms or enterprises buy Harvey, they're starting to ask their law firms to also buy it so they can work together. And so I think we're starting

SPEAKER_27: to see a bit of this network effect from that. Really push your, your effective customer acquisition costs down or is this more like a, an edge benefit that hasn't changed the economics of sales?

SPEAKER_20: Yeah. Oh, I think we've seen this make a huge difference. Like there are, there are deals we

SPEAKER_21: get and we see this actually in both directions where we see law firms going to their clients and saying, Hey, we can work better with you if you buy Harvey. And then we have inbound from enterprises saying, Hey, we want to buy this so we can work with our law firm. And then we see it in the other direction where we've seen some large enterprises buy this and then reach out to their entire panel

SPEAKER_31: of law firms and say, Hey, can all of you purchase Harvey so we can collaborate? And so I think this

SPEAKER_13: is actually quite a big effect. That's fantastic. I mean, everyone wants distribution and in the world of selling to large companies, it's very hard to get in the door, let alone to get something close. But I presume that if your law firm's client is saying, please use this, that probably reduces the time to sale by a factor of 10, if not more just for you guys. Yeah, exactly. All right. Now I want to talk about the use of AI inside the legal profession, because we all know that it works well in coding because there's a huge corpus of online information. There's tons of code you can pull in and learn from. My thought was that the world of legal stuff is a good fit for AI because it has its own kind of grammar to it. It's structured in a particular way. There's a huge kind of corpus of case law and legal writings out there to pull from. Is that correct? And is there something that I'm missing that makes AI a good fit for AI?

SPEAKER_20: Yeah. Yeah. No, I think that's exactly right. Like, I think there's a ton of analogies between

SPEAKER_21: code and legal. And these feel like the two domains where you've really seen these types of companies be able to be very successful. And I think it's for the exact reason you said. And then I think the one thing I would add is a lot of the job of these associates is just synthesizing massive amounts of information. And so a lot of times when you get staffed on a litigation or a merger, if you're a junior associate, it's not just, can you understand all the legal context? It is, oh, I'm representing this pharmaceutical company. And I actually just don't know how pharmaceutical supply chain works. And I need to understand that and then understand how that connects to historical litigations. And so there's just this massive learning problem. And I think what we see now with things like Harvey is these associates can use web search to go understand the industry, and then they can use Lexis to do this research. And then they can connect it to here's a discovery corpus with all the emails, and then use all of this to start drafting like emotion or something like that. And it's so it's these really complex workflows across all these different data sources

SPEAKER_38: that you need to synthesize. And you guys recently announced some sort of deal with Lexis, right?

SPEAKER_21: Yeah, so we have a partnership with them for for case law. And so we're both we're working with them to kind of provide this in our platform and kind of expand this as well.

SPEAKER_13: The road, though, to applying AI to the legal profession started off pretty hot. There's a funny anecdote, I think it was on somewhere on open AI's own blog, how you guys used GPT three early on to do landlord tenant questions from its legal advice subreddit, and found that quoting for 86 out of 100 questions, attorneys you talked to said they would just send it on. But then later on, open AI also said that you guys also tried things like fine tuning foundation models and using RAG to try to bring more information into AI models, but found that that was kind of lacking. So it seems like there was like initial fit between existing AI technologies and legal questions. And then you ran into some complications along the way. So can you just kind of like chart for me how that progression has gone?

SPEAKER_44: If you're not listening to your customers and iterating based on their feedback, your startup will fail. So if you're drowning in feedback from support reviews calls, it's time for an interpret. Interpret is the customer intelligence platform that helps you turn random comments and feedback into actionable insights that help grow your company. Their AI system is already trained on your business and your product. It then reads all your support tickets. It reads all your reviews, call transcripts, and surveys. Then their agents actually update your team in Jira, Linear, Zendesk, and Slack. So this feedback doesn't just sit in some email or messaging thread, but it actually helps keep your team moving the ball forward. Find out why Hanva, Notion, and Perplexity are already using Interpret to stay on top of what they need to fix or build next. So if you're ready to turn feedback chaos into customer intelligence, head to interpret.com slash twist to book a demo and see it in action. That's E N T E R P R E T dot com slash twist.

SPEAKER_47: Yeah. So, so I would, I would think of the big difference there is consumer versus corporate law.

SPEAKER_21: And so when we started Harvey, Winston and I were really interested in kind of access to justice. And could we use this technology to provide legal services kind of more affordably, but for consumers and the models at the time, this was before GPT-4 were already quite good, kind of like you said, if you took a lot of these, Hey, I got evicted, here's my lease, what should I do? Like so much of this is online that the models were quite good. And I think the challenge is, there's a bunch of regulatory issues around unauthorized practice of law, things like that, that I think it's something now we're starting to work with, for example, the Supreme Court in Singapore. And so governments and court systems to see if we can apply this technology. But I think the core business now is more corporate law. And that's where the challenges are much more difficult because most of that data is not public, right? If you think of what these very large law firms are doing for corporations, it's, I have a massive litigation, I want to do a very large acquisition. Most of the data you're working there is very sensitive client data that actually never goes into the public models. And so a lot of that is missing. And then the thing that you mentioned we did with OpenAI that I think is still a really big challenge is we are trying to make these models better at legal research. And so case law is semi-public. And so the models have a lot of access to it. And I think what you see is their legal reasoning on things like Supreme Court cases is actually quite good. And my intuition is the reason for that is, it's not actually because the Supreme Court cases are public. It's because there's so much public analysis of all of this data online of all of these legal scholars talking about here's the implication of this case, here's how it relates to this case. And so a lot of what you see the models doing is being able to interpolate between that, and then it looks like this very powerful legal reasoning. And then the issue we run into is when

SPEAKER_48: you try to extrapolate that to here's the more nuanced cases that are not Supreme Court cases, but a law firm would be researching when they're doing kind of a specialized matter for a client,

SPEAKER_21: there isn't that writing and that work around it publicly. And so the reasoning model, the reasoning of kind of the general purpose models falls off. And so I think we have this big intuition where a lot of the legal expertise is locked up in these law firms. And what we need to do is help them train these systems with their expertise. So each law firm can kind of put that into their own system.

SPEAKER_13: So you guys have a history with OpenAI. They were an early investor in the company. And then I think it was in May of this year, you guys added models from Anthropic and Alphabet to the product, kind of giving yourself a multimodal approach to the market. I'm trying to understand exactly what clients need to do. Do they end up training like their own version of a general purpose model with their own data? Is this entirely rag? How does the architecture work to allow for multiple models given the technical questions you just discussed? Yeah. So I would think it gets complicated because

SPEAKER_48: it depends on if it's a law firm or a law firm's client. And so I would think of in the past three years, not just for us, but I think for most of these domains, a lot of the challenge is still not the post-training and personalization of the models. It's really connecting all of the context. And so part of why we've been doing all these partnerships with iManage, Netdocs, kind of the document management systems of the law firms is it's still very hard to get all of the context that a lawyer needs into the model before you even start thinking of how do you customize this. And so the experience you want if you're a lawyer at a large law firm is you're always working on a specific client matter. And so you want to be able to query that client matter. And so if it's a litigation you want, here's all the emails, here is the e-discovery corpus, the case law, our private documents and all the work products we're working on. And I want my model, my agent to be able to interact with all of that. And so a lot of what we've been doing is how do you pull all that together? I thought model context protocol or

SPEAKER_13: NCP from Anthropic solved that for everybody. And it was a done deal, Gabe, and we were all done.

SPEAKER_20: I think it's a good abstraction. And then the challenge I think with all this enterprise stuff is it's almost never just a technical problem. It is how do you connect all these systems?

SPEAKER_48: But it's moving there. So I think with law firms, we're actually getting there where we are starting to be able to provide this experience where an attorney has all of that context. Once you have that, the next challenge is we can't train on any of that data. And so we're fully eyes off. We never use any of our client data. But even more so, the law firm can't do that. Because if you think of the data that the law firm has, it's actually client data from a bunch of different clients. And so the infrastructure we need to build for law firms is how do they partition all of that data? And then for a specific client, can we build the infrastructure where if you have a large private equity firm and you do all their fund formation work, like then as a pair, you train the system on that if the client and the law firm are both okay with that. And so we're starting to have some of those conversations. And then if you think of it from the client side, what the clients eventually are going to want is I have one legal model and it won't probably be a single LLM. It'll be some complicated LLM system that has access to all of my company's legal data. And it connects to all of my different law firms. And it can learn from the work that all of my law firms and eventually professional service providers are doing for me.

SPEAKER_13: You can probably learn from how people use the product and service, what they're looking for, when they look for it, what type of documents constitute a successful search. Even if you can't look at the actual information in those documents because of privacy, client litigator, confidentiality and so forth. That's a big problem, Gabe, because I think most of the people can more easily kind of just hoover up the information and use it. You guys really have to be as delicate as the healthcare industry, it feels like.

SPEAKER_20: Yeah. No. And I think this is something from when we started the company, like we brought on

SPEAKER_21: the CISO of Goldman and we brought on a bunch of these security advisors, because I think the really big challenge we're going to solve is for banks, for government, how do you build these systems in a way where you can provide that Gen.AI experience, but it meets all of these kind of regulatory issues. And I think even before you get into the Gen.AI, there's already these massive technical security legal challenges. And then when you think of the balance of like, we want to use Gen.AI, we want to get some of the training benefits, but you also have all of those issues. I think that is going to be a really hard technical product, business change management problem.

SPEAKER_13: So not to be a brat, I know there's tons more work to do in the legal space, but it feels like you're building a bit of machinery that could also be applied to other industries where data is quite regulated, but they still want to use AI. Is there down the road, a copy and paste into a different industry possible, or is this going to be so law specific that it actually won't translate as I'm

SPEAKER_19: kind of imagining to something else? Yeah, we're actually already doing this in some adjacent industries. And so we do this for tax, for example. And so I would think of the problem we're solving

SPEAKER_48: and the model where this applies is, what are all the industries where you have advisors to companies and the big technical problem that you need to solve is as a company, I need to share my sensitive data with this advisor so they can operate on that data. And this is investment banking, consulting, tax audit, HR compliance, maybe insurance a bit. And so I think this expands to

SPEAKER_13: kind of a bunch of these different industries. It sounds very complicated and tricky, but the good news is that growth has been really, really impressive at the company. You guys scaled from, I think it was 50 million ARR kind of at the end of last year, up to 100 million ARR this August. And I know you guys have, I think around 500 law firms that are clients. So talk to me about

SPEAKER_00: who's buying today. Are they domestic, international? Is this just big law? Because I've had friends that have worked at big law. I'm curious what the customer mix looks like today. Yeah. So it definitely

SPEAKER_20: started, our focus was the largest law firms and we actually started outside the US. And so we worked with a lot of the large European law firms. Now we're in over 50 of the AMLA 100. So kind of the top 100 firms and kind of a combination of a lot of the really large US ones, and then a lot of the

SPEAKER_48: really large international ones. Mid-market is growing pretty quickly. And so we're starting to sell to smaller law firms and then enterprise kind of the clients of these law firms, private equity and

SPEAKER_13: fortune 500 is also growing pretty quickly. It feels like the addition of clients of law firms greatly expands your TAM because each law firm has, actually, I don't know the answer to this, dozens, hundreds of individual clients. Tens of thousands. Yeah. See, I was way off, but that's a lot more potential contracted value for the company. Do you think that's going to become the majority down the

SPEAKER_48: road? I think for us, the interesting question is figuring out how to balance these parties. And so one of the big problems we want to solve is there is inherent tension of selling to both sides of kind of this marketplace. And what we're starting to figure out with law firms and their clients is the ideal win here is we can find a model where for law firms, we can make them more profitable on this platform and for their clients, they can also benefit. And I think sometimes when people look at this,

SPEAKER_21: they say, okay, one of these parties needs to lose. But I think when you actually get into the complications of how these relationships work, we're starting to figure out ways that you can actually make this a win-win for both parties. And then I think to your point, it expands the TAM because once we go into enterprise, there's stuff we are able to do for enterprises that is not work they use outside counsel for. And so things like contracting, tax, HR compliance. So yeah.

SPEAKER_44: With AI, like everything else, the old saying is so true. Garbage in, garbage out. Without the right training data, you're not going to get great results. But my pals at Uber, they're now working with enterprises all over the world to source, label, evaluate, and scale high quality real data for your startup. Uber is one of my favorite companies in the planet because they demonstrated their ability to scale and to build great products. First in the rideshare market, then they figured out Uber Eats, and then they moved on to autonomous driving. And all of this was only possible because of a deep understanding of how to collect, label, understand, and analyze data. Now with Uber AI solutions, you can put the team and tools that turned Uber into one of the world's best companies to work for you, partnering with top talent and experts from Uber's global network. What better partner to help you scale your business than the company that organizes over 36 million rides every single day? Book a demo right now by going to uber.com slash twist. That's

SPEAKER_13: uber.com slash twist. What is the tension though? I'm curious about this because it feels to me that if a law firm uses Harvey and is more efficient and just better at their jobs because of it,

SPEAKER_04: their clients would be happy, not annoyed that they had this technology. And the other way around, if you could work more easily with your clients, why is that a bad thing? So what's the tension you're

SPEAKER_20: describing? Yeah. I mean, I think the simplified version is kind of the billable hour where when

SPEAKER_21: you're charging for work on, on an hourly basis, there is some tension of, of the efficiency. Um, but I think when you actually look at how these law firms operate, so like one, one good example I use is, is fund formation and these aren't purely billed on billable hour. Like a lot of this work has caps and there is certain of the work that you have to do that gets written off. And so there's all these extra degrees of freedom. Um, but I, I think there is just the challenge, right? It's like, if you think of that scale, if you are a very large company and you're spending $2 billion on legal, like these CEOs are giving mandates to their general councils or their legal divisions and saying, Hey, I need you to use gen AI to reduce legal spend by 10%. Right. And so that obviously is going to put some pressure on the firms. But I think the part that's exciting is we've talked to law firms where they say, okay, maybe we're going to make a bit less on a client matter, but there's firms we're talking to where they say we can actually serve this diligence on a fixed fee or this litigation on a fixed fee, or we can, you know, take a bit of a cut here, but we can do this at scale with better profit margins. And so I actually, my gut of what's going to happen is you will see law firms that are 10 times larger than the current law firms. And they're able to operate at a much larger scale, because I think Gen AI will let them decouple revenue from headcount. And so you'll be able to get closer to these software margins and kind of like scale beyond your headcount. So I think there is a win, but it's obviously a very challenging thing to figure

SPEAKER_13: out at this stage. I want to squeeze in a couple of quick ones about the market itself. You guys, I think, told Fortune that your largest competitor is by far indirectly open AI and that as general model companies get better and better, you guys have to stay, you know, reasonably far ahead because, you know, you're doing something specific. How much competition do you think actually comes from the major companies versus the startups out there? Because, you know, Spellbook and Nexel and Lucio, there's a lot of companies that are coming after parts of the AI legal world. So who's really

SPEAKER_17: bigger in your kind of day to day? Is it open AI and company or is it more the startups out there that

SPEAKER_20: are extra hungry? Yeah, I think this is the challenge with building one of these companies is you have to operate on all these different timeframes. So for example, on the day to day now,

SPEAKER_21: when we go into deals and things like that, especially with law firms, open AI doesn't come up very much, right? Copilot doesn't. But some of the law firms use this, but it's really like, we need a legal technology vendor because this is such a vertical industry. And then where you see some of it is when you sell to these very large corporates and they're not using open AI for their

SPEAKER_48: legal, but they are starting to think about how do we, you know, re-platform our company or connect all of our divisions using this. And I think that's going to happen, right? Like you will, as a large company, you will have something like a copilot, a Claude or a ChatGPT available to a large amount of your workforce, right? And you will connect all of your data systems. And so we don't see the kind of

SPEAKER_21: the value we provide as a company as long-term the individual assistant, but this creates friction as you want to like sell into these organizations. But eventually what we need to provide and is our value prop that I think will be very different than what the foundation models provide is like, what we really need to sell to law firms is here is how you create a more profitable law firm.

SPEAKER_48: And it's more than just, here's a product for an individual lawyer. It is, here's all the infrastructure to transform your law firm into an AI first law firm. And so it is the governance and the

SPEAKER_21: change management, the training, ways to deliver these services in new ways. And so that part feels like, you know, that's very differentiated. The horizontal players won't build that. And then connected with the enterprises, I think there is short term, and we had this historically with the law firms and it's gone away, the self-build, right? Where some in-house departments will say, oh, maybe I can just build this contracting solution myself. And my sense is what's going to happen is similar to what happened with Salesforce, where a lot of folks were like, oh, it's just the database table. I can like put all my client data there. I'll just build that myself. And at scale, you really need something like Salesforce because of all the things you need to build on top of it. And I imagine

SPEAKER_48: that's going to be similar of what happens with these in-house legal departments. Most of the,

SPEAKER_21: how do you optimize your legal spend is not going to be you deploy 10,000 super smart legal agents. It's going to be all the process management and the measurement connecting with law firms and things like that. And so I think that's the part where we don't see too many competitors working on that. But I think eventually getting to that, there's friction kind of from all sides and I think

SPEAKER_13: everyone faces that. When it comes to bringing Harvey into clients though, you guys are hiring forward deployed engineers and investing a lot in customer success. I was just curious as a final question here for other founders out there, how much work do you think AI-focused startups should be doing to help handle their clients, customers and get them ready to go? And is that amount of work going down over time as AI gets more advanced or is it still a pretty big cost for you guys?

SPEAKER_19: Yeah, we actually historically haven't had a large forward deployed motion. And so pretty much

SPEAKER_20: a hundred percent of our platform is things that we have built that we are selling to every customer. I think like the way that we are thinking about forward deployed is as we go into new verticals, especially for enterprises. How do we figure out what are the right building blocks to put

SPEAKER_21: in the platform? So a good example is we are starting to build for private equity outside of legal. And a lot of these private equity firms want us to build essentially a fund operating system, right? Like a platform where they can manage all of their funds, they can find all the documents. If they want to create a new fund, they can connect with their law firm. If they want to make an investment out of a fund, they can connect with the firm that does that. But then they can also do the investment diligence and things like that. And I think what we're finding is a lot of these tools that we built rhyme. And so a lot of the diligence stuff that we need to build for this is actually things that already exist in Vault. And so really figuring out what those abstractions are. I do think as we scale, especially for these enterprise customers, there is some of this you will need to do because historically, or the problem we hear from all these clients is just, we don't know where our data is, both on law firms and enterprise, we have all these custom built systems, we need to connect them. And there is some of this where you can't just go in and say, Hey, just put MCP on everything. And we'll just absorb it like you really do need to sit down with these, with these clients and figure it out. But it feels very scalable. This isn't something that I think has been a big issue. And we've sold the same platform to all our corporates and all our big law firms. Awesome. All right. Well, Gabe,

SPEAKER_00: an absolute treat. Thank you so much for coming on. I presume you guys will be at 200 million ARR by January, February at the latest. And we'll have you back on when it's time. Harvey AI, what's the website? And just for fun, is there a job you guys are looking to hire for? You want to shout out to the

SPEAKER_92: world? All roles. So yeah, please apply. All right, Gabe. Talk to you soon, man. Appreciate it.

SPEAKER_94: Founders, let's be real. Finding the right developers is really hard when you're trying

SPEAKER_44: to run and scale your startup. But Lemon.io can save you time, money, and a ton of headaches. They've pre-vetted high quality developers and they've ensured that their experience result oriented and that they're seeking competitive rates. These are handpicked devs with a minimum of three years of on the job experience and only 1% of applications to Lemon.io are accepted into their program. So forget the long drawn out hiring process. Lemon's going to match you with the skilled developer in just a few days. And they're not just recruiting talent either. They're helping you integrate them into your team. And if anything goes wrong, Lemon will find you a replacement developer ASAP. We've had a ton of launch founders work with Lemon.io. So go to Lemon.io slash twist and find your perfect developer or a tech team in 48 hours or less. Twist listeners get 15% off their first four weeks. That's Lemon.io slash twist. L-E-M-O-N dot

SPEAKER_00: I-O slash twist. AI is an enormous deal. Everyone wants to use it, but there is one startup out there that provides a unified API endpoint or hook, if you will, that lets companies pick and choose their models at will or helps them select. The company is called Open Router. It's fantastic. I live and die by its data. So please join me in welcoming Alex Attala to the show, co-founder and CEO at Open Router. Alex, how are you doing? Doing great. Thanks for the intro, Alex. Yeah, my absolute pleasure, man. All right.

SPEAKER_04: So Open Router, I think of it as a unified API that provides access to tons of different LLMs and

SPEAKER_101: inference providers. Is that a fair summation, Alex? Yeah, there's a new model every two, three days at this point, and most people are only familiar with one. So what we try to do is

SPEAKER_103: introduce the world to the wide variety of options out there. It's kind of learning that there are like multiple people in the world that you can work with as opposed to just one. I'm blown away.

SPEAKER_108: And there are true nuances and advantages to exploring the diversity of language models and

SPEAKER_04: intelligence. So when we think about the models in question here, we're not talking about just, you know, Western closed source models, but also a deluge, if you will, of really high quality open

SPEAKER_01: source models from China. There's Mistral over in France. Where else are we seeing new AI models pop

SPEAKER_103: up from? Yeah, there's a couple in China that are doing pretty well. There's Kimi K2 Thinking, English launch last week, Minimax, M2, both really strong reasoning models. We, in France, Mistral is really like major contender for European language models. We have in, you know, obviously in the US, like the heavy hitters, OpenAI, Gemini, Anthropic, and XAI. And we're starting to see like a longer a tale of mid-sized language models that specialize on specific types of tasks. Models are being used more frequently in long running jobs. This could be like building something in the background. This could be using a bunch of resources. And these long running jobs, they don't require deep knowledge about the world, they require resourcefulness. And that's kind of a whole

SPEAKER_108: new skill and a whole new direction for like the AI economy to move in. Now, I thought it might be a

SPEAKER_117: good place to start by explaining how OpenRouter is different than Amazon Bedrock or Azure AI Foundry

SPEAKER_04: for folks who might be familiar with one particular cloud or inference provider. So for people out there

SPEAKER_118: listening, how do you guys compare to those? And why are you better? Yeah, so the primary reason is

SPEAKER_103: that there's more options on OpenRouter and you're able to observe everything going on in one place regardless of where your credits are coming from. You can bring credits to OpenRouter. All of your existing deals, including things that you do on Bedrock, you can connect into OpenRouter and have it all show up there. So OpenRouter gives you a single pane of glass for discovering new models, discovering existing ones that you have access to, doing inference the exact same way through all of them. So we normalize and standardize and fix errors with all requests. And we upgrade the quality of inference pretty significantly. There are all kinds of paper cuts and ways to shoot yourself in the foot when you switch between providers or try to access a new model. And something we see very often is some company is struggling really hard with all models today on some problem. And a new model pops up and an engineer will be like, hey, I want to try this out and see if it fixes our issue. But oh my, there's a new provider. We have to set up a new vendor. We have to do a new billing relationship. Now we have to do a new enterprise relationship. Does it have a data policy that matches with all of our internal policies and a whole bunch of brain damage happens just to see if the problem is solved? So we delete all of that and just make it so you can instantly test out new models and then use them

SPEAKER_109: very efficiently in production too. So I think a lot about API is abstracting away complexity,

SPEAKER_04: much like how Twilio, you know, famously as the ER example, abstracted away telecom complexity, it just gave you a single API call. It really feels like you guys are taking a wide number of options providers and so forth in the AI world today and really just making them incredibly simple and

SPEAKER_00: easier to consume. And my presumption there, Alex, is that if it is easier to consume, easier to try more things, easier to experiment, people will wind up using net more total LLM inference or just more

SPEAKER_101: tokens, perhaps is a better way to say that. That is definitely a part of the thesis. Additionally,

SPEAKER_103: when you have more options, you kind of improve your leverage over the whole ecosystem. So, you know, if AI had zero competition, it was only one company that mattered, or only one company shipping models, you know, there isn't a point to open router. But the more competition there is, the more like diversification of features that emerges, the more ridiculous it is for a developer to try to take advantage of all those features. And that's where we come into play. Like you get them all out of the box without you having to add complexity to the work you're doing. Also, by making it easier to sample,

SPEAKER_00: to switch and so forth, you also effectively increase the quantum of competition inside of

SPEAKER_04: the AI market. Because if I had to go to open AI only and select them and live and die with them, I'm not really in a competitive marketplace. I have vendor lock, but you guys making it easier to swap between them. It makes your Googles and your Anthropics and your Mistral's and your, you know, Kimmy's just more in competition with one another, regardless of where they're based and so

SPEAKER_128: forth. So it kind of turns vendors into just just competitive blocks.

SPEAKER_101: Like we also help bring up distribution for all of our vendors and all of our providers though. So

SPEAKER_108: like, you know, a lot of the people who try out new models when they launched would probably never have

SPEAKER_103: done it had it not been really easy to just switch and explore it. So from the provider's point of view, you can think of us kind of like a distributor, where we're just helping you discover new users. And once a user gets really, really into a provider, sometimes we'll go to a direct relationship and, and like, and then, you know, connect everything in one place back on open router afterwards.

SPEAKER_108: But that way we kind of like help people grow up, and then scale up.

SPEAKER_128: Do, do AI model companies ever get a little bit annoyed that you're in a way disintermediating

SPEAKER_04: them from their customers? I mean, I know everyone wants inference. And so they're probably happy to serve the demand, but I would presume that Anthropic would rather have a direct relationship with Alex incorporated versus me using open routers API to access Anthropic technology.

SPEAKER_138: So in the early days, it was definitely a little text, because there was, it was, it was, we were just

SPEAKER_101: kind of a strange new company. It wasn't a, it wasn't a category. And, and that was definitely, I think, a big question. And our, you know, initially, our relations were on the 10th side. And over time,

SPEAKER_103: it's, it's kind of gone completely 180. Where now the, the model providers and, and the model labs, uh, like see the value in routers. And it's, it's now this kind of new, um, AI product category, like matters, routers are around to stay. We, we have like competitors too. And the, the product category, like helps them get distribution. Like I mentioned, it also helps them learn about their models that we do these stealth models where, uh, model labs will watch a model ahead of time and learn from the, the, like our community and our power users. And, um, and like the initial benchmarks that our community runs on the model. And then we did this with GPT 5, uh, with GPT 4.1,

SPEAKER_108: um, with, uh, with Grok 4. And, uh, currently a, a model called Polaris Alpha, which, uh, Alex,

SPEAKER_04: before the show declined to break confidentiality in his NDAs and tell me what it is. Uh, but one thing I did notice just looking into its stats is that it's already seen around, I don't know, 30, 35 billion, uh, tokens process per day. I presume that companies come to open router with a model for some discrete testing, because you guys have just a volume of users that will immediately tap into it and put it to work. That's a lot of tokens, Alex. Yeah. We, we have a good,

SPEAKER_101: I think there's a couple of communities that sort of compose on top of each other for open router.

SPEAKER_103: There's this, uh, like LLM power user community that, um, that we really identify ourselves with. That's kind of how the company got started. And, uh, these are, you know, they, they test out all the models they're building or often building things as side projects. And sometimes they're, they're like running internal evals at large companies. Um, and they build a really good intuition for like when a new model should get, should be used. Uh, we also have this, a wide variety of bring your own model apps. These apps, some of them you can see on our rankings page as well. Um, are apps where they want users to just be able to choose any model they want and have it work out of the box without the app having to change any code at all. And users love this because I think this is something that the world doesn't quite get yet. Models are like content. Every time there's a new model, it's like, it's like a holiday. You know, there's, there's, uh, um, you know, there's a whole bunch of problems that might get solved. It's a new marketing moment for all the apps that suddenly can support the new model. It's, uh, it's like a new person being born. And, um, and, and, and when that happens, people develop these like personal relationships with the model. And then they, they like understand that, and then they want to go to their apps and they want to bring that with them. And so the apps that support, um, open router like that, uh, can take advantage of all of those things for free. Um, so that those also, I bet that also helps the stealth models really understand the wide variety of use cases. Because people have so many different things

SPEAKER_04: they want to use it with, bring it to you guys, get a diverse use case set. It's not like eight of your friends using it all for the same thing. No, that makes sense. Um, on the venture capital side of things, you guys announced about $40 million worth of capital raised across a seed in a series A earlier this year. I think Andreessen led the seed, uh, Menlo led the A. So talk to me about the business model. As far as I can tell you guys charge a five and a half percent platform fee,

SPEAKER_159: um, for non free access. Is that the, the main way that open router makes money?

SPEAKER_101: So it's there's, there's three ways there there's this credit top up fee, which, um, which is it's basically for off the shelf inference. If you want to do volume with us, you can contact us

SPEAKER_103: through our enterprise page and, uh, get that fee down quite a bit. Okay. Um, the other way we do it is by doing tons of volume on providers and, um, and just like working with providers to, uh, like,

SPEAKER_152: you know, do volume discounts. And then we often give those to users too. Um,

SPEAKER_04: just to interrogate that if you're offering, let's say model X from model company Y at, uh, market rates, but you guys negotiate a 15% discount that provides really comfortable margin to open router

SPEAKER_101: itself. Is that what you're saying? I, I won't go into what the margin is, but, but that's like part of how we have a margin. Um, that's right. And then the, uh, last component is that there's a whole bunch of

SPEAKER_108: inference adjacent software that people really want. Um, things like observability, web search,

SPEAKER_103: um, like improves team management, the ability to like limit certain parts of your team programmatically,

SPEAKER_108: um, easy, uh, like multimodal support, um, PDF parsing and PDF management, file context management, all this stuff that like, it kind of like, it kind of becomes really powerful if you have it managed very closely with your inference. Um, those things, you know, we, you know, we have like enterprise features that we charge for and that's, I think we're like, but the margin will shift over time.

SPEAKER_128: Okay. That helps a lot. Now going back to AI models and such, uh, you guys have hundreds of models

SPEAKER_00: on service. You have dozens of inference providers, and that has yielded a growth curve in terms of total

SPEAKER_04: tokens processed that looks like this. If you're on the audio version, I'm sharing the open router, um, rankings page. And as we can see here, Alex, pretty quick and steady growth across inference volumes. We're on track to set a new record this week. I think it's about 5.8 trillion tokens, 5.7,

SPEAKER_53: sorry. Um, how quickly does this chart keep going up over the next 12 months in your view?

SPEAKER_101: Well, I think it's absolutely insane how, uh, how like underutilized LLM still are. Um, there are,

SPEAKER_103: you know, I, I, I mean, I, on top of just running into a lot of repetitive tasks that even engineers continue to do today. Um, like the, the, I think the most exciting sort of glimpse into how many more tasks can be automated is the, the new, like types of training data that the model labs want.

SPEAKER_101: And, um, you know, like they're really interested in figuring out like how to be a product manager,

SPEAKER_103: like what kinds of things you need to know? How do you like take a bunch of different contexts and distill it into action items or, um, like, like better ways of running agent workloads, um, understanding, like somebody's calendar very deeply. And these, uh, these tasks, like after a little bit of time, they start to show up in the models and what our job is, but what we want to do is make that work visible to everyone. You know, if a model does a really good job managing your calendar, that should be clear in the rankings, um, using empirical data based on how people are using it. And, uh, and then, and also other signals that we, we kind of understand over time. So, um, the, we're like, you know, there's, there aren't that many categories that like model labs are known to optimize for today and it's just going to blow up. So I think the, uh, I mean, I think it's, it's pretty amazing how, how much more work there, there continues, uh, to be.

SPEAKER_80: Just doing that down there for folks, um, who maybe are a little bit less, uh, head in this world

SPEAKER_04: as we are, you're saying that LLMs are going to get better at domain specific tasks by better training by their, uh, makers. And so essentially we're going to see LLMs perform better at different categories of knowledge work, product management and so forth, and not just drafting legal documents

SPEAKER_103: and writing computer code. Yeah. I mean, I, you know, I'm, I'm not training models myself. And so I'm not the best person to ask about the exact ways that are going to improve. This is, this is kind of my speculation and hope, but, um, I think the, the, like what that's like a big signal that there's

SPEAKER_101: room to grow. The fact that nobody is tracking improvement on those things yet. Um, you know,

SPEAKER_103: a key kind of muscle to build when you're entering the AI space is benchmarking. Everything is about evals and benchmarks. And if you make claims without having an eval or a benchmark,

SPEAKER_101: what do you, what are you really saying? And the, uh, there's a lot of like, you know, there's just a lot of value to unlock by creating a benchmark and it also guides the

SPEAKER_60: models in, in that direction when you do. So evals and benchmarks are very useful. There's other, um,

SPEAKER_04: vectors you might use to choose which model, which provider at which time you guys built something called auto router that essentially does some of that work for your customers. Uh, I presume today it's an unbiased, um, setup in which you guys are only selecting for the best, uh, option for your customer, but is there a way in the future in which people could like sponsor that to get more people

SPEAKER_186: to try out their models for certain use cases? It's a good question. We, we, so far we've,

SPEAKER_103: we've, we've stayed away from, uh, letting any providers sponsor more traffic. Um, our goal is

SPEAKER_108: to be like a strong neutral Switzerland style gateway that gives everybody a fair shot based on, on, you know, real world metrics and, uh, and, and real world benchmarks. So, but I do think that there will be interesting ways to like shift people to new models in the future. Um, we, for example, like we do a little bit of model recommendation today, but very little, you'll start to see that ramp up in 2016. Um,

SPEAKER_190: you know, if you're 20 or 26, 20, 26, I know we're getting old, but like, we have to at least get the

SPEAKER_194: tech. You're good. Keep going. If you, yeah, yeah. They're like, there are all kinds of workloads

SPEAKER_108: that are just like letting models that are way too powerful, uh, and don't, you know, or you can save

SPEAKER_198: a ton of money. Yeah. You don't need to use GPT five high. If you can use, you know, 4.1 or whatever.

SPEAKER_103: Yeah, yeah, yeah. Exactly. Um, and having the, having like a really wide range of models and a lot of data about how they get used, I think helps there. Um, but, uh, but there's also, I think good ways of, of like making very interesting model routers that we'll experiment

SPEAKER_203: with. Um, it's kind of a good positioning for it. In your, uh, Series A announcement,

SPEAKER_89: uh, Matt Murphy, one of your board members said that he quote, uh, believes open routers value will

SPEAKER_04: grow not only because there'll be more inference spending, but also because the startup will collect data on how the models are behaving. It feels like that's aligned with what you're saying here, that you guys will be able to better recommend stuff for users and learn from them and kind of create a virtuous cycle in which people will always have the right model at the right place at the right

SPEAKER_164: time. Right. Um, and, and today, like our focus so far, our focus has mostly been about giving you

SPEAKER_103: high quality inference for the models you want to try and, and, and to use in production and to help you discover new models based on your preferences and like model discoverability, I think was like the key

SPEAKER_108: product, um, strategy for us where like, or goal, I should say, where we really wanted people to like discover new models themselves and feel like they understood how to get there and then bring those

SPEAKER_128: choices really easily into their products. Do, do the models eventually end up so close in terms

SPEAKER_04: of performance that they become fungible or commodified. Uh, I was just looking at the artificial analysis, kind of like leading thinking index, and it's amazing how many models are within one or two points of each other. Uh, there are vectors where there's, you know, big differentiation, uh, grok code fast one, ever since that came out on open router sticking a huge market share, because it's cheap and fast. But if I kind of just think a year or two ahead, everything's going to be, you know, 10, 15, 20% better or whatever. And it all might taste like chicken. Is that where we're going or am I being a little bit too, um, blase about individual labs ability to create something

SPEAKER_103: that's distinct from the competition? Well, um, you know, similar to like, you know, what I said earlier about making models shine based on these really like sometimes idiosyncratic skills that they

SPEAKER_101: spike on. We were, we're doing the same thing for providers. So a provider that, you know, is extremely, you know, has extremely, puts a ton of efforts into improving their tool calling ability should be

SPEAKER_103: shown accordingly. And that's what we do with our exacto end points, for example.

SPEAKER_168: Which, uh, focuses on tool use at the inference provider layer, right?

SPEAKER_101: It focuses, it's, it's based on a lot of different inputs. Um, we look at the, the models accuracy when

SPEAKER_103: calling tools with the correct schema. We look at the, um, proclivity to call tools. We look at, uh, we look at our own users preferences too. We have sort of like, you know, door dash style, uh, preferences that, that come in when users are using our API. So, um, you know, we, you know, those are kind of like votes, um, and at scale, they become high signal. Uh, we, and then we, we, all that data, we kind of turn into a big dynamic benchmark. Most benchmarks are static and static benchmarks become worthless after a certain short amount of time. So we believe really heavily in, in dynamic benchmarks that are constantly updating. Um, and, uh, and that like become products as a

SPEAKER_128: result, we productize them. All right. Uh, one last question before I let you go, your, I think co-founder Chris Clark, he's your COO. He said that, uh, he believes that inference costs will

SPEAKER_04: eclipse salaries as the dominant operating expense for most knowledge-based companies in the next five or 10 years. I just want to understand this. Is he saying that as, uh, AI models writ large become more and more intelligent, we're going to spend more money having them do work. I inference spend then we're spending on, on human salaries because on one hand, that's very exciting. On the other hand, I worry about everyone who's 20.

SPEAKER_101: Well, I, um, I don't know if he meant operating spend inclusive. I, so I think there's definitely

SPEAKER_166: going to be some, this is kind of an ongoing debate in the community, how, how much, uh, human

SPEAKER_103: salaries are going to compare to inference spend. Um, but I think it's kind of like a, more of a question of like what in the economy, like how much value in the economy should be done by AI models. Um, if they're, you know, improving like this over time, a little bit faster, um, versus humans. And like,

SPEAKER_101: it's kind of tough to make any of these statements in the like current day, because you have to kind of like look at the curve and then just look at how the statement changes year to year. Um, and what kinds of, what, what sort of domains don't seem to follow the pattern. So I like, it's, it's super early now, but my assumption is that there are going to be some domains where, um, AI models are,

SPEAKER_103: are, are just kind of not scaling super quickly. And so we'll have like much more, you know, much more at like human salaries directed at those jobs. Um, and those could be things like, uh,

SPEAKER_108: like actually orchestrating models as an engineer, new jobs that didn't exist, you know, prior to AI, um, uh, just focusing on coding, like, like deep infrastructure work, which the models are probably weakest at today. And that might stay up a little bit less quickly. And, uh, and then there'll be other jobs that like most, you know, humans mostly do today, and probably going to be mostly replaced by AI and where that statement will be correct.

SPEAKER_225: Well, again, a super stoked, super worried, but, uh, I'm old enough that I'm going to dodge the

SPEAKER_00: damage here. I just, I think about the future, but at the same time, uh, people want to use AI, because it's great. So who's going to stop them? Well, Alex, it's been an absolute treat. When you hit your first 10 trillion token week, I'm going to be screaming about it from the top of my lungs here. Thank you for having the data so public, so we can all use it. And, uh, it's openrouter.ai. And before you go, is there a job that you're looking for the right candidate for that you want to shout out to the audience so that way they can come find you?

SPEAKER_228: There are a bunch, uh, go to openrouter.ai slash careers.

SPEAKER_108: And, um, yeah, feel free to message me on Twitter too. If you have a, if you have a, if you want, if you don't fall into one of those roles, but you think you should.

SPEAKER_156: All right. We appreciate it, man. We'll talk to you again in six to nine months. In the meantime, Alex, thanks for coming on.

SPEAKER_203: Thanks, Alex. Thanks for having me.

SPEAKER_00: Jason, I found a really interesting question over on the startups subreddit. Um, there's a founder that argued that it's hard to build a startup today because everyone is building things so quickly that no matter what you put into the market, people will essentially copy and paste using vibe coding tools and essentially collapse your pricing ability. People have always done copycats ramp famously copied bricks to some degree down the line. Uh, so to me, this just sounds a little bit like cope from our friend. I'm curious what you think about quick vibe coded competitors and the potential for it being very hard to get traction when everyone can just roll out the Xerox machine.

SPEAKER_234: You know, as this person, person saying it's now almost impossible to build the moat, every product mark, uh, every product market fit turns into a red ocean in the blink of an eye. Before you even managed to build an MVP or get early traction, 10 other ventures are already doing the same thing, burning money and making any road roadmap to profitability useless. So that is peak bubble behavior, right? Like when we get to a bubble, there's a lot of money in the system, a lot of entrepreneurs, a lot of people chasing the same thing. So you'll see a bunch of knockoff companies. That's pretty normal. Um, what happens with those companies is though, they tend to give up. They tend to not be able to keep up in the race. So a lot of people start the race, right? You might get 10 people who are doing ride sharing, 20 people doing delivery services, and then eventually DoorDash and Uber Eats run away with deliveries for food. And people forget about Postmates or, you know, Sprig and just, you know, a long tail of delivery services, local and all different modalities of it. So then it becomes a marathon. So who wins in a marathon? It's the person who has the most discipline, the most trained, well rested, you know, best diet. And in those cases, it would come down to who has the best team, who is constantly shipping features. Um, and then that there's network liquidity. So not everything is about building the product. The liquidity of the network's also important. So if you open up, you know, a ride sharing app and it takes 10 minutes to get a ride, or you open up Airbnb and there's, you know, a hundred choices in a city, you open up one fine stay and they haven't gotten to Lake Tahoe yet. Okay. You're going to use Airbnb. Yeah. So the liquidity of the marketplace becomes defensible. A sales team becomes defensible. If you look at Salesforce, they have many competitors, but, or Oracle, many competitors, many options, but they had incredible sales teams. So there are many other pieces to the puzzle that can become defensible. In the early stages of the startup, it's really going to be about using capital as a weapon, being able to raise more money than the other competitors, and then do marketing, uh, and have more features or more distribution. So congratulations. Yes. Everybody has the idea. Some people can build it. And then a smaller number of people can scale it. And then an even smaller number of people can maintain it, you know, and sustain it. So you each of, each of those levels, you lose competitors. Everybody has the idea. Very few people can finish the product of the people who finished the product. Very few people, uh, can keep the cadence up with distribution sales. Right. And then on top of that, very few people can raise money at scale. So all of a sudden you get down to a two or three horse race in every one of these businesses.

SPEAKER_128: So moats that are almost like concentric circles of, of, of not only product, but also internal capability,

SPEAKER_234: team capacity, and market liquidity. Correct. All of these factors become important later on. So distribution's an important one. If you look at a product or service like perplexity, remember they were doing all these weird deals in the beginning. Like if you, if you buy this, you get a free perplexity account. It was like five, you get an American express card, you get a perplexity account. You get Uber, you get a perplexity account. They really were just, you know, throwing. Yes. And then affiliate programs, like just annoying people with these things. They, somebody on that team at perplexity was very good at doing annoying partnerships to just get the first crank started. Right. So now the next person who built something that was 20% better than perplexity is sitting there and like, I didn't hire some affiliate marketing partnership program person. I didn't hire a social media person. And this is where startups get really hard in competitive spaces. You have to have, you know, distribution and go to market strategies that beat the other people. It's not just enough to have Betamax versus VHS. It's an all reference to VHS videotapes. We used to watch movies with them. The better technology doesn't always win. Sometimes it's the one that's better marketed and better sold and marketing and sales. All that is another art in business. You have to learn after product market fit, or, you know, you can, that's the good news is you can buy that one. You can find people who've done it before. So you got to just get help. But it is worse now. So you got to deal with it. It is worse in a hot market, you know, quiet market. It's hard to raise money. Um, and it's easier to get attention for your product in a hot market. It's easier to raise money, but there's so many competitors. It's hard to get attention for your product. So in both cases, it's still hard to get attention for product in one. It's because you have competitors. It's a competitive market. And in the beginning, uh, it's because nobody cares about your product because it's a down market, right? Like people are just not evaluating new products or technologies because

SPEAKER_80: their businesses are making cuts. So entrepreneurship is always hard is what I'm hearing.

SPEAKER_234: Yeah. None of it's easy. I mean, if it was easy, everybody would be doing it. Uh, and it's extremely hard and it is a, it is a death march in some ways, you know, where people, there's just like Steven King movie, um, about some, like everybody marches and it's like some sort of weird long walk. Okay. There you go. So I, I, I was fascinated by the premise and disturbed. Gotta take this long walk. Uh, and yeah, uh, the last person standing gets the prize. Everybody else dies. Spoiler alert.

SPEAKER_128: It's in the trailer. I wonder if that was inspired by, uh, the Chinese communist parties, long march.

David Friedberg: It's an interesting historical question. It's a, it, this was a, um, Steven King book, right? I wonder if he was inspired by that, that bit of, uh, CCP mythos. Wow. Steven King wrote it when he was just, but 17 years old. All right, everybody. That's another great question.