SPEAKER_00: I'd have a lot of conversations with my internal team as well, like my co-founders. And the framework we used internally was which path allows us to influence the world more. We are all academics. I care about this field. I care about technology. We care very much about how AI is going to impact humanity. There's a large component that's extremely mission-driven here. You know, I would say not all startups form that way. You know, if you have some SaaS startup that's building a cool little feature, you're really trying to like make money, which is fine. There's nothing wrong with that.

SPEAKER_03: I have no issue with it. I don't think you're really thinking about humanity so much. And I'm not trying to beg you to be on a high horse or anything like that. I'm just trying to like really say like, we cared about this.

SPEAKER_00: And that's that was our initial motivations.

SPEAKER_05: This Week in Startups is brought to you by OpenPhone. Create business phone numbers for you and your team that work through an app on your smartphone or desktop. Twist listeners can get an extra 20% off any plan for your first six months at openphone.com slash twist. LinkedIn jobs. A business is only as strong as its people and every hire matters. Go to linkedin.com slash twist to post your first job for free. Terms and conditions apply. And Beehive. Power your newsletters with AI tools, referral programs, and ad network features all in one platform. Get 30 days free and 20% off your first three months at beehive.com slash twist.

SPEAKER_12: Hey, everybody. Welcome back to This Week in Startups. My name is Alex, and we have a very, very excellent episode coming your way.

SPEAKER_15: Databricks, as you know, is one of the most valuable private market technology companies in the world today. It last raised $500 million in a Series I that gave it a valuation of around $43 billion. But also, in roughly the last year, the company bought Mosaic ML. And before that deal was actually put together, announced, and then executed, we had Naveen Rao, the CEO and co-founder of that company, on the show. Well, I have dragged Naveen back because we are now over a year past the acquisition. Much has happened in the world of AI, especially on the open source front. So please welcome back to the show, Naveen Rao, now from Databricks. Naveen, hey, how are you?

SPEAKER_17: Great. Thanks for having me on. Cool. Close the loop a little bit after we were in the midst of the acquisition, and now we're

SPEAKER_20: on the other side of it. So.

SPEAKER_15: All right, let's start there. So $1.3, $1.4 billion deal, how long did it take to go from, like, you and Ali Gazi

SPEAKER_22: talking to when the actual paperwork was signed?

SPEAKER_00: Well, I guess first conversation was probably, like, end of March, and then we signed the deal, like, mid-June. Actually, going from, here's what we want to do, we actually want to, like, the concept of, yes, we want to work forward towards an acquisition was probably, like, early May. I think it was, I think it was 62 days from that conversation to the actual close, which

SPEAKER_26: is kind of record time.

SPEAKER_15: Yeah, that's a lot of work to do in that amount of time, because you were a venture-backed company, so there's a lot of parties involved, a lot of signatures. Totally. I have to ask, though, because I was just going back through Databricks' acquisition history, prepping for our talk today. Okay, and the more I think about how Mosaic ML fits into Databricks, you guys are super important, because Databricks, of course, built on the open-source Delta Lake project, really kind of popularized the Lakehouse format for BI across structured and unstructured data, as everyone knows. And then you guys brought on the ability to train, essentially, AI models for corporations that have data, which nests very neatly on top of what Databricks are already built. So, a pretty core component of what Databricks is today, and given where prices are for some AI companies, Naveen, did you perhaps sell for a little bit less than you otherwise might have? And I know it's rude, but I'm curious if you stay up at night thinking about the yacht you didn't buy.

SPEAKER_00: I mean, we're doing just fine. Honestly, the math was about, can we come together with Databricks and actually make the whole even better? That was the way I looked at it. And honestly, the valuations are high right now, but they're somewhat paper valuations. I had a conversation with Ali beforehand, and it's like, really, the valuation and stock price of Databricks is strictly less risk than Mosaic. So, we're moving from kind of high risk, very, very high beta to something that's slightly lower risk, probably still high beta as far as public stock go. But we saw that there was an opportunity to make the whole better. And really, that's what the thesis was, was, I mean, I can kind of walk through the logic. It was actually quite simple. If we stayed at it alone, you know, things were growing really fast. And let's say in three years, I was looking at a three-year horizon and it was like, can we get to, you know, call it 300 million revenue? That would put us in the, you know, $5, $7 billion range as far as valuation goes. We'd probably need to raise around, around and a half, get there. So, there's going to be some dilution taken. And then if I, so that's scenario one. And so, that has certain risks associated with it, of course, the market and what happens with AI, et cetera. Then there's Databricks, which basically I said, in that same timeframe, can we double the price? And I think that doesn't sound crazy, right? In three years, can I double the price? And then we're actually at the same place in terms of how much our stock is worth. So, I looked at it from this perspective and the Databricks path was lower risk. And really, the first conversation I had with Allie about it was, this all sounds great.

SPEAKER_03: I'd love to do it. The economics just have to make sense. Like, those numbers have to line up, right? And so, we got there.

SPEAKER_38: Yeah. Well, this brings us to founder mode. You had a tweet from a couple of weeks back about, you said the funny thing about founder mode is that if you have to tell everyone you're in it, maybe you aren't, found that really funny.

SPEAKER_15: But I do think that you have to be in founder mode to get a deal done in 62 days from start to finish. Did you have to jostle investors a little bit to get them to agree to this? Or were they pretty on board from the start when you put the deal together?

SPEAKER_00: Yeah, that's a great question. And literally every one of my investors wanted me to continue going. Really? Yeah. And it wasn't in a bad way. I wouldn't say this in any negative way. My investors actually, I've known for a long time, many of them. And they were like, look, let's go at it. Let's not sell. Let's go big, right? They're like, if you want to do this and you think it's the right path, we'll support you. But we're totally all in to keep going. So I thought that was a great message. And really, it was hard because I'd have a lot of conversations with my internal team as well, like, you know, my co-founders and sort of the framework we used internally was which path allows us to influence the world more.

SPEAKER_47: That was it.

SPEAKER_15: Do you think that that perspective is the right one for founders in general to take as they observe a possible acquisition offer? Or is that more of a Mosaic ML specific perspective?

SPEAKER_00: I would say definitely it was a Mosaic thing. We are all academics. I care about this field. I care about technology. We care very much about how AI is going to impact humanity. So there's a large component that's extremely mission driven here. You know, I would say not all startups form that way. You know, if you have some SaaS startup that's building a cool little feature, you're really trying to like make money, which is fine. There's nothing wrong with that.

SPEAKER_03: I have no issue with it. I don't think you're really thinking about humanity so much. And I'm not trying to beg you to be on a high horse or anything like that. I'm just trying to like really say like, we cared about this.

SPEAKER_00: And that's that was our initial motivations.

SPEAKER_15: Well, I mean, your comment was whimsical and, you know, as a slight edge to it, but I don't think you're off base. And also, I think what we've learned watching SaaS companies as a specific cohort in the last two years is that if you built a SaaS company, that was a very strong feature that raised money in 2021, you may not survive in 2024. So I don't think I don't think your comment, even though it was slightly sharp elbowed was was off base. Now on the mission point, though, this is a good thing to touch on because Databricks, of course, has a open source heritage, if you will. And also the Mosaic ML company put together a number of very popular open source models, including MPT 7B and MPT 30B. One of them got more than 3 million downloads, which I think at the time Naveen was the most

SPEAKER_60: downloaded open source LLM. Is that right?

SPEAKER_62: It was. Yeah.

SPEAKER_15: Yeah. Yeah. So it seems like there's a there's a real kind of like mission agreement when it comes to open source in general at Databricks plus Mosaic. And this brings us to the recent news item in which we saw that SB 1047 in California, a AI regulatory bill was vetoed. Now, one sticking point or one point of criticism in that was that that bill might have made open source AI development more, frankly, legally risky. Now that's behind us, I'm curious if there's anything that you think is standing in the way of open source AI development when we consider it in a competitive race with closed source.

SPEAKER_00: I mean, I think the potential for lawsuits on copyrighted information obviously is real. There was still the executive order that was put out at the end of last year that put some hard limits in terms of compute. It's not clear how this is going to impact open source. I think whether something is open source or closed source is a business decision, but like outlying it actually causes problems. We're already seeing it with llama models, like multi-modal llama models. We're not rolling out in Europe because of the regulations they put in. So I think that's the net of this is that the consumer suffers, right? They have fewer choices and there are fewer ways for them to modify, customize these things for their purposes. So I think it's a net negative if you start to restrict what people can do, especially this early. You know, maybe in five years when we truly understand the impacts and how the liabilities

SPEAKER_71: work, et cetera, et cetera. Okay. We can start to put some rules in, but it's just too early.

SPEAKER_73: All right, everybody I'm on the road. I mean, all the time, right? And that means I am always juggling phones and laptops, apps, all these different services I use. And when you use multiple devices and you have all these different apps running your business, you need to have one single phone number that is perfect. And that perfect phone number is open phone. It's going to simplify all the communications you have in your organization because open phone has rethought the modern business phone. It's so magical. It works with a single elegant app that you can put on all your devices and it works right on your existing phone, even works on your desktop. Our sales team uses it here at launch. Why do we use it? Well, we don't want people talking to customers on their private phone lines and our account executives don't want to give their personal phone number out. That's just weird. You want to have everything tight and open phone will make it tight and tight is right in this regard. Shared phone numbers are also awesome for things like customer support or when we run events, we like to have a field phone number. So, hey, if you're a VIP and you're at the liquidity conference, just call that number. Open phone is super affordable. It's just $13 a month. Twist listeners, get an extra 20% off because they got you covered. Openphone.com slash twist. And what if you have an existing phone number? No problem. Open phone is going to port them over at no extra cost.

SPEAKER_78: Head over to openphone.com slash twist to start your free trial and get 20% off.

SPEAKER_15: So, on the meta not bringing their most recent family of llama models, I think that was announced at the 3.1 work. We're now at 3.2 for the meta llama family. I'm curious what your take here is on this. Is that an actual meta saying, look, we can't bring them to Europe because of regulatory confusion? Or are they trying to make a point by saying that as a way to politely influence the way that AI regulation is put together in Europe?

SPEAKER_00: Honestly, I don't think there's anything to galaxy brain going on here. I think it really is perceived risk. And it's like, okay, don't want to take that risk. All of us, every company who's deploying these solutions has to look at that and make a decision around the risk. And yes, maybe there is in the back of our minds, like, okay, we want to change that environment and make it more friendly. But, you know, I think in the short term, we're really just looking at it like, is it worth the risk? Is it a risk reward trade-off you're willing to make? And llama said, no, it's not. Or meta said, no, it's not. So, I don't think there's anything else going on here. I think this is a consequence of early legislation.

SPEAKER_38: Yeah, does Databricks have any restrictions or set up changes based on different geographic

SPEAKER_15: regions for its product set? Because Databricks plus Mosaic is now a pretty broad company. So, I don't know every single product line you guys have exactly where it's available.

SPEAKER_88: But I'm curious if regulations have also impacted how you're rolling out new features, open source models, and so forth.

SPEAKER_00: Absolutely. Yeah, it is. I mean, we conform to a lot of these same rules.

SPEAKER_15: Okay, so essentially, I can't run, you know, a Lama 3.2 model on the Databricks Mosaic

SPEAKER_38: system in Europe?

SPEAKER_00: At the moment, we're still looking at the exact details of this. But, yeah, we're not trying to, like, provide an end around or anything like that. That's not the goal, right?

SPEAKER_38: Right, right, right. I was not leading towards trying to get you to provide an end run around, Mark Zuckerberg.

SPEAKER_15: But I am keeping tabs on how open source AI development is playing out around the world because, just frankly, as an open source sympathizer, just kind of, you know, personally, I'm really excited about what open source AI models have done in the last 12 months. It's given me a lot of excitement that there's not going to be too much of a performance gap between what we're seeing that's open source and available on, say, Hugging Face, and what's available via a paid, you know, open AI API call. And that actually kind of brings me to the Lama family of models. When those came out in July of this year and then later September, were they better than you guys expected them to be? Because the market seemed to be very pleasantly surprised by them.

SPEAKER_43: Were they better? I think what was the big surprise for us with the 3.1, the 3 and 3.1 was really maybe not

SPEAKER_00: the quality of the base models. We knew that was going to be within a certain window. side of it, where they actually did a lot of work there and, you know, frankly, put a lot of money into it. So that was probably surprising for us, how much they did. And I think it was really a statement of, here's our commitment. This is how big we're going to go with this. So it wasn't clear when the Lama 2 models came out, like they were good. But, you know, DBRX and other open source models came out, Mistrol and others that were better, you know, at the time. So it wasn't that that was, they were far and away, but then honestly, I think in the last, you know, six months, things have really shifted. They're, they're investing heavily and they're building really great stuff.

SPEAKER_15: Yeah, absolutely. And just so everyone knows, RLHF is reinforcement learning from human feedback. I actually forgotten two of the acronym points. Just wanted to bring that up. Can you just explain what that is and why it matters for folks out there who are AI interested, but not perhaps AI experts?

SPEAKER_102: Yeah. So when you put these models in the wild, you can't, you can't make a, give them just a raw model.

SPEAKER_00: There's all kinds of weird stuff these models can say, you know, things that, uh, might be correct, but not necessarily, you know, kosher to say, uh, perhaps, or, uh, you know, sometimes they're, they can weed down paths that go very dark and we don't necessarily want our models to do that. Yeah. Especially if you're talking about psychological safety, if somebody is depressed and they type in, you know, what's the best way to commit suicide? I don't want the model to instruct them. So actually providing these kinds of balance and guiding things in a way that to positive or saying, I won't answer really, it's pushing all the probabilities around of where that

SPEAKER_34: knowledge lives and how to, how to impress it. And it's actually quite hard to do this well.

SPEAKER_38: Yes. And if you're going to do a lot of it, you're going to have to have a lot of humans actually

SPEAKER_15: doing this, ergo it's expensive. And that's why your point about what, uh, meta put into the Lama models matters, because they're not just writing code. They're also doing the, the flesh and blood work to make these models pretty good.

SPEAKER_38: Now you mentioned, um, DBRX, which is the, uh, open source model that Databricks put out, uh, March of 2024.

SPEAKER_15: If memory serves.

SPEAKER_108: Correct. Yeah.

SPEAKER_15: And the company said at the time that it spent about $10 million training this model. And that number just kind of sat in my head because I couldn't quite figure out what it meant. It didn't seem like a huge amount of money. It didn't seem like it was free. But one thing I did recall is that mosaic ML, one of your goals as a company was to make training models a little bit cheaper, more efficient. And so I'm kind of curious one, are y'all going to be putting out DBRX two? And then two, uh, should I expect models of that size and complexity to become cheaper to train over time from companies like Databricks and also perhaps other corporations?

SPEAKER_112: Yeah. So the first one, you know, we're, we're partnering very closely with meta now. Um, so we basically said, what's the point of providing another choice, uh, unless there's

SPEAKER_00: direct innovation. So we can choose to work together with them and say, Hey, these are things that, uh, we see our customers needing, uh, you know, new potential directions to take to make more efficient. So now we're just working more closely with them. So at the, at the moment we have no plans for DBRX two on the second point, you know, I, we're constantly working towards making these things cheaper. So, uh, we're still doing base research on the, on these kinds of techniques. Like how do we extract the most from our hardware? How do we make these neural networks learn faster and cheaper? Um, and you know, we're going to communicate that with our partners like meta. Uh, and so, you know, the, the DBRX was cool because it was a very large, what's called a mixture of experts. And really this was a way to make things train cheaper to a certain quality, but not only that actually make the inference faster and cheaper. Um, and we're going to see this happening in the open source more and more, uh, mistral actually had an, uh, open source MOE model as well. So, you know, I anticipate we're going to see more of these techniques coming out and really the benefit is going to be to the user. You're going to see a faster response with higher quality and less money. That's where we're moving. I still believe this idea that we talked about at mosaic. We call it mosaics law of four X reduction in price to get to a similar quality model, uh, year on year is real.

SPEAKER_84: So we're seeing that today, we're seeing it from open AI, we're seeing it from everyone.

SPEAKER_15: So this actually brings to me a thought that I think six months ago, it wasn't talked about much, but it's now pretty common, which is that people are concerned about the value in AI models and how quickly they can depreciate based on progression. And I think we can see this in open AI price points for a model over time does decline pretty precipitously. So does working with Meta allow essentially Databricks writ large to have leading AI models inside of its platform without having to do the work to train them? It feels very much. And I say this politely like the corporate version of getting your cake and eating it too.

SPEAKER_03: Maybe that's what we're shooting for. No, I mean, you know, look, they, they want to invest in this and they have good reasons

SPEAKER_00: to do so. Yeah. And, you know, we, we're close with them. We, we have, we're, we're non-competitive in a lot of ways, right? Uh, Meta is, is doing this to enable a developer community. Uh, you know, they're, they're a social platform. They're using the models internally for all kinds of stuff. Those things don't overlap with what we do. We're focused on enterprise. And so it's actually a really great partnership. We're very complimentary.

SPEAKER_03: Uh, they want to build a developer community. We want to enable developers. So it works great.

SPEAKER_38: I just realized that now I have to ask you once we stop recording to not sell your company

SPEAKER_15: to Meta, because given that how you guys are, are sufficiently well-matched, I mean, I meta buying Databricks wouldn't be the strangest thing we've seen in the last 24 months, but it would be a top five thing. Yeah.

SPEAKER_73: Founders. I know that you're keeping a close eye on your burn rate. I am too. In today's venture market, every single hire you make has to be perfect, right? You can't make mistakes. You got to keep that runway as long as possible so that you can run more experiments. And you need talented people to run those experiments and figure out how you're going to get product market fit. How are you going to scale your company? And that's why you need to use LinkedIn jobs. As you know, LinkedIn brings you the candidates that you can't find anywhere else. LinkedIn passed the 1 billion member mark. Think about that. 1 billion members and 70% of LinkedIn users don't visit the other leading job sites. This is a phenomenal statistic. They don't even go to the other job sites. Why? Because they might not be looking and those are the best hires, but they're hanging out on LinkedIn, doing professional development, checking in on their network, building their network, sharing content, finding leads, all that great stuff. Bottom line, there's amazing hires waiting for your company on LinkedIn and nowhere else. And they have a special deal right now. Post a job for free. What? F-R-E-E. What a great price. LinkedIn.com slash twist. That's right.

SPEAKER_129: LinkedIn.com slash TWIST to post your job for free. Terms and conditions. Course apply.

SPEAKER_15: So on models getting better over time and also cheaper, the thing that I've been playing with the most lately has been OpenAI's O1 model. And I still think this is being digested a little bit by people, but it does feel a bit like a high watermark. And it was using a lot of reinforcement learning, goes a little bit slower. Is compared to the open source world, closed source now ahead of what we're seeing from

SPEAKER_131: open source, or is this just kind of a give and take of progress, if you will?

SPEAKER_112: I mean, the closed source guys are still ahead. There's no doubt about it. I mean, OpenAI is, they have to stay ahead of the curve.

SPEAKER_00: And as you pointed out, the half-life of a model is actually quite short. It actually, it speaks to the economics of this whole industry right now, by the way, is that you spend a billion dollars on a model and it's only going to generate anything for maybe six months of any revenue. And so that's kind of hard, right? Like in terms of capital investments, I mean, we don't see this anywhere else. Like if I invest, you know, a hundred million dollars into designing a car, I can sell that car for five years. Yeah. If I can only sell it for six months, you know, the market has to be really big. So that is definitely going to be a hard thing for the industry to work through. And I don't know what the answer is, but in terms of closed source, closed source is still

SPEAKER_71: ahead, no doubt about it. Uh, meta is putting the investments in to make open source catch up. And what I think is interesting is that if you look at the L1 model, I actually don't think there's, there's a huge, um, uh, a huge step forward on the model itself on the LLM. It's actually more of the orchestration of that model, how that model, uh, self-assesses and comes up with better answers because the way these models work, uh, they actually come up with probability distributions of outputs. So if you look at, if you think of it that way, like there's a, there's a probability

SPEAKER_00: that the output is incorrect, but the mean of that probability is, is correct. So you have to kind of do multiple generations many times to get to the right answer. And really that's what we're, we're seeing here. And actually it's, it's interesting that open AI who has the most resources thrown at this has gone back direct instead of like saying, I'm going to make the output probability of, uh, uh, to be correct. It's like, no, no, no, I'm going to think about retries and doing this over and over again. So it's, it, it's a peek into the direction of where things are going is this sort of idea of orchestrating the model to, you know, have retries and, and, and get to a more correct

SPEAKER_84: solution through iteration.

SPEAKER_15: Help, help. I can only get so bullish about AI. Okay. So this is super cool. So essentially what you're saying is the core technology is still advancing, but what open AI has done in this case is use what currently exists kind of recursively to improve the output, and this is the reinforcement learning elements, correct?

SPEAKER_00: Uh, no, not exactly. So, uh, this is all done at the inference time. So inference time is basically, oh, I see the model is somewhat static. So you're basically asking the model produce for me multiple results and have some way of judging the quality of those results and then saying, ah, okay, this is a better step forward than that one was.

SPEAKER_71: So if you think about what an agent is like a human agent is you're kind of thinking through multiple steps to solve a problem, you break a problem down and each one of those steps has to be correct. So if you're incorrect in one step, it's sort of taking you on a bad path. So you need to assess, self-assess. Did I make the right call on this step?

SPEAKER_136: I think I did. Then move on to the next step. And really that's what's happening now.

SPEAKER_15: How far can we, can we push this point about orchestration to improve model output quality? If we can get gains from orchestration, does that road have a long way to run as in, are we just scratching the surface of what we can do with that?

SPEAKER_138: Or has O1 perhaps kind of shown the limits of that new technique as a way to get better results out?

SPEAKER_49: Oh, no, this is just the beginning. And the idea that you can break a problem down is not new, right?

SPEAKER_00: We actually wrote a blog on this, I don't know, six, seven months ago about compound AI systems. And really the idea here is if you look at any engineered system, let's call it, I don't know, the original coding for computers, right? We would write at the assembler level and it was this big monolithic blob of code, probably completely unmaintainable. If you think about it, like by today's standards, it's like, oh my God, why would we do that? Because we didn't have abstractions then. And so we sort of thought about this big blob of code as completely non-introspectable. It was hard to maintain. Over time, we moved to a place where we could modularize, we could break down code into functions, objects. Each of these things can be independently verified and then strung together. And so advancements in programming languages allowed us to do this. We're going to see the same thing in large language models. Like first, we start out with this giant blob that just does something and it proves the concept that we can make something work. Now we're moving to a point where it's like, well, hang on, if I'm an enterprise, I actually want to modularize this. I want to say, maybe I want a language front end that can parse multiple languages or multiple modalities.

SPEAKER_71: Maybe I want a reasoning engine that understands how to reason over a particular kind of data. Maybe I want a backend that understands when I should be calling to a database or calling functions. And then I want something that assembles all this stuff, creates an output that's formatted in the way my users need it. Maybe that's JSON. Maybe that's English. Maybe it's something else. So all of these pieces come together and you got to orchestrate them and you want to independently verify them. And this way we move to a much higher reliability. Instead of talking about this giant monolithic blob, now I have independently verifiable components

SPEAKER_00: that I can actually make really good and I can find bugs and I can debug the system.

SPEAKER_15: Okay. So one thing that I saw maybe five years ago, and I loved this, was a lot of companies thinking about no code and low code programming, essentially abstracting one layer above actually typing code, connecting things, moving boxes around, et cetera. As someone who hasn't really written code since C++ in high school, I loved that. It's fantastic. I felt like I had superpowers. In the world that you're describing, is this a future in which people who are not hardcore developers will be able to set up these different models and functions like you're describing?

SPEAKER_147: Or will this, do you think, remain the remit of the engineering department, even in, say, 83 years?

SPEAKER_00: I think it's going to remain in the remit of the engineering department. Damn it! The reality, I know, it's just, I mean, it's one of these things, I think coding assistants

SPEAKER_71: have really opened the aperture of who can play with these things. I think it's, there's a certain level of complexity you could express that way. Human language is naturally kind of imprecise. You know, we talk about like coding as if it's a bad thing, but it's just a very precise way to describe a problem. That's all it is.

SPEAKER_00: There's nothing inhuman about it. It's just a more precise way to describe it in English. Like, I actually don't want to program in English. I want to program in a programming language because, once I'm facile with that, because it's more precise.

SPEAKER_71: I know exactly what the effects are and, you know. So, if I think about it like that, we need high levels of precision. I need the model to behave in a certain way under certain conditions.

SPEAKER_00: I need to describe those conditions very precisely. I can't sort of hand wave and just say, yeah, go figure it out, right? I need to actually work all the way down to precision.

SPEAKER_38: Well, I suppose my question then would be how long until those parameters you have to set or rules you want to set up or tests you want to run are entirely done in natural language versus code. Because at that point, it would open the aperture quite a lot. But it sounds like, frankly, not soon.

SPEAKER_112: I mean, look, I think there are patterns that we see today that are common.

SPEAKER_71: Let's call it 80% of the use cases can probably be captured in, you know, some kind of pattern that I can call from English. And so, I can get most of the work done with English. But then there's going to be that long tail of stuff that needs to be really precise. And I think there's just no shortcut. You've got to be really precise. I mean, it's like a legal document. It's the same thing. Yes, it's written in English, but it's not really English. It's very precise language that's meant to describe something, you know. And it could be English, but, you know, it's still got to be precise.

SPEAKER_30: Does that make legalese then a very high-level, object-oriented programming language? It absolutely is.

SPEAKER_71: No, it totally is, right?

SPEAKER_00: Actually, there have been attempts to make it into code where I have strict syntactical requirements and things, which actually enforces potentially correctness and verifiability, you know. This is a programming languages topic that's been around forever, you know.

SPEAKER_154: I suppose, though, if we do then use AI to automate the legal industry out of most

SPEAKER_15: of its employment, we can then just take the lawyers and turn them into developers because they'll already be familiar with the style of communication. There we go. Problem solved. Labor market fixed, everybody. You heard it here first. I want to talk about the enterprise, though.

SPEAKER_105: And one reason why is I am a little bit muddled about agentic AI.

SPEAKER_15: And I know you guys have talked about recently the Mosaic AI agent framework and agent evaluation services. Sierra by Brett Taylor is working on agents inside the enterprise.

SPEAKER_60: To me, it's become a bit of a catch-all term, if that makes sense.

SPEAKER_15: And so I'm kind of curious to me. How do you slash Databricks define agentic AI? And then I want to dig into that a little bit. But let's start there.

SPEAKER_00: Yeah, you're right. It is a catch-all term. It became the new cool thing to say. And it's not well-defined as to what it really is. I mean, I would say, well, what the term, how the term came about was building something that can work through a set of steps, you know, some sort of a workflow, uh, break a problem down and, and, and actually work through multiple steps to get there. So, you know, we see this all the time with, uh, uh, RPA, uh, robotic automation.

SPEAKER_158: Robotic process automation. Yeah.

SPEAKER_160: Okay. Everybody, you know, I love newsletters. I love sharing knowledge and what I'm up to through my newsletter. And if you've got a newsletter, you got to check out Beehive. It's spelled B E E H I I V. I use it for the twist ticker and our twist 500 newsletter. My team is raving about Beehive because it is an all in one platform that not only powers our newsletter, but it's got all these incredible platform features that are helping us grow our subscriber base, which is what these newsletters are about. Listen to this Beehive's co-founder is the same person who helped Morning Brew reach millions of subscribers. In other words, they took all those tactics and they put it into a platform. They've got a great feature. I just want to tell you about it's called the AI post builder. This makes writing easy. That's a great writer. Sometimes I need inspiration. Well, they will get inputs for your ideas and then shape and optimize your content for the maximum impact. It's perfect for busy founders, right? And they're available 24 hours a day. They also have a referral program that turns your audience into ambassadors. It works great. Plus, Hey, if you want to monetize, they got an ad network and you know what? It's super affordable starting just at $39 a month. So here's a great call to action. 30 day free trial plus 20% off your first three months. Go to beehive.com slash twist.

SPEAKER_164: That's beehive.com slash twist. B-E-E-H-I-I-V.com. What a great product.

SPEAKER_71: Yeah. So we want to say like, here's a common pattern I do over and over again. Maybe it's like, I have a bunch of data in one file. I cut and paste that. I drop it into an Excel format. Then I go and apply this formula.

SPEAKER_00: Like it's a common pattern we see over and over again, but it's just very hard to automate. Yeah. So I think that's what the RPA stuff is all about and agents were kind of a way to say, well, maybe I can have it observe what a person does and then learn. That's where this started. I think we're not quite there from the perspective of I can just turn it loose and be 100% correct. But what we want to work toward is doing that within Databricks or other data platforms because these are the kind of workloads that dominate inside of Databricks. People do, you know, ingest of data. They do some sort of transformation and then some sort of dashboarding and they can make some kind of decision. Can we start to automate some pieces here? Really, that's the overall vision.

SPEAKER_169: We have a few different ways that this looks today.

SPEAKER_100: Okay. I want to pause and talk about this because it sounds like when you describe breaking up a problem into smaller pieces, taking steps.

SPEAKER_15: Honestly, that reminds me of going back one step. What I see with O1 when I ask to show me its logic. So, to me, it feels like the agentic AI you're describing is kind of like what that model is already doing. And I presume here that I'm confused and I'm oversimplifying things. So, can you just set me straight?

SPEAKER_00: No, conceptually, you're right. And I think what the big difference is between that and what we're doing is the idea that I want something that's really, truly bespoke and built for my process, my data. Built from my data and the learnings that I have within my company. O1 is great. It's a concept and it works well under the conditions where they tested it. But you can break it pretty fast. I mean, if you just go on Twitter and look, there are lots of people that broke it and made it do stupid things with really simple questions. And I think this shows that to make these kind of systems work, you do need specificity. You need to work from your data.

SPEAKER_71: Your data is the ground truth of what should really happen. It's not a general problem. And, you know, this is not a foreign concept. If we look at, like, humans, the same problem. Like, if I got generic smart kid out of college and said, go solve this problem that I have 20 years of institutional experience around in my company, they're going to fall apart, too. They're not going to get the question right. Maybe if they're really smart over time, over the next couple of years, you could train them to be really good at it. So, I think this is the same thing we're seeing here is that we see the value in building things that are bespoke and experts.

SPEAKER_00: Within a particular domain, we call this concept data intelligence, where you can use your data to create intelligence, to actually automate things and make it useful for your business.

SPEAKER_15: Yeah. And I actually, I watched a clip you guys sent over about the difference between general intelligence, which is essentially a large model. It can do cool stuff, and then data intelligence, which is that model applied to your existing corporate data set. And this, by the way, is always why Databricks, Data Lakehouse, plus Mosaic ML made sense to me, because why not have a brain on top of your bucket of data? But at the same time, in that same post about the Mosaic AI agent framework, you said, or your company said, sorry, while building a proof of concept for your Gen A application is relatively straightforward, delivering a high quality app has proven to be challenging for a large number of customers. And I'm curious if that issue is around the data they need to have, perhaps cleaned and ready to be put into the model, or if it's something else that's making it difficult to turn a bucket of cool data, cool models into an actual product people want to use inside their company day to day.

SPEAKER_00: Yeah, I mean, there's a lot of things to pick on here. So data quality, data cleaning, all of that. Absolutely. You need to get your data in very good shape, because if you're going to start calling this data the ground truth, you need to make sure that's really true. So, you know, bad data leads to bad outcomes. That's true for humans, too. If you give someone the wrong information, they're going to do the wrong thing.

SPEAKER_15: Sometimes you give them the right information, they did the wrong thing, or the wrong information, the right thing. Humans are, bro, humans. We can talk about that a lot, too. I know.

SPEAKER_180: Let's not get bogged down. Sorry. It's an election year, so I have a lot of thoughts about that. But let's start. Back to you, Naveen.

SPEAKER_117: But humans are actually still quite amazing at understanding causation and how to get to an end goal, which we haven't quite cracked yet in this field.

SPEAKER_71: When we look at some of the problems I've seen from different companies is, you know, I would characterize the second half of 2023 and the first half of 2024 as every Fortune 500 is like, oh, my God, what's our Jai strategy? Right. Everybody had to have this. I've talked to boards of large companies, and they all said they're asking their management, you know, what is your strategy?

SPEAKER_00: And the strategy was go get an open AI account. Right.

SPEAKER_189: And I think, you know, people. That was my strategy, too, Naveen. And I mean, their defense.

SPEAKER_03: And that's fine. You know, I think it's a place to start. But really, you need to understand what success is. This has to have a business impact.

SPEAKER_71: Like just putting a new feature on your banking app may or may not have any effect on your business. It has to be something that your users want, that they get value from, and actually use your platform more or do more banking or do more loans or whatever it is. So actually thinking through that is still a business strategy question. It's really not an AI problem at that point. It's really, is this a good thing for my customers? Perhaps at some point, AI will be able to tell me that. Today, that's not really true. Okay. So now we can say, well, I've identified a problem. Great. Now I want to think through what is success criteria. So I can take components that are in Databricks and I can start to assemble them. I can get to a working demo pretty fast, as you said. And that demo might have some kind of performance attributes to it. It's going to break in certain places. Oftentimes, we find that this looks cool from a demo perspective. But if you actually go and try to roll it out, you'll see all kinds of degenerate use cases, all kinds of places where it breaks. Okay. So let's characterize that. Let's build an evaluation and give it a score. This is a 37% correct for my, for my application. Or it's 30%, 37% useful for my users. Like, start to score this stuff and evaluate it. Once you score it, now you're in a place where you can actually start to improve it. We have all kinds of tools that can say, okay, well, let's try this to see how it moves the score. But you need to get to that point. And I think this is actually the blocker for most of these applications. Beyond all the plumbing stuff of security and governance, you need to get all that right. So I think there's a lot of steps that are kind of in feeding progress for enterprises. And we're trying to break down those steps and make it systematic for our users.

SPEAKER_38: So I'm familiar with the CEO of a company called Skyflow, Anshu Sharma. He's a friend of mine and likes to explain things to me. And one thing that he was early on was, you know, helping corporations keep PII safe. And then the Gen AI boom came and everyone was talking about putting the data to work.

SPEAKER_30: And I presume that there is the same kind of tensions between people wanting to keep their data properly permissioned and only the right people can see it.

SPEAKER_15: And I can't ask about Naveen's work data and so forth. Has Databricks and other vendors solved that issue?

SPEAKER_178: Or are we still working out how enterprises can bring their data safely into an AI context today?

SPEAKER_43: Yeah, that's a great question. It's actually a big part of what we do at Databricks is we call this data governance.

SPEAKER_00: Yeah. The cool thing about Databricks is that we already had a framework for this. We call it Unity Catalog.

SPEAKER_71: And this is basically a universal source of truth of permissions and lineage and, you know, logs of what happened. If you're a regulated industry, you're a bank or a healthcare company, you need that. You can't actually go willy nilly and say, okay, here's some data.

SPEAKER_03: I'm just going to throw it into a model and, you know, hope for the best. Like, it doesn't work that way, right?

SPEAKER_131: That's a one-way ticket to sitting before Congress being told that you're a bad person.

SPEAKER_03: Yes. And not actually not just sitting in front of Congress, like massive lawsuits and huge damage to your business. I mean, what's really interesting is that banks and other regulated industries, they work on trust. Their whole business is based on trust.

SPEAKER_71: Like, I'm trusting that that bank is doing the right thing with my money when I put it in there. Otherwise, I wouldn't give them my money, right? So they have to be kind of conservative in this regard. And, you know, that's been a real big blocker to get this stuff out to their customers. What we're seeing today is they're thinking about a blast radius. Like, okay, I can try building a solution. Maybe I can expose it to internal users as a tool or an aid. Sure. And start there and then maybe work your way up to something that goes external.

SPEAKER_00: That will happen as we start to trust these solutions and be able to engineer them through all of the edge use cases. But really, this is a huge blocker. So what we've done is, inside of Databricks, we've integrated a bunch of stuff that came

SPEAKER_71: from Mosaic as well as a lot of things that have been developed here under the umbrella of Unity Catalog. So, for instance, if you fine-tune a model, that model is fine-tuned on some data.

SPEAKER_00: That data had some attributes associated with it. Maybe it had certain access controls. Those access controls need to be inherited by the model because that model actually, you know, is a function of that data to some degree.

SPEAKER_71: Or when you use a vector database and you do a pattern like RAG or Retrieval Augmented Generation, there may be data in that vector database that can be viewed by certain users and not others. When a user hits that endpoint, the vector database needs to be aware that, oh, this user can't see that data, so don't pull that data. So these things, they seem simple. They're conceptually not difficult problems, but to manage it at scale gets very, very complicated. And so we were able to leverage this actually quite mature framework, which we just open sourced, by the way.

SPEAKER_00: It was open sourced at the Data and AI Summit in June. And on stage, Matei, Zahari, who's the CTO and co-founder of Databricks, actually hit the button on stage that open sourced it. I thought it was kind of cool.

SPEAKER_200: No, no. I mean, I'm always here for pageantry.

SPEAKER_38: And why not push the button live? Was it open source onto, where did you guys put the code online?

SPEAKER_89: I think it's on a GitHub repo.

SPEAKER_47: Cool. I'll throw a link to that in the show notes if anyone wants to check it out. Absolutely.

SPEAKER_71: So anyway, I think the point is that we had a quite mature way of thinking about this in terms of data governance. And we've applied that now to Gen AI. So different components like fine-tuning, embedding, vector databases, function calling, all of these things live under the guise of this Unity catalog, which sort of takes away that security and governance plumbing side of things to unblock some of these use cases. Now getting to correct models that are useful models, all of this stuff is the next frontier,

SPEAKER_203: I'll call it.

SPEAKER_57: Okay. But listening to you talk about this and thinking about the issues that have come up and that

SPEAKER_15: have been fixed or at least worked on by Databricks and other companies to get enterprises from, we have an open AI account to, we're using our own data in production for a feature that's either internal or external. It seems that a lot of the pathway has been smoothed. So has that helped a lot of companies go from, we have a generic strategy, which is we're tinkering, we're playing so we don't fall behind, but we're not really in production yet, to, this is something that we've delivered both to customers and also our employees.

SPEAKER_133: Yeah, they're getting there, right?

SPEAKER_00: This is still so new. I mean, remember, ChatGPT still was not even two years ago. We're coming up on it in a month. It was November of 2022.

SPEAKER_03: So this is actually quite unprecedented how quickly the industry is moving, right? Two years is amazing to actually even get a POC going on this stuff.

SPEAKER_12: Well, that, that, that is, that is, yes, that is enterprise speed. If you're curious why it took two years and that's fast to get a proof of concept, go work

SPEAKER_15: for a private equity-owned company for a while and you'll find that two years to do anything is fantastic. So I hear you. It is. It is.

SPEAKER_71: So we are, we are starting to see that now and, you know, having conversations, you

SPEAKER_00: know, almost daily with these, with enterprises about how to do this and how to do it well. And what are some of the best practices we've seen having these governance, uh, tools out

SPEAKER_71: there? Like we launched this stuff this year. These are, these are within six months. We've launched a lot of, a lot of these features and, you know, you need to be a trusted partner.

SPEAKER_00: Like I know there's a lot of startups building cool stuff and absolutely continue building cool stuff. There's no reason not to, but you know, it's hard for a big company.

SPEAKER_117: Like, uh, I don't know, but name your big favorite bank, you know, they, they can't take

SPEAKER_212: and use favorite bank. I mean, pick your favorite super villain.

SPEAKER_144: Yeah. Who charges you ATM fees? Chase, I guess. Okay. Sure.

SPEAKER_215: But yeah, take a chase. Like for them to use a startup is quite hard, right?

SPEAKER_71: I mean, they're just, just getting a tool approved. Maybe they can get some POCs going, but they need trust in their vendors. And, you know, Databricks has already built this trust, so putting these features in on top of a trusted platform they already have, it has actually been quite transformative. Our business is growing very fast. Um, and you know, I can't talk about the precise numbers, but you know, it, it is, it's, we're seeing a lot of uptake.

SPEAKER_196: I've written a lot about Databricks's growth over time. And this is actually a beautiful segue to what I wanted to ask, which is before.

SPEAKER_15: Gen AI became a household name. The thing that everyone talks about in the world of technology, Databricks was growing very quickly by itself in the, in the pre mosaic ML days, if you will. Yes. Now, of course, when you pull up Databricks website, it is the data and AI company, clearly an emphasis across the branding and really let's be honest, the industry. So now when new customers are coming in to the Databricks world, are they coming in because they're more interested in the, the lake house product data storage and BI, or is the AI

SPEAKER_207: now a leading vector by which you guys land new logos and accounts?

SPEAKER_112: Uh, I would say it's a combination of the two, honestly, because we still see a lot of companies

SPEAKER_00: out there who are struggling with, uh, migrations of their data platform. I mean, again, going back to enterprise speed, I know it sounds like, you know, things that

SPEAKER_71: were like 10 years ago, but these are still problems from 10 years ago. Like these legacy systems, I, I actually talked to a company, uh, not in the U S but, uh, uh,

SPEAKER_00: in Asia that was literally still running on, you know, old IBM mainframes from 20 years ago. And they're like, how do I modernize? I mean, I know this sounds silly, but like, this is how these industries work, right?

SPEAKER_47: And that's how COBOL programmers have a hundred percent employment right there. Exactly. And actually they're, they're getting paid a lot. Yes.

SPEAKER_38: Because that's actually, that's my backup plan to journalism, by the way, is if this all falls apart, I'm going to go buy a couple of books on COBOL and just nestle myself deep in someone's basement.

SPEAKER_224: It's going to be great. Yeah. Take a year, go learn it really well. I think you'd have, you know, great career prospects for the next 10 years.

SPEAKER_117: Uh, so I think, uh, we are seeing a combination of companies who want to leverage their data

SPEAKER_71: for AI. And this is, this is squarely the vector that we, we are winning on right now is that we are the combination of those two things in it.

SPEAKER_00: And frankly, it's not something that's new to Databricks. In fact, Databricks started there. Spark started there. It was actually the, uh, Netflix challenge. Uh, if you remember this thing, I think from 2012 or 2011, where Netflix published a bunch of data and they said, okay, go build something that can match, you know, preferences of our users to movies they might want to watch. And they published a big data set. So, uh, Matei and others actually made spark in response to that because they want to scale it out over a whole bunch of CPUs.

SPEAKER_71: So, uh, you know, AI has been something that's been front and center for Databricks.

SPEAKER_226: In fact, many people use Databricks for AI and then use other data platforms. Uh, they connected it. Yeah.

SPEAKER_38: Okay. I want to touch on that because one thing we've heard so much about, uh, at an industry

SPEAKER_15: level is the need for more compute to avoid being GPU poor. How well do you know Jensen? Will he send you H 100s? But you know, prepping for today's show, I'm seeing Databricks on GCP, Databricks on Azure, Databricks on AWS. And you guys, you know, you'd mentioned you're not working on DBRX too.

SPEAKER_207: So does this mean that Databricks doesn't need to build a hyperscale network of data centers around the world? And if so, is that a risk to the company?

SPEAKER_68: Well, we are, we do not build data centers. So we never have, uh, we run on top of the cloud. We're a software company.

SPEAKER_00: So being GPU poor to be rich, we actually do still a lot of fundamental research. So we still have a sizable footprint of GPUs. Uh, but we, we're not building those open source models. It doesn't mean we're not doing other work that's on the internal internally facing side, externally facing. We're running lots of inference. We're seeing inference kind of take off for these models. I mean, as we get richer use cases, things like agents that we've been talking about doing multi-step, like there's a ton of pressure on inference, making inference really fast, making it really cheap, making it very high quality. Uh, all of that takes GPU compute. So I don't see the world where we are not doing a lot of stuff on GPUs. We're just going to have more and more of this going forward. And, you know, we're, we're expanding into multiple geos. We're not having everything in one place now. So I think the, the pattern of deployments of those GPUs is maybe different. Instead of having one giant monolithic cluster, we're breaking things apart. But the numbers I don't see going down anytime.

SPEAKER_38: I worry about platform risk because you guys are dependent in a way on these major scaled cloud platforms.

SPEAKER_15: And at the same time, Microsoft is making its own AI models. It has its own PI tools. Yeah. They don't want to share right now. They'll be besties with you because it's convenient for them. But it just, you guys have such an ability right now to raise capital at whatever price you want. I mean, so to me, I'm just kind of curious why not offer a data bricks cloud alternative to these big tech companies who I don't think ultimately want you guys to win.

SPEAKER_237: Yeah. I mean, I would say you should ask my boss about that one, but tell him to text me back.

SPEAKER_239: I mean, geez, come on.

SPEAKER_117: No, I mean, I think all possibilities are there in the future. We'll see how things go.

SPEAKER_00: I mean, what we have done is establish a particular point in the stack of trust. So we, we do own this, these abstractions for data. Now we're owning a lot of these abstractions on how to take that data and make models that are really great. Yeah. And, and how to deploy them. So when you have that abstraction, you can move down the stack. Typically moving up a stack becomes very hard.

SPEAKER_71: So you see this, if you're a hardware company moving to cloud becomes hard. Yeah. If you're a cloud company, perhaps moving down to build your own hardware is less hard.

SPEAKER_68: When we see this, every cloud company is building their own inference chips now. Yes. So, you know, how's it going to run in the future?

SPEAKER_131: Pause. That does not include data bricks, just to be clear.

SPEAKER_244: Correct. I mean, we don't do the cloud, so we're not doing hardware.

SPEAKER_38: I'm just making sure that when you talk about these other companies, it's not you guys. You're not making your, Google is Microsoft is meta is everybody else.

SPEAKER_245: Okay, cool.

SPEAKER_71: That's right. Exactly. So how, how are things going to shake out in 10 years? I don't know, but right now, I mean, there's a, there's a good reason for the clouds to work with us.

SPEAKER_00: We drive a lot of experimental revenue to them. We, we store a ton of data and now we have the tabular acquisition, like lots of data in the cloud is on our formats.

SPEAKER_03: So, you know, that gives us a pretty good control point. Um, and you know, the clouds are, are, are incentivized to work with us. At some point, maybe that's not true.

SPEAKER_15: Well, I think what I'll do is I'll just go start buying small servers and then plugging them into the back of your guys's HQ and then slowly build up a cluster that you guys can then offer as, as pass. Uh, to me, I, I don't know.

SPEAKER_38: I just, one of my things that I'm most scared of right now is that the largest tech companies, the ones that are the hardest to knock off that have the most market. Wait, are also the companies that own the major cloud platforms.

SPEAKER_15: And that to me makes them so monolithic and hard to kill that I'm worried that we're going to see less innovation over a long timeframe over many different, you know, technology cycles. But it just makes me a little scared and I know there's a lot of money to build a hyper skilled cloud. And so there's only so many companies that can do it. And so the fact that you guys currently don't plan to just me as an individual that bums me out.

SPEAKER_38: Although I understand from a business perspective, how it would be frankly distracting to you guys right now.

SPEAKER_207: I just, I hope, I hope that in five years you're, uh, showing me around, you know, data bricks data center number nine or whatever it is.

SPEAKER_68: That'd be great. Yeah. But I mean, if you think about it on a long time horizon, uh, one of the largest incumbency events in tech happened in the late seventies.

SPEAKER_00: What are the two largest companies in the world today? Apple and Microsoft. They both started then and they both started with the PC. They've leveraged those that, that advantage over and over again through multiple tech transitions. And, uh, you know, it, it hasn't really slowed innovation, right? They've been on, they've been maybe behind, but they buy, buy into it or, you know, buy a startup that's, that's doing something innovative. So I, I don't know.

SPEAKER_148: I mean, I, I, I like that argument.

SPEAKER_15: I, I just wish that instead of the magnificent seven, it was the magnificent 27, you know, like just more competition.

SPEAKER_71: I don't disagree, but I mean, it's sort of like how these ecosystems evolve.

SPEAKER_00: You have to have certain capital to, to have that incumbency and, you know, they have it and it keeps going, right? It keeps rolling forward. Yeah. It's great for you guys.

SPEAKER_15: Cause you can work with all of them at once. And therefore no matter where your customers are, you are as well. So I get that. There's a lot of talk about small models versus large models. And my impression going back to the models you guys made at mosaic is that quite often we can see a 7 billion parameter model be as good as a larger one. If it's well tuned. I just, what is the mosaic slash databricks perspective on corporations using larger versus smaller models today?

SPEAKER_68: Yeah. I mean, I think it's about building the right thing for your application. Do the thing that matters.

SPEAKER_00: If, if latency really matters, go small. If quality really matters, maybe go bigger. But one thing you'll see as a trend is literally no one has built a model bigger than PPT for that happened. I don't know, 16 months ago, something like that. And that model is still the biggest one out there. And even open AI has been building smaller and smaller models. You have GPT 4.0 and 4.0 mini. 4.0 mini is probably less than 10 billion parameters. I don't know with specificity, but it's very fast. And we can look at, look at that as kind of a metric for how, how big it is. And yeah. And I think we're seeing that they're able to extract higher levels of quality from a smaller model.

SPEAKER_71: So the science is by no means done here. There's a lot more work to do. There's a lot more we can wring out of smaller models. And then chaining these smaller models together as in these compound AI systems is the way forward, right? That's how we're going to do this economically. That's how we're going to do it in a modularity fashion where we can very independently verify each module and build end-to-end systems that are very reliable.

SPEAKER_107: Well, then do large models then exist as a way to help create smaller derivative models? Yeah, that's, that kind of had to happen.

SPEAKER_71: It seems like we built this giant model and then we said, oh, well, we can actually take the outputs of that model and actually start to modify smaller models to have better performance. And so, you know, Microsoft did a bunch of work with the five models that they published.

SPEAKER_00: Those are like one in 1.3 billion parameters and they're very high quality. So we're seeing that trend over and over again, where I can, it's not distilling. I'm using synthetic data generation from a bigger model to basically bootstrap and train a smaller model and it works. So maybe we need to go that route of building something really big to go and build something really small. That's high quality. Okay.

SPEAKER_105: On the synthetic data point, I wasn't going to bring this up, but you brought it up so I can blame you if we go over a little bit.

SPEAKER_15: Synthetic data to me makes a lot of sense, but always just based on the way my brain thinks, considers it always to be of slightly lesser quality than non-synthetic data. Is that a bias I need to get rid of in my thinking?

SPEAKER_47: I wouldn't say get rid of it yet.

SPEAKER_03: It's kind of, I don't know if you remember that movie Multiplicity where the guy makes a copy of himself to do more work. It's actually kind of a funny movie is basically make some copy of himself to like have one at work and one at home. And then he makes another copy and a copy of a copy. Michael Keaton. Michael Keaton. That's right. Yeah.

SPEAKER_38: I've not seen this movie, but the, the, uh, it looks hilarious. So that's going on the family list. Thank you very much.

SPEAKER_117: Yeah.

SPEAKER_71: So anyway, it's a little bit like that. I mean, you, you, you start to become more, uh, uh, unbound from the real distributions as you make more copies and more synthetic generations of it. It's like, you think of it as making mashups. Some of those mashups are not accurate reflections of the real world. We call this concept model collapse. If you train a model on a bunch of these synthetic objects, maybe you get something that actually doesn't give you anything useful. So far we're seeing it as a good way to augment models and actually allow them to, to explore the distributions of the real world. Uh, if you can keep tabs on keeping quality and being grounded. So I think we're, it's a, it's a useful technique. It's not something that's going to solve everybody's problems. I still think we have some major issues in the field of AI. We've gotten very brute force, you know, 20 trillion tokens of data to train a model. That's 20,000 human lifetimes of text. This is not how, how humans work. I mean, humans and animals actually use a lot less data to get much higher quality, uh, causal models. So we have a lot of science.

SPEAKER_00: I still think to do, to build, to build these things, uh, economically and, you know, very low power. Your brain runs on 20 Watts of energy. A mouse runs on 800 milliwatts and they do amazing things.

SPEAKER_38: No, that's one of the most encouraging things that I've seen. I've seen the people showing like the, the brain Watts versus data center Watts and how many, you know, equivalent flops you can get and so forth.

SPEAKER_15: Do you think that eventually, and I, this is completely off topic. So this might be a stupid question to be, but like there was a, there was a, an effort. I read about this in like business week, like 10, 15 years ago to replicate the human brain thinking that that is the neural network that is so efficient, can do so much. We should do it. We seem to have kind of moved away from that in the transformer revolution.

SPEAKER_273: Do we bend back towards biology over time? So, okay.

SPEAKER_00: Those are like the human brain project and stuff like that. I think that's what you're referring to. I mean, I'm a neuroscientist and I thought those were like very poor facsimiles of what, what was going to lead to intelligence. So it's because you're trying to replicate something without understanding why that thing exists. It actually becomes, you know, you get one little parameter wrong. The whole system falls apart. These are delicate systems. So I do think going down the path of trying to extract principles makes sense. Now, I do think that looking at biology still is a, is potentially a useful path forward. There is a different dimension to more data that we are missing currently in these systems. They're not, they're not trying to understand how to make better decisions and self critique. Maybe we're getting there now with this kind of like this, this chaining idea. Uh, but you know, our brains have multiple systems that actually, um, interact in, in very interesting and precise ways. So you may have read, uh, Daniel Kahneman's book, thinking fast and slow. Yeah. Very, you know, this is, this is a great book. If anyone hasn't read it.

SPEAKER_71: Um, it actually has roots in neurobiology. There's, there are systems that are very old in our brain that kind of make learning without parameters, without, without, without a model. Then there's these other systems, which actually try to simulate different realities and then make good decisions. We use both of these things in concert and systems we build today. Like the AI systems don't really do this. They don't have a good grounding of this is a real representation of the world that I can verify. They're just sort of like almost pattern matching against, against things that we're seeing in the training data. So am I saying we're on the wrong path? Not necessarily, but I think there's a lot of work to do that. We can take some, some inspiration from biology without direct mimicry.

SPEAKER_38: I think the biggest takeaway from our conversation today for me is the idea that there's still so much improvement coming in our ability to build intelligent software systems. Like the orchestration point from 01, what you just said about this.

SPEAKER_207: I mean, this to me feels like a, we have 10 years of incredibly quick development ahead of us. And we're going to end up with, I mean, something truly marvelous at the end. That's absolutely.

SPEAKER_68: Yeah. Look at, look at the internet, right?

SPEAKER_00: I mean, I was a, I was a freshman in college when really the first web browser came out in 1993, all the basic concepts of what the web was going to look like sort of existed then. But look at what we have now, we have cloud computing, we have, you know, ways to do live updates and, you know, applications that run within a browser. None of that stuff was really contemplated at that time. Yeah.

SPEAKER_71: And, you know, it was, again, went, went from this big monolithic thing that sort of worked till now we can engineer modules, make things work really interactively. And we created a set of systems that are highly reliable companies like Google basically figured out how to build resilient infrastructure. This needed to happen to have like, you know, nine nines up time, you know, all of this engineering has to happen. And we're just at the beginning of this chat GPT was two years ago.

SPEAKER_270: You know, we've got 10 years of innovation left ahead of us to make these things really, really amazing.

SPEAKER_38: I think you should point to that sign that says chat GPT came out less than two years ago. Every time someone complains that AI hasn't solved world PC, I'd be like, all right, it's been 22 months. Everybody calm down. Actually, I just realized that my, my, my first child is basically as old as chat GPT. Huh? That's going to be a really weird milestone to have in my life.

SPEAKER_15: Okay. One last question before I let you go intervene. A lot of startups out there working on AI drive solutions, services, lots of people building cool stuff in AI. Databricks has been acquisitive. It's how you joined tabular, of course, was a big deal. I'm curious about two things. One, how often are AI founders calling you up very politely? And also just how interested is Databricks today in doing more, perhaps smaller tuck in acquisitions of AI focused startups?

SPEAKER_93: Yeah.

SPEAKER_71: I mean, I, I do angel invest and, you know, I, I typically try to pick companies or founders that, uh, I feel I could, I could be useful to them as an angel investor. I kind of see myself as a de facto advisor or helper whenever I can. And, uh, you know, and things that I find interesting, you know, if it's a new cool technology, all right, let's, let's, let's give it a go. I, I'm really interested in places where we can apply AI to actually solve vertical problems. Uh, healthcare, healthcare is actually a very important one to me because I think it's something so fundamental to what it means human. And we have the pieces to actually go make healthcare accessible to many people, but it still hasn't happened. There's a lot of structural reasons, things like that. So let's get to that. Uh, so I like to invest in these kinds of things. So yeah, I, I, I do get called up quite a lot and I'm happy to talk to any founders who are building something cool. So there's that. Uh, sorry.

SPEAKER_271: What was the second part of the question?

SPEAKER_38: And just how acquisitive is Databricks going to be say in the next 12 to 18 months? A lot of companies have been built.

SPEAKER_60: Many of them probably aren't going to make it to the next level. So how many do you guys want to tuck in?

SPEAKER_00: Yeah. I mean, we're always open to these tuck ins. We've done a few of them. We're always looking. We have a really good, uh, corporate development team. That's basically scanning, finding, uh, places that fill in holes that we have. So we're always looking at this and, but we're, we're very thoughtful about it. We don't want to buy something just because, you know, um, it looks cool. We're going to look at it like, okay, how is this going to work strategically with what we're building? Does it solve a problem that we don't have a good solution for today? Does it have a customer base that we want access to something like that? There's gotta be some reason. So, uh, we're pretty thoughtful about how we assess it.

SPEAKER_71: And when we bring these companies in, I mean, uh, some of our agent evaluation stuff has come from a tuck in that we had a company called lilac AI.

SPEAKER_00: They actually created a really nice UI for, you know, uh, embedding data and visualizing it. We've actually morphed that into something that is going to be much bigger and very impactful for our customers. So we are absolutely interested in more tuck ins and, you know, we're growing fast. We're growing. Our revenue is growing more than 60% year on year. Uh, and so I think we are going to be one of the major players for these, uh, companies to go find an exit toward and, you know, keep in touch with us, build a relationship with us.

SPEAKER_15: You literally just threw the, we're growing fast 60% growth. I was not going to bring up you guys going public, but I mean, dear God, you just, you're demanding it. So I, I know the answer is we'll do it eventually. Not yet. Blah, blah, blah. But like H1 2025, like if you guys don't do it, then I think I'm going to submit like my tears in a vial, like, come on, give me the filing. So just so you know, that's coming next summer. If you guys don't pull the trigger, but Naveen, thank you so much for coming on. We appreciate it. We'll have you back in another 250 episodes or whatever it was the first time. And, uh, where can people find you online just before we go? Yeah, I'm on Twitter pretty actively.

SPEAKER_00: So Naveen G Rao that's N A V E E N G R A O is my handle there. Um, I'm also on LinkedIn, of course, so you can find my name there. But, uh, yeah, Twitter is usually where a lot of the AI stuff happens and where we have a lot of cool discussions.

SPEAKER_203: So, uh, follow me. Absolutely.

SPEAKER_15: For everyone else, more interviews, more live news, more twists coming your way. I'm Alex. I'll see you soon. Bye. Okay.

SPEAKER_295: Everybody. Welcome back. It's time for a jam with J Cal session. What's a jam with J Cal session.

SPEAKER_296: This is where a startup founder tells me what they're working on, who their customers are, what their product does, what they're trying to achieve, essentially their vision for how they're going to change the world. And then I asked them, Hey, what's your biggest challenge? Are the things you're struggling with? And then using my experience, having invested in 400 companies and taken well over 10,000 pitches from founders and having done 2000 of these episodes of this week in startups. Sometimes, you know what? I know where some of the bumps in the road are, and we have a dialogue about how to solve problems, which is what startups are all about. So with me today is a gentleman named Elliot Easterling. He's the CEO of a company called bonbon.tech.

SPEAKER_297: And that is our partner today. Dot tech is a great domain name. I use it for founder Fridays dot tech. Many people out there are using dot tech domain names because they're awesome. They let people know you're a technology company. And so thanks to our friends at dot tech for supporting this segment of the show. We pick all the companies that come on. Welcome to the show, Elliot. Thanks, Jay Cal.

SPEAKER_299: Thanks for having me. Appreciate it. I just want to walk you through a little bit of the business. I'll take you through a deck and then I'd like to jump into some questions I have about about some of our go to market.

SPEAKER_301: Review the deck.

SPEAKER_302: I like reviewing the deck. Let's see how you did.

SPEAKER_303: Great. Awesome.

SPEAKER_299: I'm the CEO, as you mentioned, of bonbon technologies, and we are a rewards platform for publishers, allow publishers to reward anything. And that ultimately drives a lot more engagement and much higher registration rates. So let's just get started. Really, the pain that we're trying to solve is for ad focused publishers, the 99% of publishers that are ad focused. And they've been really suffering from wave after wave of big tech changes, things like cookie deprecation, which started in Safari and now is moving to Chrome. Search results pages referring less and less traffic out social media algorithms referring less and less traffic out. And then now AI, which is going to have a massive impact as big tech platforms use AI to drive more engagement on their platforms and ultimately refer less traffic. So we're seeing as publishers are looking right now for solutions. They're planning for a smaller user footprint, but they also want to drive a lot deeper, more profitable customer relationships from those users that are on their platforms. And so we built basically is a fix by rewarding engagement. So we built the first published rewards platform that gives consumers relevant rewards, access to unique content, simple and transparent data and privacy controls, and ultimately a better user experience. Publishers get logins and logins re-enable cookies and lost IDs. They also allow the publisher to build direct relationships, which they're starved for in the world where they're mediated by these big tech platforms. We also give the publishers gamified engagement, a points program that drives things like revisitors, page views, watching videos.

SPEAKER_306: And ultimately that's all delivered five times more monetization per user. All right.

SPEAKER_296: You got me so far. It's interesting. Like I'm, I'm, I'm, I find myself nodding as a publisher. Just so we, we check in here as you run me through the deck. Yeah. Publishers do have these problems. And if their consumers are logged in and you know a little bit about them, you can monetize them better and people like gamification.

SPEAKER_310: So let's see how this works. Yeah.

SPEAKER_299: Cool. Yeah. And you're a media maven. So I think I knew you would, you'd really understand the business pretty quickly. So, so what we've sort of, we've cracked the code on sculpting consumer behavior with this rewards program. So we built an optimization engine that drives outcomes like 300% higher registration rates, 100% more engagement, 250% higher ad rates. And then we're finding that 54% of people after they log in will actually complete their data profiles and provide even a richer understanding to the publishers or who's visiting their site. And this is all at a moment where publishers need this most. And so let me just walk you through really quickly, kind of like a sample of the tech. Finally, I want to see the product.

SPEAKER_313: Okay, here we go.

SPEAKER_299: And so on the left, you see a little icon, a little tab. That's us. We control that. And that's all the user's navigation. They can access the rewards program. But on the right hand side here, what you see is that at any time through the process, we can allow a publisher to trigger a rewards window, either inline or as a pop like this, that essentially runs eight or nine offers simultaneously to figure out through machine learning. What do the users of that site care about most that will make them willing to register? And ultimately, this technology delivers like a three X higher reg rate from anything they've been doing prior.

SPEAKER_306: Okay.

SPEAKER_296: So for the audience that was listening, there's a little arrow on the side, like a little chip on your phone. If you were to click it, it pops up, says, Hey, would you like to win this television set or whatever, some kind of sweepstakes?

David Friedberg: If you log in, if you register, you automatically get updated. Yeah.

SPEAKER_310: Yeah, that's exactly right. You get entered into the contest. I love it.

SPEAKER_299: Yep. And then after someone enters or registers, when we verify their email, we actually gamify the registration process. Like we say, Hey, tell us more, a little more about yourself. Give us your name. 94% of people will give us name for more points. Zip code. 91% give us zip. Gender. 89% give us gender. And even they verify their phone number with us at 54%. And we're just getting started. The other things we do is we give people points for reading every article they read, they earn more points and that drives 100% more engagement. And just high, high level. We provide them with what we call three little parts of the platform. Open identity manager, which allows them to collect and manage first party data. This rewards engine, which runs hundreds of offers to try and give publishers the ability to reward anything. And then we have a bunch of front end tools that consumers interact with that basically sort of deliver the product. And we also have an API where publishers can call it to issue the rewards on their own.

SPEAKER_72: Very nicely done.

SPEAKER_297: Have you started to deploy this with, uh, any publishers yet? Where are you at in terms of building this business?

SPEAKER_299: Yeah, we are live on 27 websites. We have 60,000 bond members have actually registered. The one thing is one user logs into the publisher, they become a Bombard Awards member and also a publisher's first party data. So we've been able to build a, uh, you know, our file up to 60,000, uh, as of last week.

SPEAKER_324: Um, and we're running about 60 million monthly pages across our publisher network on. Great.

SPEAKER_326: You know, publishers are looking for tools like this.

SPEAKER_296: And if you can, uh, help them build their profiles and give them those gamification tools, I could see a number of them wanting to participate in this. The challenge, of course, is you're trying to solve a problem for publishers who are really struggling, which means they're not high growth.

SPEAKER_297: And so they might have very small budgets, right? And so that is an issue is that they're constrained.

SPEAKER_78: So has that come up and how are you picking your ideal customer profiles when? Hey, you know, the, the publications I used to run, whether it was in gadget or auto blog or the ones that Vox created themselves like the verge, et cetera. A lot of those publications are having challenges these days, right? Um, and so how do you think about that? Cause you're picking publishers, which are a group of people whose businesses are, might be flat.

SPEAKER_297: They might be, uh, contracting and they might be slow growth and they may not have budgets.

SPEAKER_330: Yeah. So we offer two solutions for enterprise publishers. We have a SAS platform, um, where publishers can pay a SAS fee.

SPEAKER_299: And we also have a free with ads version. Publishers have to have a minimum size to be able to qualify for free with ads, but we basically inject ads into all our models.

SPEAKER_324: And that essentially pays for the full program, including the rewards.

SPEAKER_296: Got it. So there's nothing to lose for those publishers by using this tool. And on the enterprise version, I assume I can opt out of like you owning the, uh, profiles and I, they're just for me. If I pay you enough money.

SPEAKER_299: No, the whole program is across publisher rewards program. That's sort of backed by bond bond. And one of the things we do is we issue, basically we, we issue the rewards out across publishers. Uh, okay.

SPEAKER_296: That's clever. So hold on. That's an important thing to pause on there. If you're going to give away this OLED TV that costs $5,000. It, that cost of $5,000. We'll get it abstracted across 10 publishers. So it's really net net $500 each, or if it was a hundred publishers, it would be $50 each, which is super clever. Okay.

SPEAKER_299: I understand. We have the benefit of that in that we basically can amortize the costs across publisher. Number two is sort of, we put the, the privacy guarantee. We allow the users to come into our system and opt out of any publisher they want. Uh, we know generally speaking, consumers don't like to do that.

SPEAKER_301: They just don't want to mess with the settings, but it is a really key important part of our value proposition.

SPEAKER_297: I mean, one of the other nice things here is, you know, the user can participate or not participate.

SPEAKER_296: So what I like about what you're doing is you might have somebody like me who looks at it, you know, you know, as a 53 year old now.

SPEAKER_297: And I'm like, yeah, I don't want anything for free. I don't want to be part of any sweepstakes. Uh, but I wouldn't might want you to know a little bit more about me, uh, as a publisher, because I want to get certain mailings, et cetera. So I have an email for just when I'm shopping that I use for my shopping websites. And I don't mind them knowing about me because I would rather get sales for men than women, or you, you, you get the idea, or I want tickets to Nick's games.

SPEAKER_296: Not, you know, I don't know, Miami heat games or Chicago bulls games.

SPEAKER_297: So yeah, that, that kind of personalization is a benefit because you don't waste my time. And then when I was younger, I might, if I couldn't afford the OLED TV, that's $5,000.

SPEAKER_78: I might very much want to join that sweepstakes. Thank you very much. So I think that's a very interesting approach too, is this data will help us personalize content for you.

SPEAKER_297: And then there's, Hey, this data might let you get into a sweepstakes, um, and game. You didn't mention the gamification, but that one was a highlight for me as well. So do you have an example of gamification yet or that's on the roadmap? Yeah.

SPEAKER_299: So one of the things we do is after people register, we sort of say, Hey, uh, and we deliver this to the email campaigns. We sort of send them a weekly email that basically says, Hey, here are some articles you could read that are personalized to you that earn you extra points. And so we're essentially starting to sort of like enable gamification through, uh, through our newsletter program. Um, and we do also offer point bonuses for doing things that go to this one publisher. And play one of their games and that will earn you a hundred points. So we're, we're early on in the gamification process. But I do think that that's going to be some of the critical things as, as sort of my promise of publishers is like, Hey, not only can I get you a lot more registered users than you could on your own, but these users are hyper engaged because through, through gamification. We can sort of sculpt traffic directly back to your site to spend time on the things that you care about and ultimately reward them for, for that behavior and that activity.

SPEAKER_296: One thing that's tried and true is inviting a friend or a friend member, bring a member kind of gamification. So you might find some interesting, you know, um, the engagement there from publishers.

SPEAKER_78: If you could say, Hey, I'm already a member. I think my brother should know about this content. You know, who else should know about this story? I enter somebody's email. It emails them. They click, they register. I get points. So that's tried and true. You see that in Robin hood, where you gift a stock, you get a stock Uber. You give a ride, you get a ride. Drop box. You give some storage. You get some storage. So I think you got a really interesting.

SPEAKER_297: Tool based and network based business here are two ways to win. And so what's your question for me?

SPEAKER_336: Any challenges right now in the business?

SPEAKER_299: Yeah. So, you know, in today's fundraising environment, um, you know, ever since the market turned publish or rather VCs sort of, they behave similarly. And they'll sort of the, the mantra now is like, I want to see revenue and all they care about is revenue, revenue, revenue. Now this is a network effect business and we win by building distribution. Right now we're building users at zero cap. And when you think about most rewards businesses, they're paying, you know, five bucks, 15 bucks per user. We're not paying for users. And so our go to market really should be a hundred percent focused on building distribution, getting more users. But at the same time, VCs and the, and the market wants to see revenue, revenue, revenue. Me putting pressure on publishers to essentially pay us is going to slow my business down. So how would you navigate that?

SPEAKER_297: Yeah, sure. This is a great question.

SPEAKER_296: Um, network based businesses do have a carve out in our industry when it comes to monetization.

SPEAKER_297: So if you're a SAS business, if you're a superhuman and you charge a dollar a day for an email product or slack and you charge eight to $35 a month, depending on which plan you are per user. Yes, of course, people want to see the number go up in your business. However, if you could prove that if you, you have these 60,000 members, and if you said, Hey, we're giving away the iPhone 16, and we just want you to take a survey about mobile phones and visit these three reviews of, you know, we're going to present you with 10 pages. Every time you visit one of these 10 pages in our network. And, uh, you scroll to the bottom and, you know, click on the iPhone 16, you get entered in. Now you'll get a bunch of sweepstakes jerks who, you know, kind of just do this, you know, scammy kind of way. But you also, if it's good content might get people to actually, who are interested in iPhone 16 and technology to visit 10 pages. That's good for the publishers. And it costs you $1,500 to set up the sweepstakes and give it away. Now you could prove to people if you could prove to people that when you do that, the publishers get page views and, uh, the cost is spread for the iPhone 16 across 30 publishers. So it's only 50 bucks a publisher and you just run those experiments without having, um, the publishers tell you. I'll pay for it. You just, you're running a $1,500 experiment. You say, look, we sent of the 60,000 people, 3% of them, you know, when and actually did something. Imagine if we had 6 million people and 3% did something. Now we'd really have a business. So that's up to you to prove it to me, the investor that you could do these little experiments and whoa, look at the interest that this created. And they had to click on an email to confirm they were in it. And if they put it in their friend's email, they got, you know, $10, they got, uh, 10 more entries and their friend got double the entry. So there are little experiments like that where you could show growth in the user base and the page where you're driving and be able to correlate it. So that that's really up to you to run those small tests and prove to us. You can grow, you know, 5 to 10% week over week. That would be what viral growth would look like in your business. Now for sales and for SAS products, you show me a SAS business growing 10% a month. I'm interested. You show me a business like yours growing 10% a month.

SPEAKER_78: I probably would think not very interesting. So you got to get to that 5 to 10% week over week. That's a lot of tests to run. And, you know, but you could actually do that with very low dollar amounts.

SPEAKER_333: Yeah.

SPEAKER_299: Yeah. Yeah. I think, I think where you're sort of pointing as a sort of unit connect unit economic story that we need to tell to investors like you to basically say, Hey, we're getting these users for free. And then we run these experiments that actually show that we're getting revenue tracks and the revenue traction is building or the activity traction is building on a per unit basis. Engagement. Yeah.

SPEAKER_297: Engagement is the name of the game. So you could say to me, you know, if you came to me with these 60,000 and said, okay, it costs us. You know, six months. And, you know, we burned a half a million. We, we burned a hundred thousand dollars in six months getting to 60,000. So we're going to need to spend a million dollars to get to, you know, what we think is 3 million because we're getting better at it. It's a fixed cost business, whatever. But if we did this many prizes, we would get to, you know, 10 million.

SPEAKER_73: And then every week or every day, we're going to run an experiment with a million of those 10 million. So we don't burn them out, but every 10 days they're going to get some sort of offer to engage them.

SPEAKER_297: And then we'll start charging the publishers because they'll be addicted to it. But, you know, you have to get the flywheel started, right? And so getting the publishers to agree, getting the VCs to agree, all this stuff takes work. So you may have to just invest small dollar amounts to show it on a micro basis. Yeah.

SPEAKER_296: And then say, imagine we add two zeros. Yeah. And here's what it costs to add two zeros to the velocity I'm showing. Yeah.

SPEAKER_371: Yeah.

SPEAKER_296: Makes tons of sense. Pretty straightforward. Yeah. And that's like your investment, but the good news is I think you can show this for very small amounts of money. How many people working on the team? I'm curious how many developers you have.

SPEAKER_324: Yeah. So we have a product person, a couple engineers. We're really, really good. We're kind of working kind of core on the product.

SPEAKER_373: Outsourced engineers or full-time employees? Offshore? Offshore, part-time. Got it. Basically is what the team looks like.

SPEAKER_303: Yeah.

SPEAKER_372: So you're in, you're in year one.

SPEAKER_378: You're an early stage startup. I take it. We're pretty early. Yeah. We're sub two years. Yeah. I've raised. How much you raised? Uh, 1.4 basically. Oh, okay.

SPEAKER_383: Wow. So you've raised a decent amount of money.

SPEAKER_296: Yeah. You should be able to get that network effect going. Just invest in giving away whatever the product of the moment is that aligns with this.

SPEAKER_297: I would also start thinking about, you know, the virality of social media and tick tock. Yeah. You went after publishers, which is where attention is going away. So they need your tool. Obviously they really need it, but their businesses are contracting and you may be selling like, uh, you know, deck chairs to the Titanic. Now, what business is growing? Tick tock shorts, video podcasts. There's a bunch of things that are growing in the world. So I'd also think about these tools and say, Hey, if we were to get 20, if we had 60,000 tick tockers and people making shorts and they were engaged in the network, what might this look like? Because sometimes, you know, a surfer is only as good as the waves presented to them. But when you go big wave surfing, right? Yeah, you're you've got these little tiny waves, three, four or five foot ones. Okay, fine. You can surf them, but it's not like you're going to be going to the north shore and do it, you know, hitting a 20, 30 foot wave where like, whoa, it's impressive if you can catch one of those waves. So I encourage you to really think deeply about the beach you're surfing at the beach you're surfing at. It doesn't, the time might be going out faster than you can build a business.

SPEAKER_78: I, I love looking at, you know, and I've, I've done this myself, right? I caught the blog. I caught the magazine zine wave in the nineties. I caught the blog wave.

SPEAKER_296: I caught the podcasting wave 14 years ago when I started still riding that wave and caught the angel and seed investing and incubator wave. Sometimes there's waves that are bigger than you. And if you catch them, right, man, that's why I'm, I'm doing more thinking about live and shorts. Um, you know, tick tock shorts, uh, well, tick tocks aren't called shorts. I think YouTube calls them shorts, but anyway, I've been thinking about other alternative formats outside of podcasting.

SPEAKER_333: And I've been thinking about live a whole bunch. So I encourage you to think about that as well.

SPEAKER_296: You have an interesting business. I mean, I would have, if you had presented this business to me to come to our accelerator, I'd be like, yeah, let's do it. But I think you're in a weird position having raised a bunch of money, not having violent product market fit or market pull yet. So you really gotta get some market pull. And I think. It might be that you, you might need to find another beach where the waves are a little bit bigger.

SPEAKER_297: So I would think about that. E-commerce is another one, direct to consumer. The people in direct to consumer were getting slaughtered. Yeah. Slaughtered. We're trying to get Facebook to work and Facebook worked for a long time. Then it stopped working. Yeah. Yeah. The ones who went to tick tock and social media and podcasts did make it work. So sometimes it's just like, it's not you. It's the environment you're applying your skill in.

SPEAKER_78: So. I think you're very talented.

SPEAKER_299: Yeah. Yeah. And I, I, we are, I can't disclose what we're doing, but I think some of the things we indicated are areas where we're testing right now, doing some experimentation. Good.

SPEAKER_364: Yeah. I'll leave it at that. I don't want to tip your cards.

SPEAKER_78: Yeah. Great job. And for everybody listening, who wants to learn more, uh, give us your URL again.

SPEAKER_373: I know that you're a dot tech, but remind everybody of your domain name. Yeah. Bonbon dot tech.

SPEAKER_395: What a great domain name. Bonbon dot T E C H.

SPEAKER_73: If you would like to get one of these great dot tech domain names, just go to get gt dot T E C H. And get one of those great. Dot tech domain names. Tell them your boy, Jay Cal sent you. We'll see you all next time on jam with Jay Cal in this week at the startups.