SPEAKER_00: Who's going to win long term? Who will have the best models and which will be the most market share? These closed models or open source models?

SPEAKER_01: I think in the future, majority of AI work is going to be based on open source models. I would say 80% of all AI inferencing or people building AI applications is going to be based on open domain models. And like some of those will be like fully like open domain. Some of them could be open domain, which are sort of supported by enterprises, you know, and, and, you know, that's sort of like really how the industry has progressed over the last like two decades. It's just really hard to beat open source.

SPEAKER_08: This Week in Startups is brought to you by OpenPhone. Create business phone numbers for you and your team that work through an app on your smartphone or desktop. Twist listeners can get an extra 20% off any plan for your first six months at openphone.com slash twist. Lemon.io Need to speed up your product development without draining your budget? Hire vetted engineers from Europe at lemon.io Go to lemon.io slash twist to get 15% off for the first four weeks. And Gelt It's time to take control over your taxes. Discover how Gelt can help you to manage and optimize both your personal and business taxes. Visit joingelt.com slash twist now.

SPEAKER_14: All right, everybody. Welcome back to the program. Today on the program, we've got Arvind Jain. He is from a company called Glean. What is Glean doing? We're going to find out today. They're trying to get corporations, enterprises, to use AI to help them sort, make sense of, and search their data.

SPEAKER_16: We'll hear all about it from Arvind. Arvind, welcome to the program. Thank you for having me.

SPEAKER_18: So tell me, what are you building and why is it important?

SPEAKER_01: So think of Glean like, you know, Google or ChatGPD, but inside your company. It's a product where people can go and ask any questions they have. And Glean will use all of your company's knowledge and data and information to answer those questions, you know, to you.

SPEAKER_05: So that's what our product is. We are an AI-powered search engine. We are an AI-powered assistant that helps people get more work done.

SPEAKER_23: Got it.

SPEAKER_14: And so you're not asking the ChatGPT, for example, to answer questions or make a marketing plan. This is specifically to search the data inside your enterprise. That's right. And ask questions against it. So I see on the site, you mentioned every department that any company could have sales and, you know, marketing, et cetera. Which categories? What's the beachhead market? Where are you being the most effective for your customers?

SPEAKER_01: So typically, Glean gets deployed company-wide. Typically, Glean will sell to CIOs. Our top users do tend to be engineers, support people, folks in sales. Like, those are the three biggest user populations, you know, that we have. But in general, like, this is a product that actually is useful to every single employee in a business.

SPEAKER_05: And therefore, we don't go and sell the product to individual departments. We typically go and sell through the CIO.

SPEAKER_29: Ah, so the CIO is evaluating new technologies and saying, this is going to go across all the verticals in the organization. We need a solution to ask questions in every department. That's right.

SPEAKER_14: Well, that means you're going up against, I assume, like, Intercom, HubSpot, Zendesk for support then. So, or are you sitting on top of those systems?

SPEAKER_01: We typically sit on top of those systems. Think of, like, Glean as, you know, an assistant. Like, it's a layer. It's a connective tissue that connects your knowledge across all of the different systems. So, while you may be using Intercom or Zendesk as your customer CRM and your support people use that as a system of record, but when they have a case that has come to them, so they'll open the case in Zendesk or Intercom and they need to actually resolve it. To resolve that, we're going to actually help them, you know, find the right answers. And, you know, sometimes those answers may be in knowledge articles in Zendesk, but sometimes they may be, you know, answers that are in some Slack conversations or in some internal JIRA, you know, issue. And, and sometimes, you know, the answers are with people, like, you know, that you can actually go on.

SPEAKER_05: So, Glean will actually help you find those people or that knowledge that sits outside of those systems and help you answer those questions. So, it's sort of like, you know, What's an example of that? Yeah.

SPEAKER_35: Give me, like, what's the best example?

SPEAKER_20: Well, I mean, like, let's say that, you know, there is a,

SPEAKER_01: as a support agent, you know, somebody files a request that my product has stopped working for some reason. And, like, you know, what would have happened is that, like, maybe there's a need to release that got rolled out and there's a bug in that.

SPEAKER_05: And right now, people inside the company are actively discussing, you know, that issue in some Slack channel. It has not made its way into your knowledge articles here. So, when you get a, you know, request, you know, from your customers, you know, you like, we'll actually quickly help you tap into like, you know, is this like, has other people run into it and like other conversations, you know, inside the company that would help you sort of figure out a quick answer, you know, back for your customer.

SPEAKER_29: How do you deal with the fact that a lot of this data is confidential? So, and maybe not everybody in the organization should see it, there's permissions in each of these systems, but you're going to index the whole thing. So somebody could ask about salaries in the company or, you know, different things. The language model has been trained on all this data. I assume you're training your language model and all this. What, what LLM are you using?

SPEAKER_01: Yeah. So, so first of all, like, you know, we are, um, LLM agnostic. So we can work with, uh, GPT four or Gemini, or, you know, which one are you using right now? We use all of them. These are all of them. Like typically we'll let customers make a choice. Like, you know, what language models, you know, they would like to use, um, which one do they pick most often? Actually, like most of the times they will, you know, give back the choice to us.

SPEAKER_05: So like, you know, we get to choose. I think right now we started out with, you know, I think majority of our deployments right now are using GPT four.

SPEAKER_14: And with GPT four, when you put that data in, how do you know if the model is the, the, the GPT four model by open AI assures you that the customer data does not go in there? Or do you have it off prem? How do you manage that issue with the language models? Because a lot of CIOs and CEOs are really concerned about giving open AI their data to train it on.

SPEAKER_01: Yeah. So, so see, like, you know, you, you're absolutely right.

SPEAKER_05: Like if you think about using AI in the enterprise, first of all, your data inside the company, like has, first of all, it's private to you as a company, but second, within, within the company, you know, there's governance on that data. Like not every employee can actually use, you know, all the information that exists within the company. So Glean actually solves both of those problems. So number one, you know, we're not actually training or fine tuning models like GPT four. We're actually using them only as summarization and synthesis engines. The way our product works is that, you know, when you come and ask a question and you're one of the employees in the company, what we will do is, you know, first, you know, based on that question, we're going to use our core search technology and we'll assemble the right pieces of knowledge and information that we think is going to be able to, you know, sort of answer that question that you have. And we will actually restrict you. So like we know who you are and what content you have permissions for. So we'll only let you use the information that you are actually individually authorized to use.

SPEAKER_01: Now, once you've actually gathered, you know, this information safely, now we will actually take the snippets of this information and ask an LLM like GPT four to summarize that information. Do you trust OpenAI? So we, so we actually work with Azure, you know, to use GPT four and, and the way we, you know, we work with these model providers, um, is that we, we have a contract with them where, you know, our customers are guaranteed full privacy, you know, for their data, like their data has never logged outside of their own clean environments and, and, and, and Azure or Google don't have the ability to actually go and, you know, train any models on that data.

SPEAKER_51: Yeah. So, so yes, our customers get full, um, assurance.

SPEAKER_29: So you trust them with that function? Cause a lot of CIOs have been a little bit concerned watching some of these, uh, you may have seen, you see the viral video of the CTO of opening AI talking about Sora last week.

SPEAKER_14: Yeah. And she couldn't answer the question of like what training data was there. And she wasn't sure. And it kind of felt like she was lying. I think based most people's thing there. So it does seem to me like the big challenge here. You tell me if I'm wrong, is that using these third party models, even, even on Azure or Google cloud, people are nervous. Are people nervous about that? And they want to move to having say an open source one on prem and, or just, you know, in their own cloud.

SPEAKER_01: Well, see, like, you know, different customers are at different level of sort of both paranoia and security requirements. A lot of the companies today, a lot of enterprises are now comfortable with storing their enterprise information in the big, in the big cloud vendors, like Google or Microsoft or AWS. Um, a lot of like, you know, business technology and systems run in these systems. And, you know, so the trust level, you know, for these, you know, the, the big three cloud providers, is actually quite high. And if you think about it, like, you know, like, see, AI is a new thing, you know, first of all, already, like, my business data is, you know, is in these systems. Right. And so now, you know, I'm also using some additional like AI models, again, hosted, you know, within Google or Microsoft. So as long as like, you know, I get those VPC controls,

SPEAKER_05: I'm actually comfortable with that. So that's sort of like, most of the customers feel that way. And if you don't trust Google or Microsoft, then then of course, like, you know, you know, then you typically are running everything on premises.

SPEAKER_01: And, and we, like, as a, as a company, like, we support, like, also, like, hosting models ourselves. So if, if a customer wants to use an open domain based model, um, as, as, as the core LLM, you know, that's, you know, in their clean experience, you know, we also allow them to do that.

SPEAKER_14: Yeah, I mean, there was a big instance of a bunch of Samsung employees, I guess, using chat GPT for, and then their source code and other information was then trained into chat GPT for I'm sure you've seen that. And I'm guessing that comes up with CIOs. Can you explain what happened there?

SPEAKER_01: Yeah, so in that particular instance, the employees in the company, they were actually like, you know, using the standard chat GPT product, and they were actually pasting, you know,

SPEAKER_05: you know, their code inside of that, like, you know, code or sensitive, you know, documents within the company, you were posting that, you know, within that chat GPT interface, and then asking, you know, chat GPT to do some work on it. And, and so when you actually use these services directly, like these are meant to be consumer services, you use them directly, you don't have any controls, like, you know, every, you know, data that you actually put in that system, like, you know, like open AI, you know, has, you know, like, you know, is allowed to actually go and, like, train their future models, you know, using that information.

SPEAKER_01: On public interactions, right, on the public interactions. And that's what happened there. The way to sort of, you know, make sure that, you know, you're not, you know, exposing your private data as a CIO, like, you know, to ensure that your employees are not sending information, you know, to these public consumer based products, and that's, that's when you use a product like clean, because if you use clean, you know, now you have a very safe and secure environment, you know, where people can still go and ask questions.

SPEAKER_05: And, and, and we will make sure that like, you know, any information that is being sent, you know, to Azure or Google, like, is actually following, you know, that contract and that security agreement that you have, you know, from them that they're not going to use that information to train. Yeah.

SPEAKER_69: Juggling multiple devices and apps to run your business is a mess. Open phone is here to make it simple by simplifying your business communications with one easy to use app. Open phone has rethought. Every detail of what a modern business phone should be. And here's the magic. It works through a beautiful, elegant app on your phone, or you can just use it on your desktop, making it super easy to get a business phone number for your entire team. And you know how brilliant open phone is. My teams use it every single day. My sales team loves it. My ops team, they use it all day long. And here's the features that we love. You can create a shared phone number like customer support with multiple employees, fielding all the calls and all the texts to that one number at my investment from lunch. We pride ourselves on replying to every single call or email instantly. And open phone is the number one rated business phone on G2 for customer satisfaction. So here's your call to action. Super easy. Open phone is already affordable. Starts at just 13 bucks a month, but twist listeners get an extra 20% off any plan for the first six months at open phone.com slash twist. And if you have existing numbers with other services, no problem. Open phone is going to port them over easy peasy lemon squeezy. No extra cost. Head over to open phone.com slash twist to start your free trial and get 20% off.

SPEAKER_71: How do you compete against the native tools, you know, getting more and more robust. So you can't possibly write an LLM that's going to work as good as, you know, the one that's built into, you know, Salesforce. Salesforce eventually just to Salesforce build in a language model yet or no.

SPEAKER_01: So, so typically, like, you know, companies like, you know, SaaS companies like Salesforce or Atlassian product, you know, or, or, you know, any of these systems, they're not building language models, language models are built by no, but they're building language models into their product.

SPEAKER_02: So and you're not building language models either, right?

SPEAKER_01: Well, so that is complicated. So we, we build language models. We build smaller models, you know, to sort of, you know, build semantic understanding of your company knowledge. But, but we as well as Salesforce and, and, you know, like other, other products, SaaS product companies, all of us use, you know, APIs to, you know, you know, to these large language model providers like GPT-4 or Gemini to do some work. So typically in that model, what happens is that, you know, you're basically sending prompts to, to these models and, you know, having these models do some work on that prompt and return back response and, and, and you sort of create these AI experiences within your application. So now to think about like, you know, if you think what's, you know, Salesforce, they're going to actually have some AI features within their product.

SPEAKER_05: You know, Coda will have some features within theirs, Jira will have some. So yes, so you're right that, you know, every application in the future, you can imagine that they will have some AI smarts.

SPEAKER_01: They might even move to the chat interface, right?

SPEAKER_05: You know, and yeah, they may have a chat interface in addition to actually do some, some of the work that, you know, people do with those products. For example, you know, today, like if you want to create a new issue in Jira, like you like, typically you'll go in that, in that app and click a button and then fill a form. But you can imagine that, yeah, they may have a, they may have a chat interface, you know, that allows you to sort of go and, you know, create that, you know, new issue using a national language interface.

SPEAKER_01: So yeah, so those things may happen in the future, but, but that's not like, you know, what, what Glean is actually solving for like, you know, Glean is, you know, solving different problem, which is that if you think about your work inside a company, it happens, you know, across many different systems. Like, you know, I'll give you one user journey as an example, as an engineer, let's have to actually go and build this new technical component. And so that journey for me is going to start with first talking to people.

SPEAKER_05: I want to be actually having some conversations like this one, like, you know, on Zoom, I'm going to be talking to some other engineers, talking about design choices. I may have some conversations in Slack, you know, we'll be talking about like, Hey, like, what about, you know, this approach versus that approach is actually a, you know, a Jira, which actually tracks why, why am I even building this technical component? What were the problems that we're trying to solve then? So, you know, first, like you have all of these different things, then like, at some point, I'm going to actually write a design document, like maybe in Google Drive, to sort of describe my design. And then later on, I'm going to actually write code, and that's going to go in GitHub. And so if you think about like, this whole journey, all the information about this project, like it actually spanned all of these different systems. So now think about like, you know, six months down the line, somebody comes and ask a question that, Hey, why did we use, you know, this programming language, you know, to build this component? Where is the answer? It's not in one place. It's actually, you get that answer by actually consulting, you know, all the knowledge that sits in like those five or six different systems.

SPEAKER_01: And so that's, that's where the power of green comes in, like, if you think about, you know, your work, we are tying together knowledge from all of these different systems in one place. And we give you as a user, like, we remove the burden from you, like, you know, hey, where should you go and find things? Where should you go ask questions? And It's a great place to start.

SPEAKER_14: If I was joining the company, or I'm the CFO, and we're the chief operating officer, and somebody tells me about project bluebird, and I don't know what's going on project blue. Hey, give me an overview of project over bluebird. He gives me all those. But if I'm not the question I have, the next question I have there, because that's kind of a cool feature to be able to go across the entire enterprise. So I totally get that. But there's a discussion going on in a slack room that I don't have permission to permission for. So how do you deal with that? Maybe there's a bluebird project bluebird slack room. It's got 20 people in it. But I am the COO on a CFO. I don't have access to that room. I was never invited. Yeah. But you have that in the LLM. So how do you let me know that there's a conversation there that I don't have the rights to see because of the way slack works or Jira works?

SPEAKER_29: Like the COO doesn't even have a Jira. That's right. They're gonna have a get up account. So how do you deal with permissions?

SPEAKER_33: Yeah.

SPEAKER_01: So two questions. First of all, like, you know, the way our system works. There's no data.

SPEAKER_05: None of your enterprise data is actually in the LLM. The LLM is basically just the standard, you know, like language model that is trained on the world's public knowledge.

SPEAKER_01: We're not using it to store knowledge. But now, like the way Glean works is that when we actually connect with all of these different systems, we have actually built an understanding of how permission works in those systems.

SPEAKER_05: So, for example, when Glean connects with Slack, it knows, you know, the concept of channels, it knows that certain channels are private, certain some of them are public. If it is private, it knows who are the members in those channels.

SPEAKER_01: So now, you know, we're going to index every single message or conversation, and we know exactly who are the people who have access to it. And this is all, all of this information is stored in our search index. Same for, you know, a document on Google Drive. You know, we know, like, you know, who are the collaborators on that doc. And we store this permissions. And now, and so our, and this is unique about the Glean, you know, technology. So it's fully permissions aware. Now, when you come in and ask a question in Glean, you have to be, first of all, signed in, you have to present to us your identity, who you are.

SPEAKER_05: And we are able to now, you know, retrieve documents from the index, but only work on the documents that, you know, we already know that you have permissions for. So, so, so building that, like, you know, that's part of our core technology is to understand, you know, like these authentication and permission models in each one of these individual labs and sort of, and make them, make them work.

SPEAKER_29: So, you know, Jason, the COO can't see Project Bluebird. No, I'm not in that group.

SPEAKER_81: So when I do a search, it won't show you anything. You will show me anything.

SPEAKER_20: Yeah, we won't even tell you we won't even tell you that. Hey, there's some useful information. But we're not like we can't share it with you talk to somebody else because sometimes even that is dangerous.

SPEAKER_29: Like even like, let's say you did a search. Yeah, who's on a performance improvement plan. That's right.

SPEAKER_14: It's like, there are seven conversations about performance improvement plans happening with these people in them with these names. You were like, ah, wait a second. Who are the seven people who are in a, just the fact that there is a file in and of itself is information. That's right. And do these, you know, Slack has pretty robust AI coming on board notion and CODA also have AI built in. Have they built API's into it? Or are you just having to rebuild all that from scratch against their services? Or are you doing the search in slack? As if I was logged in.

SPEAKER_01: No, they actually returns. We do like our we do search natively in our system. So the way our system works is that we bring content from those systems and index them and clean. So as content is being produced as somebody sends a new message, we get a notification. And we'll then use, you know, take that message and index that in our system. So this is this is continuous. This is done in real time all the time. And now when you come into a search, like that search is entirely served from within our system. And that's important because for two reasons. One, like, you know, like you need to be fast. You can't actually in an enterprise, you know, you will. You have 1000 systems, you know, that you're using.

SPEAKER_50: You can't actually, you know, when a user comes and ask a question to you, you can't actually send a message to all, you know, hundreds of them and wait for responses to come back.

SPEAKER_88: Yeah, no, search correlates to search usage correlates directly with the speed of return. That was Google's big lesson, right? That's right. Exactly. If you make it faster. Yeah, people use it more.

SPEAKER_102: Yeah, in fact, that's, that's, that's one of the things I worked on at Google, you know, was actually making it fast.

SPEAKER_05: But the second thing is, the second thing is also, like, search is a hard problem. It's sort of like magic, like, you know, you come in, you type two words, and like, I need to sort of now figure out from those 10 million documents, the one that exactly you're looking for.

SPEAKER_01: So this is a lot of work that needs to be done to build a great search. And I think like, like, what we're seeing typically is that like, each one of these individual SAS products, like, you know, they're, they don't have so many resources to put on search, like we have hundreds of engineers and we actually make search better.

SPEAKER_29: It's such a good point, right? Search is like an afterthought for them, or they may just use some open source library, they never update their search.

SPEAKER_14: Exactly. Even like, I've been complaining about Twitter search since Twitter was born. And there was a third party called some eyes that they bought to make their search a little bit better. And that was 10 years ago.

SPEAKER_05: Yeah, because you think about like enterprise, enterprise software companies, they don't win customers that way. Like, you know, in Jira and Asana are competing, they're competing on features, not by saying that, hey, my search is better than yours. So, so I think that's another thing.

SPEAKER_01: So like, you know, to really solve the search problem, you have to, you have to do a lot more work. And also just one more thing, you know, on this, like, you know, what, why, why, you know, thinking about search in, in the way we think is important. It's really important to sort of take all of your enterprise context. And that sometimes gives you signals on like, you know, both, you know, what information is actually relevant and important and to whom. So one example, let's say that, you know, somebody writes a, there is a document that, you know, that talks about benefits. But whenever somebody asks a question in Slack that, hey, like, you know, where's our benefits policy, like, you know, somebody in HR shares that document with you.

SPEAKER_05: So there's a lot of, you know, like, there's exchanges that are happening in Slack, which tells you that, hey, this particular document is authoritative for that answer.

SPEAKER_73: So it's great training data in that way.

SPEAKER_01: Yeah, so and then slide just being one example, like, you know, like, every, if you think about there are these interconnections between, like, you know, how a Jira issue is created and how and how it's referenced in the Slack conversations when you think about your enterprise knowledge, it's like it's a graph, like, you know, there's knowledge, there's a lot of different pieces of knowledge, and they have all interconnections with that. And, and, and similarly, there are people, and, you know, and like people like, you know, of course, you know, they're like, you know, we're talking about like, you know, there are engineers that are support people, they're salespeople, and, and, you know, we build these learnings that, you know, engineers are actually, you know, you know, clicking on this, you know, are using this document a lot more than salespeople and vice versa.

SPEAKER_05: So all of those learnings, you know, is sort of what enables you to find out what is more relevant information for whom, and that's the core of like, you know, what Glean is, and that's why it's so important to actually have that full enterprise wide view of your people, as well as your knowledge.

SPEAKER_108: Right now, startups have to do more with less, we all know that it's rough out there, folks. So if you need great tech talent, but you don't have the time to interview dozens and dozens of candidates, you need to check out lemon.io. Lemon.io has thousands of on-demand developers to choose from, and these devs are vetted, experienced, result-oriented, and they charge competitive rates. Great developers can be incredibly hard to find, and when you do find them, it can be hard to integrate them into your team. Lemon.io handles all of that for you. Startups choose Lemon.io because they only offer hand-picked developers with three or more years of experience and strong portfolios. In fact, only 1% of candidates who apply get in.

SPEAKER_110: And if something ever goes wrong, lemon.io will get you a replacement ASAP. You know what? A bunch of our launch founders have worked with lemon.io, and they've had great experiences, which is always good to hear. Go to lemon.io slash twist and find your perfect developer or tech team in 48 hours or less. Go to lemon.io slash twist and find your perfect developer or even a tech team in 48 hours or less, and twist listeners get 15% off their first four weeks. What a deal. Stop burning money. Hire developers smarter. Visit lemon.io slash twist.

SPEAKER_29: What's really great as a byproduct, and how you can tell me if any of your customers are asking for this, there is a concept of compliance and legal reviews.

SPEAKER_14: So for example, people think like their DMS on slack or their emails, even if you delete your emails, those things are stored. If you have the settings done properly, like in Google docs or Microsoft teams.

SPEAKER_29: Yeah. And so because you ingest everything, you do have the ability to do a God mode where the compliance could say, Hey, did anybody say this?

SPEAKER_14: Let's say it's insider trading, you know, did anybody share Project Bluebird, let's say Project Bluebird was an acquisition? Did anybody say the word Bluebird? And you could actually see across all documents? Yeah.

SPEAKER_19: Does that exist like a super God compliance mode?

SPEAKER_05: There is there is a compliance mode, which is highly restricted, and it's available only to your governance, you know, and legal teams, like, you know, for exactly the use cases like that, you know, you discovery or, you know, also like for privacy compliance, like, for example, you know, a big use case, you know, that that that's out there is, you know, you have a, you know, a, you know, a, you know, previous, like an ex customer, or an ex employee, you know, who comes and tells you that, hey, you know, delete all my data. Right, you know, and then you have to sort of, you know, there are laws that actually required to do that, like, you know, and so how, but how do you even figure out like nowhere is all that information? Where is that data?

SPEAKER_14: Facebook had this issue because Facebook had been doing backups of backups of backups. Right. And that's why when you delete your account, people are like, oh, you know, they say it takes 30 days.

SPEAKER_29: I think that's because they have all kinds of mirrors and mirror images of data so many different places. They want to be thoughtful and thorough about it. It would seem like a CEO God mode would be one of the great features of being able to look across this whole amount of data.

SPEAKER_14: If you're working at a company, you should just know by default, anything you do on your laptop is your companies and every email you send never use corporate. Yeah, devices. Yeah. To order from Amazon or do private communications. Gosh, it's 2024. I don't know why I have to say this to people, but I'm shocked that people will because I'm on the board of so many companies or investments. Some story will come up that people were sending things to each other on Slack or teams or doing something on a corporate device that is completely insane that you should not be doing.

SPEAKER_120: That's right.

SPEAKER_05: Yeah. And I think the from our perspective, like, you know, we like, you know, we help companies, you know, from a compliance perspective there. But, but green is actually not a system of records. So that's not like, you know, another system that you do worry about, like in the sense of when you need to delete data for somebody, like, you know, you don't have to actually go and explicitly go and delete that. And clean.

SPEAKER_121: Well, you do have to get rid of the data and clean if it's indexed. Right.

SPEAKER_05: Because you have in the industry, we stay in, we stay in sync with the actual system. So like if they didn't slide, it's automatically deleted in our system.

SPEAKER_123: Super complicated to take care of all that.

SPEAKER_122: Huh?

SPEAKER_01: Yeah, that's where the complexity is. But it's interesting thing. Like, you know, you talk about AI, you know, like everybody wants AI. And this is one of the key problems that businesses are running into, which is like, look, you know, we have all this information in our company. And yes, like, you know, we've set some rules, you know, permissions, but you don't always get it right.

SPEAKER_05: Like, you know, oftentimes, you know, there'll be some documents somewhere that like, you know, sensitive and the person in HR, like they didn't know how to set permissions. They made it open to, you know, everybody in the company. And, and you start living with that, like, because nobody can find anything anyway. So like, who cares? Like, you know, it talks somewhere that, you know, but let me ask you. Yeah.

SPEAKER_126: I was just going to say that.

SPEAKER_01: So that's a, that's a big issue today. Like, you know, with like AI, because now AI does all of that work for you.

SPEAKER_05: Like, you know, like in this new world, you just get to ask questions. Right. And, and there's this AI, you know, like, for example, our product is connected with all of the company information. And it's going to actually answer questions back for me. So it sort of makes, you know, these governance gaps, you know, like, you know, you're going to pay for it now, like, you know, they're going to become a big problem problem.

SPEAKER_20: And so that's one of the things that we hear a lot from, you know, CIOs, you know, they feel AI is powerful, but they're also scared of it.

SPEAKER_59: When you see, I mean, you must have seen the Devon demo last week, the, the AI coder going out and like doing jobs on its own. Did you see that last week? The Devon?

SPEAKER_05: I didn't see it, but I've sort of, you know, seen like things like that and heard discussions about it. Yeah.

SPEAKER_14: So, you know, now that you're indexing the whole company, you're watching all this data and code and customer support tickets and sales all occurring. It would seem to me that you understand, you know, what a salesperson does all day, what a coder does all day and all their activity buzzing around. Yeah. So that's great. I mean, you understand who the most productive employees are on a certain level, right? You could tell me who's working, who's making the most commits. And this exists already in JIRA, you know, you could tell me, Hey, this person's work hours. They work three hours today, according to the data we've seen. So is there some idea here of looking at productivity? There are certain apps that people are using to monitor their own productivity. Then there's like people tracking their hours, but it does seem you could tell me, Hey, you know, this person hasn't done anything for four days. I guess they're on vacation or, Hey, this person is putting in. They're dropping, you know, data into all these different resources, 12 hours a day.

SPEAKER_134: They're working 50% harder than the average person.

SPEAKER_05: Well, I mean, so, you know, we have, you know, we didn't start our company, you know, with the goal of sort of building these, you know, analytics and, you know, or like some people analytics in some sense, you know, our, our goal, you know, has always been to help people like get work done faster and make them more.

SPEAKER_135: But people analytics is a really fascinating topic.

SPEAKER_01: So it's a, it is. And so, so the, so the data is there, like, you know, say better without us, you know, like that data is there.

SPEAKER_05: And you write that, like, you know, when you bring it all together, like, you know, how you bring it in clean, you know, like, you know, you know, somebody can actually turn those analytics on our platform.

SPEAKER_01: Uh, and get, and, and, and, and gain insights faster. Um, the, but I would say like, you know, like, like, like we haven't really seen, like people talk about it, but I think like, you know, I think they, like, we haven't seen like actual.

SPEAKER_05: Like attempts, you know, where somebody is trying to actually build report reporting like that, you know, using the data on our platform.

SPEAKER_14: You know, the, the negative interpretation of it is employee monitoring. Yeah. So you can, you can see employee monitoring and then there's employer productivity. That's right. And you know, they, they, they're just, if you're doing, if you're running a call center, you really need to monitor it because people might say something stupid to a customer and somebody who's on a call center all day, they expect all of those interactions to be monitored.

SPEAKER_29: Now, a higher level knowledge worker, a sales executive developer, they don't expect it, but they might very much want to be productive. And so I know people who run productivity software and you have it on your iPhone, right?

SPEAKER_14: It tells you which were the most popular apps. Yeah. And I've looked at it a bunch of times and I'm just like, ah, I want this. I know like six people in my organization want it, but I bet there's like 10 who are absolutely paranoid about like that data being there. But what's important for people to understand is with AI, with a system like lean or any other system, the byproduct is your collective work is going to be in a database somewhere. Yeah, which means you can really study it and figure out with what is this person doing in our organization? Like, do they need to be here? Or do they need a raise? Does this job need to be eliminated? Or do we need 10 more of these people? Or do we need to study this person?

SPEAKER_71: That people analytics to me is incredible.

SPEAKER_102: Yeah, I think that's, that's really powerful.

SPEAKER_05: And actually like, you know, but, but I would say one more thing, which is, you know, today, you know, part of it is that, you know, you can have like, you know, a few people in the company that could sort of do these analytics on an organization wide basis. But part of it is like, what about you yourself? Like, you know, like, you know, you can go and clean today and say like, Hey, tell me, you know, where I spent my time last week. Right? I was going to tell you, like, you know, like, if you're meeting a lot, like, you know, like, you know, like, for example, I can ask and clean, like, how many, how many hours of meetings I have last week? And it has access to that information is going to answer that back to you.

SPEAKER_01: So part of it is that like, you know, like, you know, like, how can we help you as an individual sort of have more insights into your own work? And, and like, so like, one of like, one of the one of the very popular use cases are like popular questions.

SPEAKER_05: And, you know, the people asking clean is, like, every Monday morning, they will ask for, like, summarize, you know, all the work I did last week, you know, because they actually need to share because they need to share that information with their manager or with their team, like, you know, posted in like, you know, whatever, their scrum, you know, notes. So, so you can do that and clean and like, you know, go through your Jira's and your GitHub's and your Slack conversations and sort of give you like a really nice summary of what you did last week. So, so there are the analytics or summarization that that you can actually bring to each individual for themselves. And, and like, you know, like, and you sort of start to like, you know, bubble up, like, you know, as a as a manager of a team, you can ask the same question, what did my team do last week, and it'll actually do it, it will do it for you, like, as a manager, you'll, you'll be able to sort of get that summary, but only with information that you as the manager actually have access to. So you could actually, like, so if there was a employee doing something, you know, writing, you know, working on a dog that they're not shared yet with the team, you know, the manager won't get to see that. But, but yeah, so yeah, so there are use cases like those, we are actually, you know, you know, going from the angle of like helping each individual with their work.

SPEAKER_50: And, you know, with their own sort of productivity. We haven't seen that much of like, you know, the, like what you mentioned, which is that they could be. Yeah, I mean, I have my own little ways of doing it.

SPEAKER_14: Sometimes I go into notion. Yeah, and it will show me, I think I'm the administrator is why it shows it to me all of the changes in the database.

SPEAKER_29: Yeah, and I click on it, and I see Bianca, Andre, Heidi, you know, coming up over and over again, the three people on my investment team. And I'm like, wow, they're super product, productive inside of notion all day long.

SPEAKER_14: And I noticed that like, oh, wow, they're really taking good notes. And sometimes I'll just take a look at the document. Now all those documents are public, anybody could look at them, but it's really nice to see the pulse of the company. Right? Yeah. And then there was this really cool reporting that I got just by opening up slacks admin to add somebody. It'll show you how active each person was in the last 30 days. Yeah. It just shows you like how many messages they sent. Yeah. How many they got, and then how many days out of the last 30, they logged in. Yeah.

SPEAKER_59: And I was like, shout out to these, you know, 30% of the company that logged in, you know, 28, 29 or 30 out of 30 days. Like you can take a day off from it.

SPEAKER_152: I would never not check my slack. That to me would be crazy as the CEO.

SPEAKER_05: Yeah. I, that's the one I actually like, like a lot myself. I think it, it does tell you a lot about, about the company. Like, you know, when you sort of look at these data. Yeah.

SPEAKER_29: Or then you look at the bottom and you're like, like one time I was like, oh my God, this person was logged in like 14 out of 30 days. And I was like, oh, they took two weeks off. They had a honeymoon or something. Totally fine. That's the time you want to turn it off. Right. But then other times it's like, is that, should that person have turned off their slack for two weeks and them or whatever number of days might be time to have a conversation about that. Let me ask you about search. You were at Google.

SPEAKER_59: Yeah. In five years will people be doing search engine searches or will they be doing chat GPT searches or like, you know, chat interface searches.

SPEAKER_14: I'll take open AI out and I'll take Google out since you weren't there search engine versus chat interface and just having a conversation, which will be the majority. Yeah. Of, you know, users searching for knowledge, which will be the majority in five or 10 years.

SPEAKER_05: Well, I think in five or 10 years, there won't be two different interfaces. There's only one, you know, because I've ultimately like, you know, what are you doing? Like, you know, you have a question. You need an answer. Sometimes, you know, sometimes your question is about research.

SPEAKER_67: Like, you know, you want to read a, read a document actually in response to like what you're looking for. Sometimes, you know, you're looking for a one line.

SPEAKER_162: So we'll just move to a chat interface. No, we won't be on this like 10 blue links. No, I don't know.

SPEAKER_20: That's not what I said. Like what I'm saying is the, what I'm saying is that there's only one interface. But that interface, you know, is, you know, is adaptive.

SPEAKER_05: It's rich. Like, you know, takes, you know, what's the kind of question that's coming in and like, you know, appropriately give you the right answers back. If you think about, I think, like, I think that there isn't actually this dichotomy, like, you know, that, you know, we make off right now. Even in Google, for example, well before, well before, like this whole generative answers and like the conversation interface that you could go and ask, you know, in Google, like five years, you know, from, you know, like five years back, you could ask the question. Hey, what's, what's the temperature like in, you know, what's the weather like today sports or weather and time. Yeah.

SPEAKER_97: So currency exchange stock ticker price. And it just gives you the answer, right?

SPEAKER_20: It will give you the answers. And it actually also, it also tell you, like, you know, other interesting questions you may actually ask.

SPEAKER_05: Like, so there was sort of this is that this has been a progression, right, you know, where I think the search interfaces will sort of be like that, where, you know, you're going to understand the intent of the user and what they're trying to look for. Sometimes you can actually give them, like, you know, resources, links, you know, to go on, you know, they should go and read more details. Sometimes you're going to see summaries or like, you know, quick answers on it.

SPEAKER_66: Are you grinding hard to grow your business? I bet you are. You're listening to this week in startups. Of course you are. But don't let your hard on profits slip away because of overpaying on taxes. You need to check out GELT, G-E-L-T, is the secret tax weapon trusted by savvy founders and CEOs. Their elite solutions will transform how you handle your taxes. You can integrate your personal and business tax planning with one provider. They have tech driven efficiency that simplifies tax management and in-house advisors with startup and business expertise. This means no more overpaying your taxes. You're going to save time. And these expert opinions will boost your financial health and help you grow your business.

SPEAKER_14: GELT helps you stay compliant so you can focus on your startup's mission with year round tax advice, personalized strategy sessions, and a comprehensive tax library for continuous education. When you optimize your tax strategy, you optimize your competitive edge. So here is your call to action. It's time to bring GELT's elite tax team into your business. Visit joingelt.com slash twist and get 15% off your first year. What a generous offer. Thanks team GELT. Join gelt.com slash twist to get 15% off. Transform your taxes from a liability to an asset. That's 15% off your first year at joingelt.com slash twist. They had the one box snippets, all this stuff, but it's going to just move to an answer. So what does that do to the cost per click business model of the internet?

SPEAKER_29: Because right now, if a search engine, I'll just say any search engine, it could be any of the popular ones gives you, um, doesn't answer your question.

SPEAKER_59: And it forces you to click some number of people will click the ads because the ads are generally answers to the question.

SPEAKER_14: You know, how much does the latest Volvo have, you know, is there a convertible Volvo and it might be an ad for convertible Volvos or use convertible Volvos. But if we're just going to just answer people, Hey, yeah, the Volvo, you know, made four different convertible models. There are one active and these are the three historical ones. Okay, I'm done. I don't click on any of the ads. So what's going to happen to the cost per click and model over the next 10 years as AI just answers everybody's question.

SPEAKER_01: My view is that I don't, I don't think like, you know, the models sort of disappear already in Google. Like, you know, there's a concept of cost per click. Google always also like, you know, you know, we'll talk about like cost per conversion.

SPEAKER_05: There are all these different sort of, you know, degrees of like, you know, like how you're actually ultimately driving a sale. And like, you know, you as, as, you know, the sort of facilitator in that, like, you know, what is your cutoff, you know, that transaction. So there is, you know, there are cost per impression. There's cost per click. There's cost per conversion. And so I think like what will happen is that like in the future, like for when you ask a question that has commercial intent and that is, there are four different commercial parties, you know, that could actually, you know, all provide you with an answer. They're going to compete and, you know, like, you know, the search engine may show like an answer coming from one of them and potentially like, you know, there is like further follow-ups, you know, that you take you to those sites. And, and then you get like, you know, higher, higher.

SPEAKER_29: So there could be a different type of funnel or modality for monetizing answers. So you give the answer about this Volvo. Yeah. And at the end, it says, would you, or, you know, follow up questions.

SPEAKER_14: Where can I buy a Volvo? Where's the closest Volvo dealer? Are there any incentives for buying a Volvo? Who can lease me a Volvo? Are they use Volvos? And all those. Yeah. If you click them could include either cost per click links, or it could put you into a conversation. Yeah. That's the ultimate. Hey, what kind of car are you looking for? What's your budget? And then give that lead to. Yeah. Local Volvo.

SPEAKER_01: Yeah, I didn't even summarize it like, you know, very, very simply. I think like Google is getting paid, you know, because, you know, that's where users are

SPEAKER_50: going and seeking those answers. So as long as that stays, that means, you know, they, they should be getting, you know, they're cut off, you know, that.

SPEAKER_186: How long were you at Google for?

SPEAKER_50: I was there for years. I was there from like end of 2003 till 2014. So about like, wow.

SPEAKER_59: You were there during the early days. You, you, you haven't been there for 10 years. Yeah. So what do you think of all this? Um, you know, brouhaha, this Donnybrook around, uh, the Gemini project and all this woke

SPEAKER_14: DEI stuff that was included in it. How does something like that happen at a big organization? And what do you think Google's chances are of kind of being able to release product faster? Like how did it get to this point? Because it did seem like Google was so efficient in the period you were there and just giving us products that solved our problems as consumers. And now it seems like they're doing something completely different.

SPEAKER_102: Well, my take on just the AI models first, like, you know, from Google is that I personally

SPEAKER_05: feel like, you know, they're actually in a strong position. Um, like, you know, you know, whatever goes wrong in the model, like, you know, they get more attention than anybody else. But if you think about, if you think about Gemini, like actually, you know, it's, it works, you know, it works really well, like as a, as a model.

SPEAKER_50: Um, you know, they also have, I mean, like, you know, if you think about Google, like, you

SPEAKER_05: know, they're the best set of engineers, the most AI talent, like by far, even now, you know, they have the world's biggest data centers, they got all the machines, they got all the money. So I think the, the, I think the calls for like, you know, the, the doom, like, you know, scenario, I think like, it's in my opinion, you know, it's sort of like, I think it's, it's like, it's fun for people to talk about, but I think like, I feel like the company is in a, is in a really strong position.

SPEAKER_58: Yeah. You think they can still win? Yeah.

SPEAKER_189: I think so. Yeah.

SPEAKER_45: Yeah. It seems like maybe they've got maybe too much process.

SPEAKER_02: Like it used to move much faster, right? When you, when it was a smaller org. Part of it is yes.

SPEAKER_01: Like, you know, they need to organizationally make improvements, but part of it is also like, you know, the burden of like success. I mean, I think about like, you know, they could not like as, as, as a company, you know, like whose core business is to help, you know, people find information and correct information.

SPEAKER_05: Like, you know, like they, they were sort of rightfully cautious about like not putting these models, you know, that hallucinate like in front of people.

SPEAKER_14: Yeah. Giving, giving the wrong answer is really anti Google's mission. And it does seem like this is why Apple hasn't released a ton of AI features.

SPEAKER_71: It's because they also like to have a lot of fit and finish and polish on their products.

SPEAKER_20: And so that's, and like, you know, an upstart, like, you know, they can, they can launch whatever. And that is sort of like, you know, like, you know, in reality, like sort of what is like, you know,

SPEAKER_05: like, you know, cause them to be a little bit on this backseat.

SPEAKER_14: What do you think is going to win? Open source, you on just open source crock over the weekend, obviously, Facebook and meta.

SPEAKER_29: All their models are open source. Apple is working on an open source image editor, generative product, and even opening. I started as open and then when closed.

SPEAKER_00: First, who's going to win long term, who will have the best models and which will be the most market share? These closed models or open source models?

SPEAKER_01: I think in the future, majority of AI work is going to be based on open source models. I would say like 80% of all, like, you know, AI inferencing or like, you know, people building AI applications is going to be based on open domain models. And like some, some of those will be like fully like open domain, some of them could be open domain, which are sort of supported by enterprises, you know, and, and, you know, that's sort of like really how the industry, you know, has, has progressed over the last like two decades.

SPEAKER_05: Like, I think it's just really hard to beat open source on any, on any technology, like the momentum you get, you know, with it.

SPEAKER_01: So, so that's sort of, that's sort of what I feel like, you know, is going to happen. The, um, the, uh, from a who's going to win.

SPEAKER_05: Yeah.

SPEAKER_29: How far ahead is opening high, if at all, do you think opening eyes 4.0 is much better, 10% better? How much, how big is their lead? If you were to say in the number of months or quarters, and then how soon before open source and, you know, everybody else catches up or exceeds them? Yeah.

SPEAKER_50: Yeah. Yeah.

SPEAKER_05: So on, on, on text, text based models, like I, I think, I think right now they, um, all testing internally, like, you know, it feels like, you know, they're still ahead, but the, the gap has been closing every quarter. It's actually not significant right now.

SPEAKER_50: Like it's not significant in the sense that I think now, like our team, for example, is, is, you know, continuously thinking about, like, you know, we need to actually use, you know, the smaller, faster models. Cause you know, like, you know, like, you know, because they're faster, because they're cheaper.

SPEAKER_01: And because, you know, like the, you know, it's sort of like, you know, how you design, like, you know, sometimes you can, like, you know, like if you make 10 requests and like sort of triangulate those, you know, interesting things.

SPEAKER_05: You can actually get a better response and like making one like costly request to a costly model. So there's already in that, in that, um, in that domain where it's not straightforward anymore, like, you know, to decide like what's the right model.

SPEAKER_56: So like the things are getting really quite close.

SPEAKER_121: Uh, how do you define AGI?

SPEAKER_00: You must talk about this and think about it. General intelligence. What's the test that you put on it?

SPEAKER_14: I mean, obviously we have, you know, all kinds of the classic, uh, tests, but what, what do you think would be a reasonable definition of artificial general intelligence that we could all agree on?

SPEAKER_198: Or you might agree on?

SPEAKER_01: Well, like, you know, in an, in an enterprise, like, you know, when you feel like, you know, there is, there is a person today, you know, they have a role, um, you know, to perform.

SPEAKER_50: And, and that role is not completely taken over by an AI agent. Got it.

SPEAKER_05: And, and, and I think that's sort of what I, you know, what's our definition, like, you know, within our context, but I would actually also tell you, like, you know, we, we, we talk about big things. And I think we're, we're far behind, like, you know, in terms of like, you know, where real technology today is, um, you know, people talk about having copilots. I, you know, I feel like, you know, that's, that's a big bar, like, you know, as a word, you know, to describe the technology that we actually have in front of us today. I mean, like, you know, this, this, this is really powerful, but there's a lot of work to be done. I mean, like, you know, it's a copilot.

SPEAKER_02: We're in the copilot phase. And the next phase will be, I feel like it's not the, we're not in the copilot phase.

SPEAKER_01: Maybe just developers are. Well, I, I think the, like, you know, you are getting, like, maybe, you know, 10% of what an assistant would do for you.

SPEAKER_05: You know, like, you're not getting copilot is actually a lot more, like, lot more stronger than, like, an assistant, like, you think about your own personal life, you can have an assistant, you can actually have somebody who can replace you as a copilot. I think the AI technology is actually, like, not even at a place where they can do a better job than your EA. Interesting. So I would agree with that.

SPEAKER_105: Yeah, it feels like, I like your definition.

SPEAKER_29: One of the employees at work gets replaced, and you don't know it's an AI. I like that. Yeah, pick a random person in your organization, replace them with an AI. And when you talk to them in Slack, or you talk to them in, you know, GitHub, or whatever, you talk to them in a Google Doc. Yeah, you can't tell the difference. That'd be a pretty good one.

SPEAKER_14: Yeah, I feels like we're making, you know, steady progress there. But yeah, it feels like we're in the copilot era. But yeah, you're right. I never thought about that way. You wouldn't give them you wouldn't have them control of the plane right now. No, no, you wouldn't go to the bathroom and let them fly the plane.

SPEAKER_207: That's right. She'd be like, I'm going to stay here and watch you fly the plane. I'm not quite sure. I trust you.

SPEAKER_20: Yeah, but but at the same time, like, there's this real value. Like, you know, like, you know, I think, you know, even between like, you know, we, we want to be that assistant, you know, for everybody who works.

SPEAKER_05: And I think we're bringing, you know, like a great deal of assistance.

SPEAKER_02: But it's, it's a lot is, you know, it's a long road, like, you know, like, you know, how do you charge for it? Is it per seat? Is it by data source? Or do you just look at like per seat?

SPEAKER_173: Per seat 10 bucks a month or something 20 bucks a month?

SPEAKER_05: Yeah, a little bit more. A little more. Oh, okay. Yum yum. But the, yeah, but that's, yeah, that's the model. Like you can connect that. Yeah.

SPEAKER_14: Got it. So if a thousand people, a couple of hundred dollars a year per person, it's probably, yeah. And that's what you're going after. Midsize organizations need this. It can't be like 50 person companies, maybe not worth the, the juice ain't worth the squeeze. Are you going after the midsize?

SPEAKER_01: We, we, we are focusing on companies from like a few hundred people all the way to the largest enterprises of the world.

SPEAKER_05: The need is quite universal, but like, yeah, from a focus perspective, like, you know, we are like majority of our business is actually is an enterprise.

SPEAKER_98: And it takes a while to get all these services into the database, right?

SPEAKER_215: It's got to take a couple of weeks or months to tweak everything and get it all plugged in, right?

SPEAKER_01: No, green is actually green is actually very turnkey. That's actually one of the big requirements for when we started our company was like, you

SPEAKER_05: know, we can bring, you know, clean to, let's say 2000 person enterprise, you know, the, you

SPEAKER_50: know, and like, you know, it's up and running, you know, within, within, within a day. And that's pretty good. And, um, yeah, cause I think like the, I, I think one thing that helped in that is help

SPEAKER_01: is that like, you know, like the, the new, like, you know, SaaS based it environments

SPEAKER_50: are actually quite accessible and, and you can be up and running pretty quickly. All right.

SPEAKER_00: Listen, everybody check out green. You have green.com. We should've been. Yeah. Green.com. All right.

SPEAKER_57: Good domain. Pretty great domain. It's a million dollar. Well, maybe half a million dollar demand right there in my estimation.

David Friedberg: It's in the dictionary. Uh, great job and everybody check out clean and we'll see you all next time. Bye. Bye. Thank you so much.