SPEAKER_00: there's really not that many great local italian places oh there is a good pizza joint uh called centro pizza on broadway okay so you know broadway and burlingame there's like the street and there's the great street yep broadway has a place called centro pizza and they make brick oven pizza and it's amazing it's the best pizza in the peninsula i found um centro does it good SPEAKER_07: yeah it's pretty great okay there's your cold open everybody this week in startups is brought to you by in broker's startup insurance program helps startups secure the most important types of insurance at a lower cost and with less hassle save up to 20 off of traditional insurance today at a broker.com twist while you're there get an extra 10 off using offer code twist lemon.io need to speed up your product development without draining your budget hire vetted engineers from europe at lemon.io go to lemon.io twist to get 15 off for the first four weeks and eight sleep good sleep is the ultimate game changer now you can add the pod cover to any mattress go to eightsleep.com twist to check out the pod cover and get 150 off at checkout SPEAKER_10: all right everybody welcome back to this week in startups my guest today is mike anoop he's the co-founder president and head of labs at zapier if you don't know zapier i'm about to make you happier zapier is SPEAKER_12: an amazing tool i discovered god it's close to a decade ago that helped me do really interesting automations between google docs my email you know basic stuff if somebody signs up for my newsletter put SPEAKER_10: them into this google this uh google sheet if it's somebody's in this google sheet pipe it into my slack room when somebody signs up for launch fund four as an lp and i've been doing these automations over the SPEAKER_12: years uh and i train everybody on my team to learn how to use zapier notion coda the google docs suite um SPEAKER_17: and zapier and all these products because you can automate so many tasks your partner and your SPEAKER_00: co-founder wade's been on the pod i think twice in the past uh but mike is this your first time on the SPEAKER_17: pod i think so yes uh first time thanks for having me well i just want to say also congrats i i mean when people saw zapier and i guess your contemporary if this then that was a there was like a couple of companies trying to do this and everybody was like yeah that's a niche business it is not a niche business explain to everybody when you started the company then we'll get into all this ai stuff which is SPEAKER_10: why i wanted to have you on because ai changes everything with what you're doing and we're going SPEAKER_22: to do a bunch of interesting demos and talk about how startups and everybody can be using ai and zapier SPEAKER_17: to plug everything together but when did you when did the company start and then when did you realize SPEAKER_10: that you were onto something big and then what's the footprint of the company now because i hear all kinds of numbers like somebody told me you're making over 100 million in revenue i don't know if that's true but where's the company at today and where did you start yeah um well i think it surprised SPEAKER_28: me as well in terms of like how big the business could get when we started it um brian wade my two SPEAKER_30: co-founders and we got started back in columbia missouri small college town at university of missouri we got started a startup weekend so that's kind of what brought the three of us together and we were all working with like apis and our day jobs and side jobs i was like one of the early moderators and like big users of the facebook api when it came out in like 2009 2010 um so you're all using these apis um like in contract work and we're just doing the same things over and over again with them and i think brian was the one who pitched the idea at startup weekend and the idea was like hey there's this huge you know wave of apis they're really cool like wouldn't it be even cooler though if like more people could actually use them because you know sort of has still have a very technical bench to be able to take advantage of them um and that was kind of the thesis and as we started looking online and you know if you sort of go online and search around for using the easy apis or more more commonly what you so as you'd search for how do i connect these two services together all you would find on the internet back in the 2010 2011 2012 era was basically developer documentation you know you'd find a stack overflow link of like oh yeah great here's a here's a bunch of code you can use to connect you know salesforce with gmail for example um or high-rise with you know or something popular tools back in base camp yeah yeah really old school there you do that same search today and like the sort of landscape of the results it's totally different um but that was sort of the the sort of landscape that it looked like back then and i think our observation was you know okay well you know you see all these forums where folks are almost begging the vendors for integrations you know you go to say the high-rise forums and just see these forum threads with like hundreds of their users begging for like hey can you add this like random xyz integration and it never really made sense to them to make add more than one two three or four you know the top requested ones just because the long tail it's a bit of an n squared problem right every new app that gets added there's an integration that wants to get integrated with it and we realize well okay we're probably never going to like capture you know the direct native integration experience it's like the vendors are going to build those directly themselves but we can provide this sort of ubiquitous platform you know maybe we can get five ten percent of like all of the integration markets out there uh because we'll be able to service a set of users that just the vendors themselves are never we're going to be willing to service that was kind of that knows the original thesis that like hey this could be more than just like you know a small niche it's about a small niche sas company um over the first few years as we got kind of uh started building you know i think one of the things that really uh changed like my perspective of what the business was i always thought for a long time i was actually not a personal user of zapier for the first couple years like um i was building for our customers and i always sort of saw it as kind of boring productivity software um and that was like that's how i view the software you know it's cool great business like boring boring b2b software and what sort of started to change my mind about it was um several years in we started going to a lot of these like conferences with our users and with partners and we started having a lot of people coming up to us and like sort of SPEAKER_37: like shouting our name like shouting giving us huge high fives and just being so if you like SPEAKER_38: thank you there was like passion from the user base yeah it was weird that was palatable right SPEAKER_17: when you double clicked on that because this is really the key you had what um we call in the industry uh market pull not only were people were looking for this product and this is beyond product SPEAKER_10: market fit you had people searching the internet how do i integrate these two things how do i create some glue how do i solve this problem and then they're so delighted they would scream your name at a SPEAKER_41: conference at you yeah yeah they'd see that big orange t-shirt and like that would they'd run out SPEAKER_37: to us and like this is a great feeling yeah and what it really was was like these these these users SPEAKER_30: were not and certainly the software can be used this way they were not using the software for pure like optimization time optimization use cases not to like hey save me five minutes a week or save me an hour a week these users were like doing something that was like transformational for themselves or for their team or for their business it was like it was almost like a skill in mind like hey i thought i couldn't do this and because the happier existed i was able to do it um so you know you think of like a solopreneur or like a one or two small person business that thinks like hey it's out of reach for me able to build a business and because i actually have access to these tools and i can build an inbound lead generation through you know like a google form and a lead scoring mechanism and an outbound email thing with mailchimp like i can actually do it now maybe i was budget constrained to be able to hire a developer and i didn't have the skills or the time to go learn how to be an engineer to kind of stitch together all these tools myself um so it unlocked a lot of focus i think to be able to do do things with software that yeah just previously felt out of reach and i think that like that feeling was what drove the drove the passion and i don't know it got me way more excited about really trying to grow the business as much as we could yeah and the company's now worth five SPEAKER_22: billion yada yada you've raised a ton of money you've got how many customers how many employees SPEAKER_30: ballpark several hundred thousand paying customers over 10 million people folks have checked out and tried zapier over the last decade we've been around quite a while at this point um over 5 000 apps at SPEAKER_48: this point and you've blown past 100 million in revenue that that rumor's true the last number we SPEAKER_50: shared was like 150 million wow that's just mind-blowing it it took a decade or just over i guess right SPEAKER_22: you're kind of on your 10 year past your 10 year anniversary yep um but it really took if you look SPEAKER_17: at that 10 year plus journey at what point did you have that inflection point where hey this is really uh starting to ramp up because i think some people get discouraged during those first couple of years when maybe you have light product market fit and like you said you didn't think it was a big deal SPEAKER_10: if you could pinpoint you know that moment when you said hey you went to the conference people start yelling out your name they see the orange shirt what moment in time was that and then when did the SPEAKER_20: business actually start to crank and make revenue yeah i think the 2014 probably was around the year SPEAKER_30: where we started just to get enough like recognition in the market from like users and customers and partners to get that like passion and hear the excitement that was also the year that we got profitable so you know one of the other unusual things about our business is we've raised very little venture capital only a million dollars back in 2012 once the balance sheet um since then we basically run the business on cash uh from customers um so then any of those fundraising you've done is just secondary or something since then yeah yeah we we've sort of offered we wanted to um offer an equity program for everyone in the organization a couple years ago so yeah we went out to the market to get it we've never raised monies we didn't know the share price it's actually worth so we went out SPEAKER_64: and actually got a share price and said okay now we're going to start we can build our compensation models around that and actually offer that to everyone now going forward in the organization SPEAKER_22: and um i remember salesforce ventures was one of the early investors obviously went to y combinator another great hit by yc um and then you uh just set up a secondary plan for your employees how do you uh everybody has a lot of questions about that um how do you look at executing it um you know this employee stock option plan equitably fairly keep people motivated yada yada SPEAKER_30: you have a process there yeah there was there was definitely history too it's pretty interesting for us um so when we first started the business you know we were three three dudes from missouri so we really had more of that i guess ethos and how we kind of ran the business which was like you sell products you make money you scale the business based on what you make um you know we just didn't have the like silicon valley like raise 100 million dollars that wasn't sort of our default operating model uh coming into the business and because we were able to get profitable really early um you know one of the things we thought to do we actually we actually did offer equity to early employees like we kind of you know we went through yc so we got the like kind of traditional startup advice like oh we set up an option pool and you know offer equity so we did and the reality of those all those early employees because we were hiring out of our networks when we've been remote since 2012 as well we were hiring out of the midwest we're hiring internationally in europe and like none of those early folks really valued the equity part of the no they've never seen anybody SPEAKER_22: make money off equity in fact they've seen people get lied to with equity in some of those places SPEAKER_75: that would ever be worth something so they just are like hey give me cash and if you want to give me a SPEAKER_30: little extra cash we switched we switched to um profit sharing really early on probably it was probably around 2014 when we got profitable i think when we sort of switched over that model and said you know this is what our sort of teams are telling us they want our employees are telling us they want so like let's let's talk about instead and it was way less overhead too for offering it because it's you know give we had a global sort of uh employee base just like the logistics of offering perhaps sharing work were a lot simpler so we we actually bring in that model for a really long time and up until you know closer to 2019 going into 2020 where you know zapper wasn't a lottery card anymore like it was in the early days like okay we built a real business you know north of 100 million recurring revenue this is not something that's like going to go away so we said all right we want to start offering equity for everybody and give everyone a chance to sort of like participate in the upside of the business at that point um that's where we kind of kicked off the logistics to like okay let's actually go try to figure out how we're going to create a secondary market can we get a share price for this asset figure out what it's worth build that into our sort of our compensation models so now we still do have like a bonus program that looks more traditional like we kind of pivoted our profit sharing into more of a bonus program um but we added in the sort of mailchimp famously SPEAKER_80: you know did this so we we call companies like this internally at our firm alicorns you know it's like a SPEAKER_22: unicorn in a pegasus and my joke was they fly over traditional funding rounds com.com we invested in that company when it was like a four and a half million dollar company and nobody would invest in it 40vc said no we said yes and then alex and michael came to me they're like oh we're raising a little bit of money and doing a little secondary are you cool with that i'm like yeah whatever and they're like yeah it's at 250 million and then i think the next round after that was 1.x billion and they didn't need the money like you they just did it off money and 37 signals was similar um SPEAKER_00: what's the uh oh survey monkey was another silver so survey monkey and mailchimp both did it this way it is possible um you raised under 2 million and you got to over 150 million in revenue just let that sink in that is the definition of market i'm not like dogmatic about not raising money um no SPEAKER_30: you know i tend to like zapier's done some weird stuff right we got profitable early we've been a remote company since the very beginning of the business back in 2012. um you know i i like to think of zapier a bit of a as an existence proof of like alternative ways to grow and scale companies um now i'll also be clear i think a lot of those things got pulled out of us rather than us like expressing them intentionally or proactively like hey we found a niche in the market that was like underserved and we're able to actually like do this thing um you know one of the things i think that's under realized about zapier is i actually think it's one of its fundamental innovations is a bit of a business model innovation more than anything else well so like in the you know in the 90s and 2000s like integration was is a thing like apis existed yeah middleware microsoft biz talk if you remember yeah uh that that tech that tech um that product um but it's just like costed millions of dollars to have like huge fleets of you know integrators basically custom engineers and it to come into your organization like stitch all this software up together and our sort of i think innovation was hey we found a way to actually deliver the software at a in a usable fashion and we found a way to reach customers through search which cost us sort of zero dollars right and so every time we're adding new integrations to the platform which by the way are also built by a majority by our partners for free there's you know not a there's no money that changes hands there so like partners are building integrations sort of for free to get access and deliver integration their customers um that opens up and adds new search landing pages to zapier which we get new customers then from google for zero dollars effectively so we're able to sort of find this flywheel that um meant we were able to acquire customers for very very low cost which means we can deliver the software at 10 bucks a month 15 bucks a month 20 bucks a month starting price yeah and that let us reach a sort of set of customers and users in the world that otherwise just weren't being served historically um and so that's just that is how we make money still through today we have software it's a service you know we have um starting around 20 bucks 50 bucks 100 bucks goes up from there you know we add on layer on features around teams and companies and organizations and things like that we're starting to add more of a traditional sort of sales and go to market plan as well for like up market customers and mid market customers um but by and large we make money directly by selling software and users and charging for the amount of the amount of like tasks and usage they have with zapier listen i work with super early SPEAKER_89: stage companies at launch like literally year zero they haven't even incorporated yet and then we hit the series a people have thousands of dollars in mrr and they maybe they've only raised a couple of hundred thousand before that series a and they don't have their insurance set up and in fact we recently had a great startup that didn't have dno and we had to really stop everything because they were having board meetings they were making massive decisions there were legal issues and they didn't have the basic dno insurance that protects directors and officers so we send them right to a broker and broker is business insurance built specifically for startups a single application will help your startup get four quotes for four lines of coverage in 15 minutes think about that four quotes four lines 15 minutes and they're going to connect you with one of their expert brokers for unmatched service that goes beyond your policy we use it at launch it's easy peasy lemon squeezy it's easy breezy what more do i need to tell you i use it i love it a lot of our startups use it they love it try and broker today with the code twist and you'll get 10 off their startup package in broker.com SPEAKER_60: twist that's e-m-b-r-o-k-e-r.com twist and use the code twist for 10 off okay let's get back to this SPEAKER_22: amazing episode something interesting happened recently um you mentioned that you were playing with the facebook api i believe um they quickly deprecated that jamoth talked about it on all in i SPEAKER_95: think recently when they realized like hey wait a second this is like the core to the business we don't want you having access to this uh zuckerberg being savvy and then famously now uh elon and SPEAKER_22: reddit i just had steve huffman on a couple weeks ago uh or last friday i think actually was and he talked about them i don't know if you saw the episode where he talked about turning off SPEAKER_17: access to the api or i'm not turning it off charging a reasonable fee monetizing the api so i'm wondering SPEAKER_00: just with this collection of examples they're all happen to be social sites which is interesting SPEAKER_97: um yeah that's my observation as well okay linkedin is another one from earlier that you didn't mention SPEAKER_00: that craigslist has no api never did and will in fact sue you if you use any of their data for through a scraper yeah so maybe you can talk about how if you want to pick out a business example uh mailchimp SPEAKER_30: and shopify also went through sort of an interesting api breakup uh i think when they started competing with each other like you can go look at their like public blog plus around this but yeah effectively you know mailchimp i think was starting to introduce products in the market in order to compete and like those some of those were going head-to-head with shopify so you know both of them sort of mutually said well it's not great for my business just be giving away value to my competitor so like we're gonna start like disallowing our use case or not not providing the same native integration that they previously had and one funny outcome from that was uh like we had customers for both of them basically coming to us and say hey my like vendor of choice is gonna stop like supporting this native integration can i just use that paper and uh that led to both of shopify and also like basically just sending us a lot of their users who depended on the native integration because we have a lot of sort of stepping as a bit of a neutral like while you were speaking i typed in SPEAKER_17: mailchimp spotify api and you're the number one result i think on zapier because you're uh neutral you're sweden um but what do you think of this charging for the api because obviously that changes your SPEAKER_22: business uh you now have to i guess um ask people to put in their tokens to uh do this and then i i guess with ai and open ai specifically you know they you know they have calls and stuff like that does that dramatically change your business or do people just have to fill up their you know most like SPEAKER_30: first party vendors most software providers is actually the preferred way to go like opening eye is kind of introducing a bit of a new way to do product monetization where like hey you have a direct billing relationship with open ai and if you want to use a platform product like zapier to plug in that intelligence layer into a sort of a workflow you bring your own key right you bring your api key to zapier i my my sense is this is actually like the smart savvy way to go about it for a lot of these products i kind of actually wish that things like you know twitter and all them would actually adopt more of this model where it's like okay if i have a i'm going to establish my direct billing relationship with my sort of you know first party vendor and then allow that that user to bring their token to other tools then you just charge for the user to have access to those tools you can you know you can say okay well allow access to sort of the api for say twitter in this example allow twitter apps as long as that customer is you know twitter blue and already paying for it for example um i think that's sort of like i don't think anybody wants to get disremediated it's like why SPEAKER_103: would you ever let like a third party company charge on your behalf anyway i think it's SPEAKER_17: yeah it got kind of weird i think if you look at the time period where you started your company it started right at the kind of end tail end of web 2.0 and the web 2.0 movement really where api started in 2005 6 7 8 people were just looking people were under resourced twitter was under resourced as a company they couldn't raise a lot of money they couldn't afford to have ios developers uh and you know when apps came out so they're like hey you you all have at it reddit didn't have SPEAKER_10: enough money it was kind of free outsource development was how the community looked at SPEAKER_00: it and you're like hey you make some value for yourself in the world um don't do anything stupid and you know have at it uh and then the problem i guess of course becomes then when you have to go SPEAKER_22: public like reddit does uh or twitter has to turn a profit eventually uh you need to tighten the screws here and then you find out whoa these people were really abusing the api they were taking our data or users and selling it to people and get all these kind of gray markets etc so i think it's actually kind of cool to for the idea that you could just fill up your card and i have a couple startups who are doing this at openai they fill up their card and uh yeah then they run it down and it's like okay i need to get more what do they call that uh card that you get when you are in college and you go to the SPEAKER_117: cafeteria whatever that card is called meal card yeah your meal card it's kind of like your meal SPEAKER_120: card like just points on the card get points on the card so uh all right i think for a lot of like SPEAKER_30: b2b companies is apis are are actually in their interest right where social companies have a bigger downside that they have to protect against which is disintermediation like you know i think in twitter's case the stories that i had read on the online was you know folks basically there was like a bunch of first-party uh platform apps that were using the twitter api that were starting to get consolidated under one owner and twitter sort of got spooked and said well shoot we don't want like our business front end to get disintermediated with our users through one owner so like we're gonna tighten things down um so if you can figure out a clever way to like protect against that outcome from happening then i think the sort of it's all upside from a sort of monetization standpoint i think users at this point in time around subscriptions are like used to the idea of like oh okay if you're going to provide an ongoing service there's you know there's an expectation that there can be a cost associated with that whereas on the b2b side and the prosumer side integrations are like purely upside and there's almost no disintermediation or risk in fact these integrations are usually really good we actually ran a bunch of studies one with typeform that showed uh integrated users churn SPEAKER_124: like 10 less oh yeah i mean we use super sticky the fact is we use notion we use typeform and typeform SPEAKER_10: we love and survey monkey we love we love all these products if they didn't have integrations um SPEAKER_17: yeah we might actually use uh a product like you know google sheets allows you to do forms we might SPEAKER_10: use a pro a product like google sheets instead and just be like ah it's not as good but it has SPEAKER_00: integration so we'll go with that right it's almost like you would pick we we would not use certain products certain sas products if they didn't have our users too they will select we we have a SPEAKER_30: quite a few users lately who actually say i'll go to our like app directory page on zapier and use that to choose that part because i can like trust so at least i know what integrates with things in like SPEAKER_92: they have a sort of the right mindset around that okay listen you got an idea for a tech startup great SPEAKER_60: you think you want to change the world you think you got this this is the one well you've got that same problem that we all do you don't have an engineer or you don't have enough engineers to make this happen and you need product velocity you need to go fast and how are you going to go fast and how are you going to control your burn rate if you got no engineers well what if you had a partner who could provide you with more than a thousand on-demand developers and those developers were all vetted experience result oriented and passionate about startups and building great products well what if they also charge competitive rates does this all sound too good to be true i know it does well then you need to head to lemon.io because your dreams have come true right now startups choose lemon.io because they only offer hand-picked developers with three or more years of experience and ones that have strong portfolios only one percent of candidates who apply to work at lemon.io actually get accepted and if anything goes wrong lemon will get you a replacement asap a SPEAKER_89: couple of great launch founders have worked with lemon.io people in our portfolio and they have had great experiences so i want you to learn more at lemon.io twist and when you go there you're going to find your perfect developer or any entire tech team and you're going to do that in 48 hours or less and twist listeners get 15 off the first four weeks i want you to stop burning money i want you to hire developers smarter and faster visit lemon.io twist and give it a shot all right i want to talk SPEAKER_141: to you about i want to actually talk to you because you have some big thoughts on ai regulation we'll get to that in the tail end of the show in the third act but for the second act here let's go over some SPEAKER_22: of the cool stuff people are doing on zapier uh with integrations because my core and i'll let you fire up your and share your screen while we're talking my core premise here is there's gonna be like SPEAKER_10: a permanent hiring freeze at companies because everybody's getting 30 percent more efficient a year using these tools at least i think and they will continue so why add more people if writing a job SPEAKER_22: wreck takes more time than writing a script and automating something so that's the core tenant i come to with this you you think that resonates with most businesses be it when faced with a hiring wreck SPEAKER_10: versus making things more efficient with tools like yours what should you do that's what a lot of users SPEAKER_78: want like users want automation technology to do work while they sleep like that's that is what buyers SPEAKER_30: want that's the dream i don't think we're there yet and all the use cases even even internally um earlier this year in march we held a company-wide hackathon we actually told everyone the company our pencils down we're going to take an entire week and the company needs to re-educate themselves around like what is possible with this technology what's not possible and figure out ways to work in your workflows and we've got a 20 of every individual person at zapier 20 of all employees worked in some sort of ai into a zapper workflow that they use as far as i actually have not talked with a company that has a higher percentage than that yet so i actually think there's amazing perhaps interesting things to learn there but um you know it was it was you know it's existential for us uh you know we had to do this like ai and automation are essentially synonymous i think going forward SPEAKER_37: and you know zapier's business is basically hey it's software that works while you sleep right so SPEAKER_17: there's and you're of course referring to auto gpt or baby gpts i guess people call these where you give a set of instructions to an ai and it performs them over time perhaps even getting better at the SPEAKER_00: task uh with some scripting or instructions so let's do some examples well it's not too far apart from SPEAKER_78: you know even how you think about um what zapier does today it's just hard to use for most people SPEAKER_30: like zapier if you set up a zap it's a workflow it is going to do something forever and without you using the keyboard to come interact with it but it's really limited it's constraining right it's like rigid and it's also hard to set up you mentioned at the top of this podcast like hey i have to educate my new employees how to use this technology right it's just not easy enough to actually like use right out of the box yeah the penetration rate of this tech isn't very deep yet so yeah i think which is the opportunity right folds but like it's still too hard to use and i think that's where the language technology really has a chance to shine um but yeah the demos that actually have uh are the first one is actually they so they're mostly centered around the chat gpt plugin that we've launched uh back in we're one of the launch partners with openai back in back in march um and i'll show this off in uh this order we'll go through maybe a couple examples and we can end on um one of the epis how this actually works so this is a example of i think how we've seen most of our users like even in some internal employees it's after adopting this stuff is you know they'll still i've heard a lot of anecdotes actually internally where basically folks want two tabs open all day they'll have a chat gpt open one tab and zapier open the other tab um a lot of reason is because the models of how the software works are uh like completely different right chat gpt is a piece of software you have to interact with in order to get value out of it so you know this example is one i've used myself is you know i grabbed an email from my inbox that matches a certain format and you know draft a automatically draft a sort of reply to that okay so that you're in chat gpt4 you're using SPEAKER_22: the plugins you pick zapier and you say hey i want to check for an email in my gmail account and you've already authorized it to go to gmail and now it finds the latest email and does a reply for you SPEAKER_50: yeah this one summarizes the reply and then i think if i kind of just you know zoom forward here SPEAKER_30: what one of the actual downsides with sort of the plugin architecture on how chat gpt works right now is everything sort of has to go through these like confirm flows which is i understand why they do it um you know sort of the safety argument around it however i do think that there's probably some edge cases where we actually take a stronger safe stance than than than their platform does and like it kind of creates a weird system where like two safety systems are trying to be in the middle and it creates a really awkward user experience um so i think there is a sort of more stuff we can do there but yeah here's an example of where you know all right the plugin's going to come back and grab the email response and summarize it back in yeah so this is an example where we actually open up a tab on zapier to give a preview of what the plugin action is about to do um i think this is an example where uh we've actually answered our own safety things to like let the user explicitly know what actions are going to do on their behalf instead of just letting them kind of roam free uh on zapier on your account and now we're back inside chapter gpt after the user's confirmed and it's pulled an email and summarizing uh the email and i think then in this demo we actually even follow up and ask chat gpt hey can you sign that with with my name and rewrite it in my tone and then uh send it through SPEAKER_10: gmail as well and you can actually fire off now from the chat gpt interface using the zapier plugin if you've authorized it it makes you go to that step it will actually do the send from the chat gpt SPEAKER_30: interface it will in this case we're creating drafts this is kind of another one of those like probably tips i would have for most folks that are adopting this tech is like you know the technology is really really good at drafting things so you almost want to lean into use cases where you get to get a preview of it and you can add it and mark it up and have sort of control over it before you press the send button yourself you absolutely can hook this directly up to like sending an email directly um but the one we found most folks inside zapier adopting is you know these flows where it goes through creating a draft for you being able to review it and approve it before hitting send basically the vision for what we want to try and get this to is it feels like no off flow um you know where wherever whatever product you're in if you're in chat gpt or you're in any other sort of ad product you need to plug in sort of an action library into it um you know you click a zapier button you say that that vendor says hey i'd like to get access to you know your gmail account your salesforce account and you know your type form account and the user says you know it looks like an awful little pop-up the user says yep that sounds good and now you're back inside the sort of first-party product and you can go from there um in the initial version that we released it's like one there's one extra step which is in addition to having to sort of approve it and allow you also explicitly today have to choose which actions from those apps you want to like allow the uh sort of chat gpt to have access to so today for example with the gmail when you saw um when we set that up you had to like say yeah okay i want to allow chat gpt access to gmail but i also want to have it allowed to send a draft email so there's one extra sort of step today we have to choose those and SPEAKER_158: that's some somewhat of a limitation of sort of the language model technology and somewhat of a SPEAKER_00: limitation of the api experience overall right now yeah it's it's there is a it's a little kludgy you have to log in i remember doing this you have to log into zapier and then there are some links that you have to go to to make sure you can search your gmail make sure you can send from gmail and uh that i'm sure will be abstracted in the coming weeks and months yeah yeah i don't i don't SPEAKER_177: disagree with you at all by the way um and yeah i think if you and i have i've looked at and sort of SPEAKER_30: obsessed over some of the like usage and some of the numbers from this stuff um you know i i do think that a lot of the plugins and folks i've talked to retention is is a problem right now with them you know i think if you kind of look at like it's almost like a bit of a numbers game um you know if you're sort of able to spread your user base over enough users you can sort of find a percentage of them that you're gonna find you sticky use cases to build this like chat and plug-in thing into their workflow the reality is most people in the world haven't even worked chat gpt into their workflows yet so like no asking them to then add on a plugin that is also like you know an active development like it's i i think we're still quite a boys away before you're gonna see like kind of the refinement you needed from a lot of these this like plugin ecosystem and how even just getting the penetration of chats beginning to like legitimate like sticky use cases i think is still still a search for most SPEAKER_00: most well you know it's it's it's going to be a slow process and then it's going to be a really SPEAKER_17: fast one because once um you know the first 10 people in an organization figure this out and they become bionic and they're able to do really interesting things with their gmail box like say who are the people that i was you know emailing with back in 2012 to 2020 that i'm no longer SPEAKER_10: emailing with summarize my conversations with them and then suggest which one you know some emails to catch up with them and like whoa that's going to be super powerful or like you know you're a venture capitalist hey what founders um was i talking to 10 years ago what are they doing now and it's like SPEAKER_00: like this is going to lead to um being able to do things that would take so much time nobody would SPEAKER_17: ever even consider doing them like you would have to hire a full-time person to hey go through my emails from you know the 2010 to 2020 period find every founder put them into a google sheet and then look SPEAKER_10: up on their linkedin and see which ones are still at the same company boom it's like yeah whoa and it SPEAKER_30: works and it works so it's super powerful we sort of see that exact same thing that's the message you know we delivered internally which is how we got so many folks that have started it out in real use cases is you know hey there this is like a chance to go learn the technology the future is not going to be like and i can't i really struggle to think of any sort of technological disruption that like you know displaced jobs in a quarter but over the course of two three years five years like you're going to see folks that accelerate ahead because they understand and use the technology you're going to have like infants in the job market who natively know this stuff um especially if you look at the adoption rates of like chat gpt and education coming like all those folks graduating through college and entering the job market like they're going to have a skill set that i think a lot of other folks probably SPEAKER_103: probably won't have um i think that's gonna be a leg up for a lot of them any other uh language SPEAKER_00: models now built into zapier that have integrations google bard yeah we have well none like chat gpt SPEAKER_30: where we have a plug-in launched yet um there's nothing to announce at this point yeah um we do have all of we have way more language models actually built into zapier sort of first party though so uh the kind of this is the kind of the brain vendor about it right um when we actually went to go build the chat gpt plugin one of the reasons we built that was you know we saw this huge influx of ai apps launching on zapier we had like hugging face and human looper all sort of getting built on zapier and we sort of realized like oh wow there's this huge explosion of ai products that's happening in the market that are not going to get on zapier and we wanted to offer an api to them to be able to bring zapier's integration platform into their products just felt like first time we've ever actually launched a public api it's a bit of an almost embarrassing point that we're 10 years in it's like we're finally now just launching a public api that other sort of vendors can can pull in but we felt like it was sort of what needed to happen at this point given the pace of um like products that were getting released uh but in the more traditional way where you you know go to zapier.com and you're building workflow you know we we have uh anthropic now with claude is on zapier we have uh the google the bard version we've got um opening uh integration as well so you know you can build those into more traditional workflows um but i do think some of the more exciting interesting ones are like the paradigm shifts where you have like a completely different you know front-end SPEAKER_12: interface for how you build and use this stuff if you want to get ahead in your career you need to sleep well and if you want to be a stallion if you want to be a workhorse you need to get a great SPEAKER_60: night's sleep it's that easy so do what i do and that's getting an eight sleep right especially over the summer right temperatures are going to rise you want to stay cool and eight sleep mattresses allow you to set the temperature and they're selling more than just mattresses right now they have something called the pod cover by eight sleep and this fits on any bed it's just like a fitted sheet so if you don't want to get a new mattress you but you do want eight sleep you just get the pod cover it's going to keep you cool all night all the way down to 55 degrees and it's going to improve your sleep by automatically adjusting to the temperature on your side of the bed and you know your partner spouse they get to set their own temperature so no more battle with your partner or your spouse over the thermostat nope it's going to adjust based on you and your partner's preferences and you get personalized sleep reports eight sleep i love it in the winter oh i keep it nice and toasty but in the summer i like those cool crisp sheets and i don't want to have to blast the ac in my house and burn all that extra fuel no i just use my eight sleep i dial it to exactly where i like and uh you know i get a better night's sleep temperature really does correlate with a good night's sleep so go to hsleep.com twist to get 150 bucks off the pod cover stay cool this summer with eight sleep please and they're now shipping not only in the us but canada and the uk and some other countries in the eu and australia so hsleep.com twist for 150 off the pod cover can't go wrong SPEAKER_12: talk to me about linkedin they do not allow people to use their api SPEAKER_10: is linkedin very protective of it because it seems to me like nine nine out of ten times somebody gives me examples of where this is going linkedin comes into play um how do you think about linkedin SPEAKER_178: and and how do they operate with zapier at this time yeah i'm trying to remember it was a it was SPEAKER_30: five plus years ago i think when they went through their api sort of tightening phase they had a pretty open api at the time and then they sort of tightened it down and got rid of a lot of their like uh like automatic message sending stuff the like contact scraping stuff they sort of segmented out into their i think it was around the time where they really wanted to go after recruiters as their like kind of key customer and buyer and so they kind of shaped all of their api usage around that persona and kind of just said you know what all other uses we're just going to delete and get rid of we don't care about them yeah um the one that most of our customers and users wanted was the things like lead hydration right where i could take an email address and go like get information about that lead particularly for like marketing flows you know you talked about hey you've got a you know say a founder of contact form that you have on type form and you get an email address and you want to like automatically pull out a bunch of information so you don't ask the founder to like type in the resume or whatever um so folks were using things like that for those use cases and the reality is the market now is kind of like address the gap uh you know there's a billion different sort of lead hydration companies that all sorts of scrape public information and you know wouldn't be surprised just some of that originally did come from linkedin but you know they had that big public SPEAKER_00: scraping case they had a big public scraping place with i think a company based in israel that they SPEAKER_50: lost um yeah it was like back and forth they won the first one they lost an appeal um i'm not actually sure what the current like status at all is well i mean the the great irony of this is SPEAKER_00: they tightened their grip and said hey you can't do certain things and so what that does is since there's a need like we saw with napster back in the day if people want something you know uh whether it's a tv show music or to enrich a lead uh enrich an email and hydration i've never heard that that's a great term um you know get an email and then get the person's title and where they worked previously SPEAKER_17: it's really smart um then some gray market company uh doing gray hat stuff is going to do it and they're going to scrape all of linkedin and then they're going to back into it and you i know this because i've had so many companies do this with facebook data linkedin data and then legal letters get sent and um the only people who get really impacted are the good actors who want to play by the rules and then the people who get who benefit they get punished and the people who get uh rewarded are the gray hats and the and the black hats who are going to just scrape the information offshore SPEAKER_30: and they don't answer to anybody so yeah yeah sort of the you know mass core of our users and customers too i would almost every time there's an api application or terms of service change or something big like those are the folks getting affected and by large amounts zapier's customers are very small businesses thinking yeah like one to four sized teams and companies um talk to me about like the SPEAKER_120: large enterprises is there anybody who's taken zapier to like a large organization and coordinated it SPEAKER_10: and do you have that feature because right now i have my team using it but i don't know if we have SPEAKER_30: a central relationship between more like we're like building some offerings here we're trying to figure out how to do this basically so that's sort of the tldr um we do have some like examples netflix is probably one of the larger ones where you know they have a very it forward perspective where you know their it organization basically has an identity of saying our job is to make you more efficient more productive like we get that's why we get paid that's why we're sort of here um versus you know a lot of more traditional old school mid-market plus companies where you know hey they're like technology side of the house might be looking at the rest of businesses hey i'm a cross center i'm about protection i'm about control i'm a risk management um and those are just very fundamentally different perspectives i think when you start talking about like introducing new technology like language models and like ai to mission to business process um so we are seeing more and more of like it or is thinking this way um and we're starting to put together some like packages of software and services around actually basically doing what we did internally which is like i actually think we have some expertise now at figuring out how do we actually adopt ai use cases into real workflows that do allow folks to get legitimate time back and allow them to move on higher value activities and use cases and workflows and jobs um and actually make like deploy that into the organization um and that's like the common thing that i get asked whenever folks come up to me and talk to me these days around all this ai stuff is like how are you guys actually using it what are you seeing your users actually use it for because i think a lot of folks are like it's still the tech there's broad awareness for what language models and ai can do at this point i don't think there's SPEAKER_22: broad penetration for it actually into real use cases yeah yeah so talk to me about um how you look at again i guess everybody's got an opinion on this but ai regulation i saw some of your tweets SPEAKER_17: um do you think that this is moving so fast that there's going to be significant negative consequences SPEAKER_00: for humanity uh do you think we need to slow it down or do you think we need some thoughtful regulation how do you look at um this because open ai you know uh here's my i'll speak like most of what SPEAKER_30: i know um certainly like i think there's some interesting philosophical like things we could jam on i don't feel equipped to like have that argument or even debate at this point but i can speak to what i know which is zapier we have like millions of legitimate useful workflows and automation that we've users have set up over the last decade i know that i also know that it is way too hard for most users to use that even today i know you gave us lots of a few praise at the top here and said hey i love that the reality though for most users it's not we've fought for a decade on trying to make zapier easy enough to use for the sort of traditional you know professional who does not know how to code does not know it's not technical um however we still have a long way to go and we've fought on that problem for so long that i think we are reaching some limits of the paradigm of traditional software to actually put like workflow and automation and into the hands of like end users um and i got this ai language technology is the first thing i've really seen i think offers a step function not just like an acceleration of a smith curve but actually like a step function and an adoption rate of how many business users and users can actually set up and use more technical concepts things like automation you know most of the people use appier even though they might not call themselves technical they're still builders at heart right they have that that sort of like um like identity or like okay i'm gonna go create something um and i think this where this like language model technology helps it drives down the barrier to creation by by just a ton so you know i get excited first and foremost i get really excited about the idea of like oh wow well we have like 10 million people who've like tried zapier uh maybe this could get us 100 million folks who've like tried and used automation successfully and i think that's like a really positive thing and especially if we model all the use cases on like what people are using zapier for that's all great you know i just want to make more of those people and um you know i think i think there's a chance to at this point so first and foremost that's kind of where my head goes first is like i think the technology transformation particularly in sort of the prosumer b2b business workflow automation space um and it's not just time saving you know these are legitimate like it is like an unlocking technology for a lot of users around what they can actually use language models and automation to do um it's not things that they weren't otherwise doing on the open source side i do think like so we haven't talked about this much basically what i i actually i know you introduced me as sort of the president of the company i gave up my exact title uh last year last summer in july um i quit the exact team i went to brian away my co-founders and i said i i think we have i got to go on in this language model ml ai stuff we got to learn what this is going to do um so brian said he was going to do the same thing around the same time so both of both him and i basically said we're going to get rid of our exact team roles and we're SPEAKER_20: just going to go full-time and focus on research and engineering for ai particularly in the context of sort of zapier um so back to the laboratory yeah basically out of the exact suite no more fancy SPEAKER_224: bathrooms back down to the garage right back to the garage uh quite literally um so like and i do SPEAKER_30: think like you really do have to go hands-on to learn what's like possible this tech if i look around i think one of the coolest things is when i like look around at like you know my peer group of founders you know folks have been around for 10 plus years all of them are doing something similar which is like going back and actually getting hands on the technology and i think you have to to to learn what's possible um and i actually think zapier plays a role here too because i think this point about you have to go hands-on to learn what it can do and what it can't do is well beyond like hey i'm going to open up a github repo and download code locally and run it things like zapier can be gateways for a lot of users to discover what is possible and what's not same way the chat gpt is offering a view of like what's possible what's not um you know we have users that are like basically experimenting trial through trial and error with workflows and zaps and plugging you know a reasoning engine you know language model into the middle of a workflow to say lead score or you know draft a reply to um you know a message that i received or draft a pull request that i received or you know um score the jira tickets or summarize uh customer feedback and like dump it out into a slack channel but there's a lot of trial experimentation regarding and i think those users are figuring out what it can't do at the same time right they're figuring out oh i shouldn't just automatically send an email no not ready for that yeah be careful i shouldn't insert this into an hr hiring decision where i'm not going to review the decision definitely not um yeah so like i i have a lot of trust when i look at our users of how they do their own experimentations to find what it's good and bad for like i think we got to put that experimentation mindset in our hands so that's why i get really excited about the open source thing about hey the more we sort of can get this technology diffuse into more individual hands at the end of the day i think it's going to allow more people in the world to understand what it's good at what it's bad at and like calibrate and i don't know i have i have a large a high degree of trust i think in sort of folks ability to figure out that and navigate that chart like you know deal with the antidotes of the technology as long as you give them enough time to and that's where my maybe the philosophical hack comes on it's like okay if you really can like sort of drop a like i don't know some sort of step function technology change in a month and like okay maybe there's like a moment where there's like enough disruption there it's worth asking the question right now but like based on everything i've seen the last 12 months of language models that we're that is not what we're dealing with um what we're SPEAKER_22: dealing with is more hey there's something that would take you 100 hours might take you one hour or something that took you 10 hours might take you one hour or you're still going to learn how to do it which is really important i think um yeah but you do agree that this is going to make companies massively more efficient and you're going to need much much fewer people to do much more i mean it's hard SPEAKER_50: for me not to agree with that statement in the limit condition yeah based on where folks want this technology to go like as soon as it exists yeah there's a huge demand for it yeah and so then the SPEAKER_22: question becomes you know we as technologists looking at society uh are left to wonder are there still problems to solve because if a 10 person team can do the work of a 20 person team they could solve twice SPEAKER_10: as many problems it's not like there are not a long list of problems to still be solved in humanity SPEAKER_41: and so that's where i look at it i mean the sort of classic use case probably even wade might have shown this when he was on the show is like the typist example right or even literally the word SPEAKER_30: computer that used to be a profession in and of itself back in the 50s and 60s typist was a profession like yeah silly i got trained in middle school how to type on a keyboard right i was just SPEAKER_22: having this i wrote in my uh i started uh doing some email newsletters again and i was like you know SPEAKER_10: there used to be i started my career as a pc support specialist what a pc support specialist did was they set up your computer they upgraded the memory they put in a larger hard drive they set up your ethernet card and uh then they sat there for two days installing software on your computer using cd-roms all those SPEAKER_64: apple engineers took your job jason where they made the beautiful ios onboarding flow when you get a new SPEAKER_17: phone exactly and now it's like yeah you don't need a pc support specialist to come and set up your computer and your microsoft office for you you can simply uh oh here's my sub stack thank you um i guess there's the section and you know then when i saw you know startups and when you were setting up your startup you were right at the point of cloud computing so did you rack your own service for zapier SPEAKER_244: and have a colo we no we did not linode do you remember that name linode yeah great a box on there SPEAKER_245: somewhere but yeah that was the very first cloud service we used yeah they were the pioneers right SPEAKER_17: and so you you you were the first generation of startup founders to not have to go order pcs SPEAKER_10: and build a rack and find a co-location facility rent space go down to the colo facility have a sys admin and you're you might not have even had a sys admin or somebody at your co-location facility um the generation right before you if you were working at flickr or facebook you were racking servers SPEAKER_60: and you had two or three people on your team who were managing that for you uh yeah linode.com twist SPEAKER_250: and get a 500 credit i forgot there's a plug thanks for the plug i do think one other if i can add SPEAKER_30: one other point of congratulations side i do think one side thing that i i think is i i i'm fearful this is how it's going to play out i'm not sure we'll have to check back in in six months 12 months and see if this is true but i i am a little disappointed that i think the way that most of the research around ai and lm's heading is towards more closed companies like they're not being as forthright and like forthright and sharing of basically like the technology the progress the architecture like they're we're kind of getting into the space where people are realizing how much value is in i SPEAKER_251: think the research and they're just a lot more close pulling up the ladder behind them which i i do SPEAKER_30: think is gonna but that's my like that's almost my like sort of anti like acceleration viewpoint like i think there is a there is a path where actually progress slows down for a little bit of time right now because we're either sort of one approaching some of the asymptote limits of what we're going to exploit out of transformers and language large language model the kind of current architectures we have and two like more research is getting sort of closed up so there's not as much open sharing not as much progress and as you know like the reason open access basically is because of some of the progress that came out of sort of public sharing from another competitive company with google SPEAKER_00: um so i do think that's a bummer that is such a weird move that they went from open ai to closed ai SPEAKER_10: they literally took the reason they existed and reversed it they're like this technology is too SPEAKER_17: powerful for everybody to not have a say in it and for it not to be transparent then they got a couple years in like you know what this technology is so powerful it's too powerful for everybody to SPEAKER_10: know how it works and i i am a hundred percent agree with you do you think that leads to the SPEAKER_30: developer community i completely trust like sam and greg like sure as stewards of technology i can't think of two other better people that i would like try to put charge of that problem um but like if i look at the second order effects of what that then leads to like it does i am a little worried that it does lead to like more closed up nature we're going to see less progress we're going to see SPEAKER_238: less sharing we're going to see less like you know technology getting pushed to the edges um and SPEAKER_22: those good news in all of that is it seems like the open source community and the open source models are advancing much faster i don't know if you saw that google uh memo uh but there was a google engineer who's like listen at the pace that open source the open source community is rocking on this they're going to just beat us and we don't have a mode and either does open ai so it's almost like SPEAKER_10: they're squeezing um too hard and that's leading to people saying you know what i don't want to build SPEAKER_00: on a closed system which you know you may call it open ai but i don't want to have the risk factor of working with open ai so i'll look at some alternatives and the reality is open and more SPEAKER_30: than anyone has pushed for the technology to be developed in public though so i will make the argument in their favor i know that i'm sort of you know poking at a few things and decisions i've made but like i also think we wouldn't be sitting around in this conversation zafari would not have shifted its viewpoint no they've had this i mean they've they've sent mixed messages to the market yeah i think that's fair yeah so it's like the largest message is we have a product that you can use for free and i do think that that has changed a lot of folks opinions around what's possible and SPEAKER_72: and you look at the api every six months they seem to just drop the api price 90 right they've done that SPEAKER_41: twice i think i think you can bet on like costs going down i think you can bet on um context SPEAKER_30: windows going up i don't think you can there's not like a smooth ramp on architecture improvements though i i do think that that is like one thing i've been personally spending more and more SPEAKER_269: my time on lately is explain that to a lay person yeah that's where we understand what your point is SPEAKER_30: i don't pretend to be an expert in this either so sure um but i'll try i'll try this for my best interpretation of what i understand about sort of transformers at the fundamental level is this is basically an architecture that was not necessarily it was like published from a paper at google back in 2017 that paper was the continuation of actually quite a long journey of research as well around what i think originally started around translation like literally translating a sentence from one language say english to french right and the first sort of uh deep neural networks that did this were sort of constrained they kind of fixed the amount of characters that you could translate from x to y and that led to the invention of this technique called attention which allowed you to have variable length inputs and outputs so you could you know the word in english is a different length than the word in french so you could actually kind of deal with that problem and this led that like attention mechanism was uh like its own neural net at one point and the sort of infamous vapor attention is all you need was the dropping of one of these like ancillary or recurrent neural networks because it wasn't like an important part of building a transformer it really simplified the architecture stack and it that architecture stack also happened to be one that really ran in parallel which fits sort of the gpu scaling curves that we've sort of seen over the last decade so those kind of two things kind of in parallel allowed to invent you know progress around gpt you know one two three and now four um and sort of so on from that so but we have like there has not been at least a like well-established alternative to architecture all of the like ai products progress research papers you're seeing like a lot of the momentum and sort of attention has really shifted into what can you build on top of language models right like what can you do with a language model that has this like it's seemingly capability of reasoning right this this capability of tool use this generality around being able to generate content like what can you what can you do with that um you know i would have actually up until last summer i would have said there's like i'm like 99.9 sure large language models are not on the critical path to something like agi that like reaches human level generality of intelligence i've decayed that prediction to call it 80 85 um and the reason for that is there are some things that you can do with like gpd4 right now that no one's productizing and because it's too slow for the language model to generate tokens like you have to let these language models think out loud and they improve their performance you know the classic example is the thing that actually inspired me to go all in on ai last summer which was the let's think step-by-step paper that came out last january and this was a technique that some researchers found where if you put let's think a step-by-step at the top of your prompt and then ask the exact same question again the language model actually boosts its performance because it gives it time to generate tokens almost like an internal monologue where you're thinking about you're letting the model think out loud for what it should do and then letting it reflect over those tokens it spat out to generate its like actual next action um there SPEAKER_273: are some like performance evals that went from like 30 35 percent of like 80 90 crazy step function SPEAKER_128: increases it's really weird if you literally say to the chat gpt let's try that again and uh can you SPEAKER_256: try to find me three more and you just keep doing that you get to like the six or seven back and forth and it's like yeah i got you your answer and i and i did it right and it's like another thing too SPEAKER_277: yeah some partial right correctness is like another big challenge you've even thought that internally um anyway so like there's there's these use cases that like can demonstrate greatness in certain SPEAKER_30: scenarios oftentimes they're unreliable like the example you just mentioned where you had to ask it six times and it got one out of six right okay the question is and how do we like figure out which SPEAKER_97: one is right more consistently yeah where the second one is these really deep reasoning chains where you can actually let the models continue it's like thought process this is the like auto gpt style stuff where yeah you can let the model reason through like a tree-based search of reasoning for almost 30 minutes 45 minutes an hour in cases where we had demos running last fall and it can get to the right SPEAKER_30: answer but it's so slow and so expensive like literally it costs probably a thousand dollars stress to run that like reasoning search you're only going to do it for use cases where the product experience is like okay if it's offline and completely asynchronous and like there's huge ri SPEAKER_224: attached because like the models are expensive so well i mean find me a stock too short and explain your SPEAKER_280: thesis examples yeah um okay offline etl job would be another one like hey i need to process you know SPEAKER_97: a million records of data and i'm okay if it takes three days that's fine just like that was another good use case um so there are examples of of like cases where the model just can do better than how they're getting practiced because generally with like consumer-facing pro-sumer facing products latency matters a lot reliability matters a lot so like all these product builders are self-included or chopping off use cases that are slower expensive and as the sort of cost curve comes down as the context window goes up i there could be some interesting techniques around reasoning that are just out of reach from a sort of keep from a not a capability standpoint from a cost and like SPEAKER_280: performance standpoint that might be coming to reach and you know maybe there's a way to build SPEAKER_17: an having having these auto gpts talk to multiple language models and having dueling language models SPEAKER_128: where they analyze each other's data and they start talking to each other that kind of feels it's a SPEAKER_22: different type of singularity but it certainly feels promising if you were to say hey i'm looking for SPEAKER_17: stocks to short um please go to five language models and ask them about you know the stocks that SPEAKER_10: are most shorted right now and uh then put together a thesis based on their five and you start having the check all different gpt models around the world working on the same problem and then some of them SPEAKER_73: asking reinforcement questions to each other i mean this is you know what the rumors of like the gpt4 SPEAKER_30: architecture have you read about those no tell me what you're describing is uh effectively the the the rumor is unconfirmed as far as i know but uh around how gpt4's architecture works which is essentially they have eight different um you know attention heads or eight sort of model heads that are all trained on different subsets of their eval and uh they're all 200 billion parameter individual models and they essentially like do a mixing mechanism where they like run the input through each one and they mix the output together from the log problems and use that to generate the final tokens so not too far away from what you're describing where you have like it's like a mixture of experts i SPEAKER_37: think is how they would describe it each head is an expert of a different type of reasoning or a different type of input problem and you try to figure out an expert SPEAKER_286: right because then yeah and then i don't even know if those are verticalized experts i mean who knows how they chop those up it's like one uh expert well because of wikipedia oh man it sucks that SPEAKER_30: this is closed right that seems so interesting i would love to read about that i think that could probably accelerate progress in some way and the fact that there's only a couple hundred people in SPEAKER_22: the world they probably know the real answer is probably because they also have some exposure based on who trained that data so let's say one of them is like this is wikipedia reddit quora and SPEAKER_128: twitter data and it's you know consumer it's a crowdsourced information this is the sec academia you know uh the new york times wall street journal and like a professional you know quote unquote professional these are this is a journalist answering the question from the journalist framework based on the wall street journal washington post personas and data sets so we're informed by those and if they SPEAKER_17: were to actually say that then you would be able to make the case of well hey you're literally picking your expertise based on data sets and that's probably why they circled that would be an David Friedberg: interesting reason to circle the wagons and not share secretive about what data they use i think SPEAKER_30: they don't consider that pretty proprietary um i actually think this is a problem that goes away over the long run not because of like some cultural acceptance of this fact but more about i think the amount of data did you actually need to do to train these systems just gets dramatically lower through model innovation ah fascinating i mean there's some existence proofs here like i actually think there's a lot of really good reason to go all the way back down to like fundamental art and do like an architecture search essentially wide as you can we now have two existence proofs in the world of emergent reasoning intelligence behavior one is humans right discovered through sort of genetic evolution over billions of years and large language models which were invented of our own cord running on our own silicon on algorithms you know sort of invented the fact that n equals two there suggests that like oh wow there's so many more if you run the probability like you would be shocked if like it stopped at n equals two around architectures that let in like the transform architecture is nowhere near what you see anywhere modeled in sort of the human brain completely wildly different in how they work so like it suggests that there's probably more and the the thing that's interesting about humans is how comparatively little examples they need and training data they need in order to be sort of generally intelligent we seem humans seem to be born with some innate amount of like abilities capabilities or abilities very strange um like thought like past following pattern recognition there's some like things that show really really in toddlerhood that like they never get trained on SPEAKER_128: and fear of like fear of predators like we actually understand predators in some way natively like if you were grown if you were and i think they've done this with studies like you you don't need to have seen a shark coming at you to know you're about to die i don't pretend to be a scientist but like SPEAKER_216: there's just like some pretty compelling like like i said existence proof examples where SPEAKER_30: oh okay yeah the they're likely are way more architectures out here that we should go search for and yeah like i think a really good constraint function to go do an architecture search would be to say let's pin the amount of sample data that we put into this architecture search um so we can find like architectures that are just way more cost effective or cost efficient um and performant uh i SPEAKER_22: could talk to you for hours and we've talked for an hour mike you gotta promise me you'll come back uh maybe like six months from now i think this is moving so fast i'm gonna i'm gonna make an executive decision here uh and two things since you and i are uh you know in close proximity to each other number one we got to get some uh gotta get some ramen and then number or a lobster uh sandwich and then uh number two you got to come back on the show in six months yeah thanks so much for office SPEAKER_17: recordings we can do one in person too i know i'm i'm literally looking for i'm selling our office in the city and i'm setting up our incubator and accelerator somewhere in like san mateo area and when i get that we're gonna have an in-person studio again and we'll do a live version of this where we get like audience questions and stuff like that so i'm trying to find like a theater i'm going to get SPEAKER_22: like a theater or like a warehouse space where i could have like 50 people come in a little more SPEAKER_10: raw like kind of customize yourself i like raw yeah i hate these fancy space i a lot of people have been emailing oh i got a fancy space for you over here i got a fancy space over here in el camino SPEAKER_318: come to this office like a restaurant that just shut down basically is what i'm hearing that's what SPEAKER_10: i'm looking for is that i'm looking for a like a shutdown restaurant like an old mexican joint with a parking lot or something where i can have founder fridays but we have drinks and then i have you and i just sit and rap out about stuff uh so those two things will be on the uh on the docket uh and thanks for making zapier it's just such a great product and it's made life easier warm SPEAKER_322: welcome a nice intro uh you know what makes you happier which uh i was i came up with that you guys SPEAKER_10: advertised on the pod years ago and um i'm like how do you actually pronounce this and i think i was the one who came up with zapier makes you happier and uh so anyway one time we uh it ended up in the SPEAKER_64: footer of the website too i think it still might be somewhere in the about page i might have i might have been the origin story of that i'm not sure if i was or your marketing team thank you because like SPEAKER_49: i use it all the time you know yeah everyone's pronouncing it zapier zapier zapier is what we used SPEAKER_327: to do we used to joke zapier anyway i don't care what you call it yeah exactly go to zapier.com SPEAKER_289: slash twist is i think the landing page is still up and we'll see you all next time bye bye SPEAKER_330: on behalf of the producers and the partnership team thank you for listening to episode 1769 we'd like to take one more time to thank our partners and broker use code twist to get an extra 10 off insurance at and broker.com slash twist lemon.io get 15 off your first four weeks of developer time at lemon.io slash twist and eight sleep go to eightsleep.com slash twist to check out the pod cover and get 150 off at checkout if you are looking to become a partner of this week in startups you can email hannah at hannah at launch.co that's hannah at launch.co thanks for listening