SPEAKER_00: but people should just take note i mean the fact that you know people who are building this stuff jeffrey hinton the guy who's just a godfather of ai of course but he just he drove this um that he's warning of it and everyone i mean i you know i'm just a tiny fish in the pond the people who've been working on this for decades are worried uh this is different from when people are worried about you know crypto like or when people are worried about whatever else is the thing they're currently worried about right that the people who have been working on this for ages who have dedicated their careers to it including myself are um saying that this is very much a path with a

SPEAKER_01: juncture this week in startups is brought to you by crowdbotics great ideas can change the world and crowdbotics is the fastest way to turn those ideas into code get a free scoping session for your next big app idea at crowdbotics.com twist the microsoft for startups founders hub helps all founders build a better startup at a lower cost from day one startups get up to 150 000 in azure credits access to free open ai credits free dev tools like github technical advisory access to mentors and experts and so much more there is no funding requirement and it only takes minutes to join sign up today at aka.ms slash this week in startups and release large enterprises pose unique challenges for sas startups unlock customers with unique needs for private and single tenant hosting without the toil of diy with release delivery get your first month free at release.com twist hey everybody welcome back to the

SPEAKER_05: program today we have another great guest we're doing this like ai series where we're interviewing anybody and everybody who's making interesting stuff builders in fact of the most interesting ai startups

SPEAKER_06: today uh we have eric steinberger he runs a company called magic and uh plain english

SPEAKER_07: magic is building a colleague inside of a computer basically it's powered by ai and it's going to auto generate code based on a developer's inputs welcome to the program eric hey thanks jason thanks

SPEAKER_10: for having me all right how many people work at this company uh we're 10 now 10 people where you based uh half of us in sf half of us in vienna in austria okay great and you're in sf i will i'm just moving i'm getting my visa have my my visa and one week well congratulations are you going to be in san

SPEAKER_05: francisco are you going to be in the bay area yeah i'm planning to be in sf wow would you have an

SPEAKER_13: office there yeah yeah small office you know small team but where about whereabouts in town are you

SPEAKER_00: uh not sure yet we're probably going to get a co-working space just to be quick um yeah for the first

SPEAKER_14: phase and then eventually the reason i ask is a lot of ai people are saying they're going to be in the

SPEAKER_05: city of san francisco where a lot of startups and obviously the big tech companies are no longer going to san francisco explain the decision to be in san francisco as opposed to saying being down in palo

SPEAKER_07: alto mountain view cupertino san jose san mateo wherever i don't i mean i have a few friends who are

SPEAKER_00: just really good ml engineers nsf and uh great i i don't i'm not very familiar with the area uh so i

SPEAKER_15: think sf is the place that i've been i've spent some time in berkeley but i you know i think sf is

SPEAKER_14: better for what we're doing great awesome uh it does seem like a bunch of ai developers are choosing

SPEAKER_05: to be in san francisco which is great for the city since there's like 15 salesforce towers of open space uh so let's do a demo here it's always great to demo but when we do our demos here on this week in startups we sportscast them which means we try to explain what we're seeing on the screen so let's pop up your screen and let's show people we like to show rather than tell and in this case

SPEAKER_07: we'll do a little bit of both so if you're just listening we'll describe what we're seeing on the

SPEAKER_00: screen uh so eric take it away great thanks jason um so you know after spending about a year in the cave uh building an lm training and inference framework to to build our own code models and scraping a lot of code data um we finally managed to build something that uh we think has a a qualitative edge on on other products in the space namely um you know you're familiar with code

SPEAKER_23: completion tools they give you i'll give you completions here i can like i'm gonna go put a new line here and i'll get a completion um that's standard but so what you're showing on the screen is

SPEAKER_25: you're developing in uh python maybe i can't see yeah that's right i'm doing i'm developing writing

SPEAKER_00: sorry i'm writing code um in a python project that uh here this is like a reinforcement learning code base i used to do research in a few years ago great um i mean so when you start typing in a command they're like you know so if this is like it's it looks like copilot on first glance i got like you know um uh the i get completions and um but what's what's really um different here so i should say this model we're just training we're still training it it's like sort of an intermediate checkpoint but the one thing i want to demo here is that the key difference you know copilot takes in like say the current file or maybe you know like a few adjacent pieces of information and you're referring

SPEAKER_05: to this files you're referring to github's right copilot which is the original og auto completion so if you were using gmail as a user and you see it shows you one word and then it started showing two or three words um this is something that developers use and developers have a more constrained possibility set than say people just writing email is that correct so therefore it

SPEAKER_32: can maybe give a little bit more code and it knows the existing code base right so copilot doesn't know

SPEAKER_38: the existing code base and i think that's the that's the key thing that's exactly what i wanted to to cut to to allude to you're right that coders you know a lot of code is predictable right you know sometimes it's just like super clear what the next line is sometimes it's really hard and copilot generally and general like completion auto completion tools find it easy to complete things that are

SPEAKER_00: straightforward and and somewhat obvious and find it harder to do hard things but you know kind of getting getting better at those with language models you know being scaled up um but they have this limit of context size um even gpt4 you know it can fit more but it's still limited what we've developed over the last year uh is a technology that can really look at all of your files it can see your whole repository so you can see here i've opened up a few folders just to show that there are a lot of files and i can keep going like the folder depth this is pretty you know substantial um this this code base overall is i don't know the number of like lines of characters but much much much much larger than um any of the the gpt's could fit into their context window um but our model does so for instance instead you know if you were to auto complete from the start of a file usually here if i were to go to the very first position the model has no idea what i would be about to do unless it sees the repository it would it would it's starting from a blank canvas so if i take out here for example if i take out my name and you know i want i want to like type something um okay just demos um if i start

SPEAKER_21: typing it knows that i wrote the the code base right and how does it know that um because it goes to

SPEAKER_00: other files where i have written this uh this same line um but at the same time you know you can do your you can do your normal completions so you know as you said like this is for example um kind of a more obvious line right like here i have something similar above and then it adds this this minus one piece which is intelligent because it's like player zero player one it figures out that i want to access the other player so it just sort of as you see um it just told to complete that that

SPEAKER_46: for me um okay so when you say that what you mean is it's uh you have a string of code above it it

SPEAKER_05: recognizes that code your learning model knows and your auto completion model knows the code you wrote above it it assumes now that you're writing about another similar thing and it can then put in gray hey

SPEAKER_12: here's what i think you're going to do next it reads your mind and then instead of typing you just hit like the tab key or something to fill that in exactly that works great so just like in gmail but this is as if it had read every one of your gmails and it knew that you like to often let's say if you were a vc you might talk about your portfolio companies you might talk about other partners in your firm it would have that information so when you says hey you should talk to my partner susan

SPEAKER_50: it fills in a little bit about susan exactly it's it's able to take in the context of what the developer has done and produce code that is similar as you said i think you said this very if you put it very well it reads your mind and and tries to do what you are about to do you know we can do a few more like this right i can write this line of code and because the function is called do pickle which in python just means store a file in a certain format to disk it just wrote that for me which is of

SPEAKER_00: course relatively straightforward but it's very useful to do that faster um but now the thing that again our ai does different from existing ones we just wrote this function this do pickle function if i go very like could completely different file um where here where i use this function and i can even close this right we're not we're not using um the fact that uh you just edited this it's it just sees the

SPEAKER_50: the entire repo so if i this should hopefully work this gray text the intellij suggestion knows it because it's semantically analyzing the code which if it goes down it free of import statements and so on very specific doesn't always work only works when it's kind of syntactically clear but here our model

SPEAKER_00: successfully understands that we have a method called do pickle somewhere in the repository and that's probably what we want to use if you tried this with copilot for instance github copilot or other

SPEAKER_50: products um you'd find that um because it is limited to the current file and slightly small bits of adjacent

SPEAKER_00: information that are only grabbed heuristically as in algorithmically designed and not learned by the model it would find this difficult and so we what we find in practice is that as you code when you write code you know you often have a lot of code already in your repository and our um ai is able to reuse your code to use your coding styles to use functions from uh from across your repository in various contexts across uh the the project um and so that's just what that's what we've been working on over the past um past months or past year at this point and uh working on a few other really interesting things but uh very excited to get like tangible um models now and and uh launch very soon we're giving you a sneak peek here the the model will finish training um relatively soon and then uh planning to launch that uh with with the the same that i just showed you when do you think you're going to launch it

SPEAKER_54: yeah this this month great probably the most common challenge i hear from founders is related to building either they aren't technical and are searching for a technical co-founder or they can code but they're just spread to then this is one of the first major obstacles you're going to face and i know how discouraging it can be but there is a solution do you have a great idea but you don't have a technical co-founder well crowdbotics can be your cto as a service boom just like that this means you can focus on building an awesome product and delighting your customers rather than wasting your time on infrastructure planning architecture compliance and all that boring stuff crowdbotics also offers professional scoping to help you flesh out your project at the mvp stage and beyond so cut off the hassle and get back to building that perfect product for your delighted customers when you think crowdbotics i want you to think getting your time back to focus on product product drives everything in a startup so let the folks at crowdbotics show you how it works schedule a free scoping session and get your detailed build plan at crowdbotics.com twist that's crowd b-o-t-i-c-s dot com slash twist

SPEAKER_14: and so when you have a company like this uh you have to

SPEAKER_05: do you build your own language model or do you use say chat gpt's existing language model which has been trained on a lot of open source products and then do a local version of the repo of this company's code and then merge those things two together or are you building also your own so

SPEAKER_50: you're not dependent on chat gpt4 or open ai for that matter yeah uh we built our whole uh language model training and inference stack from scratch um the main reason for that is that we wanted to be able to innovate on the architecture of the model we um do a few things different from the sort of the standard way uh which is also why it's taken us so long to launch something um the way we really look at code gen and the coaching space more broadly is that i think ultimately there will be a user experience that um frames the idea of fully automated development in a way that anyone and everyone

SPEAKER_00: finds um attractive and and useful and reliable enough there'll be some way to talk to your computer to to automatically write code and and that ux will replace whatever comes before of course including

SPEAKER_50: auto completion because if you don't write code in an editor why would you need auto completion so we're very focused on making sure that what we do in the short term and in the midterm and in the long term is um geared mostly towards the final stage of what this space will look like and that's optimized for the immediate we could have built something like what i just showed you within a month of just existing as a company um but if we just took other people's models and you know

SPEAKER_00: other people's inference servers and so on and those things would have worked they would have been interesting products um and then would have probably brought in some revenue um but it would have

SPEAKER_05: slowed us down on that um so you're starting with a tool for developers so that they can write code faster hopefully they can write code 50 percent twice as fast i don't know what you've what what is the

SPEAKER_23: benchmark you think you'll be able to achieve this year how you measure it right like you there are there are some pipes of engineering that are um inherently dependent on uh contextual understanding

SPEAKER_50: outside of your code base right you just need to talk to people um i think those those numbers are

SPEAKER_00: often not like those things are often not considered an estimate um then there is some coding that is really just boring template writing or we hope that it just does it all for you um it really depends on what type of work you do but you know like speed ups and and this uh the average developer range

SPEAKER_05: just take the average group of developers you got a hundred developers in a development team at some mid-sized startup what would you expect across the range of developers the average would be each year in terms of gains from this type of technology yours and other people's yeah i think if you measured the

SPEAKER_00: time it takes a group of people to build a piece of software um as is with and without the technology which i think would be like the best benchmark to really quantify productivity uh sure i think it is possible to get um beyond the 2x mark even with you know a few more steps of innovation i think right now

SPEAKER_50: this number is below the 2x mark with existing tools like um you know others out there and uh because you think the current tools are making people 20 30 percent faster or something i don't want to invent

SPEAKER_00: numbers now but the numbers published by various companies or use different metrics and i i think ultimately if you measure it like end to end how long does it take you to build a production scale project um time there's just so much work in there that's not writing code and i think that's also something we're really optimizing for it like a lot of code gen companies are like okay let's automate writing unit tests but how much time do you really spend writing unit tests a bit but you know not

SPEAKER_05: that's not your when you say you think people will get twice as fast do you think that's gonna that would be like every year they get twice as fast or ultimately they'll be twice as fast how do you it's a time frame in which you think if we were to come up with our own morse law and i forced you to

SPEAKER_06: bet the time it takes with ai for the average developer to become twice as efficient

SPEAKER_75: all in they can basically finish a project in half the amount of time how many months will the doubling

SPEAKER_00: take on average going forward i think though the first ai doubling should be done this year if it didn't already is if it didn't already happen for some developers um then that that should really be something that happens this year how many months do you think it will take on average yeah i think it

SPEAKER_50: will be much faster than doubling every constant number of months or years at some point there is there's this inflection point where ai is reliable enough to just write code make a pull request

SPEAKER_00: you trust that it's right you know the same way we have great engineers if one writes a pull request another one reviews it i don't even look at the code right it just goes in um and there'll be a moment when ai writes a pull request and reviews its own pull request and it's as reliable or more reliable than your engineers wow and the difference between that isn't now right but that will come

SPEAKER_38: and the difference between service things like self-driving cars and that is that a company can take a look at this and like well this is clearly better no lives at risk right let's go do it got it so you

SPEAKER_36: think that's it would be faster than it would be faster than moore's law and i think it looks like

SPEAKER_05: this got it and so what you're drawing is a slow uh sloping up curve and then a huge giant exponential spike uh yeah no this is i think uh i would agree with uh your your central tenant and the key issue is can you trust the code and right now absolutely you can trust the copilot because every line uh every suggestion it's making a human decides yes or no but correct me if i'm wrong if enough humans are saying yes or no that's reinforcement learning the model uh will look at a correction that's made if it gives a bad suggestion it says no i i don't that's not what i'm looking for in terms of the auto complete this is what i'm looking for does it take that into account the next time it's going to do an auto

SPEAKER_23: complete so i think reinforcement learning plays a key role in all of this but that the reliability piece

SPEAKER_50: is bigger than that um because so this is actually something i love talking about this thanks for bringing it up um no matter how large you make a model if it's not aware of its own actions of in the

SPEAKER_00: past in a given interaction um it is just physically unable to remain grounded um in in its history like so to be to be more um concrete i suppose imagine you forgot what you did before we started recording this podcast and i asked you how your day was all you could do is invent something you wouldn't know better right and so a lot of the issues we see with hallucination and and truthfulness and um

SPEAKER_50: um reliability actually you could think of as a branch of those things um comes most likely i think from the lack of grounding in the models past um it is sort of the average like the surface area of the training set were just like a cloud of points um but the the surface area doesn't perfectly represent the points and i i think if models models were to perfectly represent the points they would solve a lot

SPEAKER_00: of these reliability and and hallucination issues and uh so that there's there's there are parts to it that aren't rl but rl plays a key role and i'm certainly i certainly believe strongly that you need it to to get there but um it's it's so far an open question how you make models as reliable as

SPEAKER_50: humans i think the key there is to get them to know that to get them to say that they're unsure

SPEAKER_88: when they are unsure instead of just trying and then like acting like they're confident

SPEAKER_90: all right everybody our friends from microsoft are here tom davis a senior director at microsoft for startups and you're a former founder you are here today to talk to us about the giant leaps that microsoft has made in the ai space you've been giving azure credits to startups and that's delightful and amazing but people really want access to the open ai api yeah absolutely so there's two things

SPEAKER_93: first of all we've got a benefit that we offer our startups they can get two and a half thousand dollars worth of open ai credits so they can get access to the latest and greatest models that open ai are delivering but then they get access through the hunt up to 150 000 worth of credits that we offer through founders hub to leverage the azure open ai service which has a full sla around it so when they want to go into production and really have that reliability that we provide with with the azure sla they can leverage the azure open ai service apis and they can do things like the gpt models with codecs for the coding and also for the dali models as well for images so it's a full service it's not just the great apis that you get and access to the llms they can build out their own llms using open source and then they can manage those with our ai tooling services as well amazing well done and if anybody

SPEAKER_90: wants to sign up for that do it now while you are in front of your computer aka.ms slash this week in startups aka.ms slash this week in startups well done microsoft and well done tom so there's two paths

SPEAKER_05: that are occurring here uh which you alluded to one is you're making developers more efficient more reliable at their job but then there's this other path which is hey can i just talk to the ai as a non-developer maybe i'm a product manager or the ceo founder of a company but i'm a non-developer founder or i'm just making something small and i say hey listen i want to make an app that does a high interval training and i want it to be able to track my gps log in with my google login

SPEAKER_14: and then put the results into a google sheet so that i can see them there and i can see my sprints and my recovery time and please put my heart rate in there it could write that code and we would trust it enough and and have it published to the app store or whatever uh to test flight and you wouldn't need

SPEAKER_05: a human involved those are the two paths that humanity is on right now correct yes you're choosing to build this tool the auto completion with the civilian the non-developer in mind is that correct is that my

SPEAKER_23: understanding of your startup magic i think as a company we're not just one product um we're building

SPEAKER_00: towards agi and safe agi in particular the code completion product we built is a very good way for us to have early grounding in a user experience and customer feedback and you know potential early revenue and so on but we're thinking like a lot further than just um just code completion i do believe that in the near term um this this product will be uh the the primary driver of our interaction with the outside world um and i'm i mean you know we we as we're nearing the launch um you know we're talking to uh folks who you know like to use it internally and i should say one of the key differentiators here is that we uh want to offer a a much uh more privacy focused version of uh of of uh this experience in the sense that um your companies that that don't want to run on a public api that don't want to send their code to microsoft open ai or others um we we offer private cloud solutions where enterprises are able to um host their own version of our um system and so we we think a lot on uh on this in the early days now we think a lot um on the axis of product differentiation and just early deployment of of our um ai models but ultimately i i would say we are mostly focused on

SPEAKER_50: building systems that are much more reliable that you trust with much larger tasks where you'd instead

SPEAKER_00: of going and asking for like two lines um you ask for uh you know a day worth of work and uh give it you know three minutes time and then it makes that how close are we to that piece of code i you know we'll

SPEAKER_107: see um we're we're uh we're working hard um and uh just generally the industry because right now it seems

SPEAKER_06: like doing snippets pretty easy sometimes it's gonna pull in you know uh some elements that um you know might be reused over and over again libraries let's say uh but then putting together multiple pieces to to build the app as i described it you know that this magic moment where some startup founder or just some civilian on the weekend wants to build an app and they can just describe what they want and have something come back to them because we see people doing that with images right now we see people

SPEAKER_05: doing that with text hey i want to write a blog post uh and i want it to be about how to utilize social media to build the company and it gives you a lot of great ideas and you have to polish it off um people seem to think that those blog posts will be ready in the next year or two to be published without needing polish right so i guess that becomes the issue here is the last 10 of self-driving cars the last 10 of robotic kitchens it just gets really hard so if you had to think of a year when people could do the task

SPEAKER_50: i mentioned what year would it be i think it's it's certainly possible that it's further away and that um you know we'll bump into the same 90 10 type problem as you described you know the self-driving car world and many others um i agreed at this that there aren't that many applications of technology automation where you don't run into uh the final reliability step but um ultimately i mean what people are building here is a system that is converging to intelligence greater than those of humans and ultimately i i think it is whether that's good or bad um it's a different question but i do think it is feasible that that happens in the near term um near term being under five years yeah yeah i'd be

SPEAKER_47: comfortable saying that um okay because i mean i think that's one of the fascinating things about

SPEAKER_05: self-driving let's say we've been two years three years away for about 10 years uh you know in my estimation and listen i use full self-driving it is amazing but you must keep your eyes on the road hands on the wheel because there'll be a left turn or there'll be a bicycle that comes out of nowhere there'll be it's not ready for to take the steering wheel out um although cruise i guess has some on the road in a very short you know uh constrained environment uh in san francisco now so that seems to be in your world a developer doesn't have to review the code and we seem to think maybe in the next under five years which we call the midterm i guess in our industry short term would be low number of years midterm be five and long term be 10 to 20 decades we're kind of in that midterm uh area i think

SPEAKER_14: based on what i'm hearing from you of i talked to it and it gives me the code back that being said

SPEAKER_05: the pace of code being written is just even doubling or you know if it doubles we have like some kind of if we just take more's law every 18 months a developer's productivity doubles that means we're doubling the number of programmers on the planet it's that's pretty amazing does this mean the developer shortage will end does it mean developers will be less expensive and salaries will

SPEAKER_14: go down what are your predictions you know as we move out of this towards what you think is going to

SPEAKER_00: happen on a societal basis in our industry for that matter yeah um i found it most useful to think of it as two points rather than a line and then draw the line after i think right now we live in a world

SPEAKER_50: where um humanity is constrained by human ingenuity and productivity where you need a smart person to come up with something and then you need a lot of smart people to come up with the details and build them and make them um for an extended period of time in hindsight when you look at these things like you could always imagine like okay you could build this so much faster if you had already known all the things that you figured out along the way and if you your body moved a thousand times faster so it is not inconceivable that you could build the things that today cost you know hundreds of millions of dollars to build um for almost nothing if you take out those components um and in a world where you have very affordable agi first of all you have to make sure that humanity doesn't go extinct um because then nothing happens and that's unfortunately a real risk um but if you get to the path where humanity is fine i think it's almost guaranteed that we will end up in a in a world with virtually infinite ingenuity and productivity across the board and it's very hard to imagine how fast such a world could move even with things things like more slow in the back of my head

SPEAKER_00: you know you might imagine worse law every two hours um it's it's just those things if you if you think from very kind of foundational principles it is not inconceivable that if you had all the ideas that it took to make you know apple's micro process like apple's m one chip and you didn't know exactly what to do and you could just draw the whole thing super fast and you could run like a million copies of yourself that like can communicate at the speed of computers not the speed of you know human brains there are many orders of magnitude in between um at the throughput and the volume the

SPEAKER_50: upper cap of this is unimaginably um crazy so i i don't and so now like going to the line

SPEAKER_00: um i think it starts with ai assistance it starts with augmentation it starts with things like you're describing a sort of a moore's law of in you know right now i would say the multipliers even though

SPEAKER_50: things are flashy it's quite small uh i i do think chat should be chat should be did a great job at uh

SPEAKER_00: getting a lot of people's like helping people when they're stuck which usually takes a lot of time right because it can answer questions about things that you might be stuck with um ultimately it will go from that to okay like you just do it for me why am i even doing this and then once you get to the stage where you let it plan what to do and you don't just so if it's you know you no longer just delegate micro tasks but you let it decide what they'll then i i just this this goes beyond any worse though um that we could draw up here um so so developer productivity you know like i think you can draw these graphs for the next short while um but the world is just going to be an inherent like a completely different place we're gonna have much much more stuff hopefully much more sustainably um hopefully distribute it in a way that you know there will be still be some people who have more than others but everyone will have much more than the average right now and uh i i think that the utopia version of this is is amazing and um just almost gonna feel like a different civilization um to what we're used to i i don't think there's a i i i have so far always failed to draw a line

SPEAKER_17: i can draw like this short line and then like a big job yeah i mean our brains

SPEAKER_05: uh have a hard time it took us thousands of years as a species to accept the concept of evolution just

SPEAKER_14: that that human beings emerged slowly over time from primates and you know uh mammals before that and you know etc etc now we're on a timeline that's moving so fast the concept of moore's law moving

SPEAKER_05: from 18 months to 18 weeks to 18 days to 18 hours to 18 minutes to 18 seconds it is conceivable like you said whoever the team was that designed the m1 chip then the m2 chip all of that knowledge going

SPEAKER_14: into a learning model and some artificial general intelligence it could make instead of a new m2 chip every 18 months m3 coming you know 18 months from now it could make it 18 days or 18 weeks from now and

SPEAKER_05: then we could have an m3 and m4 coming every day and then we don't even know what a world with that much processing power would be like or that much software development would be like we can't comprehend what it will learn about things in the real world medicine fusion uh energy uh transportation logistics

SPEAKER_14: you know it's it's it's unimaginable how quickly this could solve certain problems longevity diseases

SPEAKER_05: cancers dna we we don't know what this speed level will do to this this prop to these problem sets

SPEAKER_00: correct that's your interpretation of this that's that's right i mean i try to think of what the limit is and uh you know you can keep going but i'll take you like take it one step farther uh humans are

SPEAKER_50: currently made by um humans through a nine month pregnancy process so there must be there's clearly a way to piece together a few molecules out of pretty much nowhere um through a biological process to create something as complex and amazing as a human being um if you look at human like at factories

SPEAKER_00: that humanity has built for things like computers which are probably the most complex piece of engineering we've ever made or for simple things like i know i have a glass in my hand now right this is probably easy to make um compared to a computer they're they're just like nothing compared to what the human body can make and that's something that was created through evolution not through design now imagine what you can create through design if you had factories that didn't operate at the level of um computer complexity but at the level of human body complexity or beyond but now instead of it being designed through an evolutionary process that has all of the flaws you could imagine it's just piece together of raw out of raw atoms and molecules whatever it is that you want to make the universe is made of atoms not of things and we just see things because that's how we evolved but the the max level of craziness that we could get to with this technology is is i yeah i don't know what you can build once you have machines that make things of the company it's scary it's really scary um but it's also it seems like you know this is the probably the most important thing for a civilization um in in the existence of that civilization to to build to build a thing that surpasses it like evolution does that

SPEAKER_50: evolution creates things that are more fit and by the evolution by evolution's fitness function than the previous thing that were that was there just through survival of the fittest but never ever ever

SPEAKER_00: in history has one of those things made a thing that is more fit like never um but we're about to do that and that loop is probably never going to stop as you've alluded to right like you if you make an m1 give more compute you make an m2 like that that thing is real like if you have an ai that's a better ml engineer than i am like why would it be better at writing the next version right that that's real and um you know again that can go horribly wrong uh if it goes right it's just uh it's a different civil it's it's an evolutionary time scale it's an evolutionary scale event this is it's not just the technology it's not like mobile or the internet it's those things are really

SPEAKER_81: important at the human time scales but this is this is like there's a you know it's really interesting

SPEAKER_05: how you have as somebody on the inside understands this and and this is i think the problem uh we're seeing in the world uh you know when the ancient romans started writing books by hand and then giving them to the nobility right those books took months and months to be handwritten by monks or scribes whoever

SPEAKER_14: wrote them now if you were to take that book and transport the monk you know from 400 a.d and put them into a modern printing press today take their book press a button and show them 10 000 copies of

David Friedberg: their book the next day they would not know they would think you were a god yeah they would think you were an alien they would literally think you were zeus and that you took you know uh this book that had been written and took years or months and made 10 000 copies of it they would not understand uh it would be like jesus taking a fish and turning it into two but instead of two it turned into two thousand right

SPEAKER_14: like it would be like taking a farmer who was you know making uh you know pulling potatoes from a field

SPEAKER_05: and some irish farmer you know pulling potatoes from a field and then showing them a million potatoes going through a factory onto trucks they would say how many humans do you have inside that box to do that

David Friedberg: because it wouldn't it would be hard to comprehend and this could be even more

SPEAKER_07: more than that analogy and humans we use analogies to try to understand stuff but that's that's the

SPEAKER_06: analogy i i've come to be just hard for somebody to understand modern day book printing or modern day newspaper printer or modern day food processing if you were but a farmer uh or somebody you know

SPEAKER_135: handwriting manuscripts developer talent is the most precious resource for b2b startups you know that and you want your developers focused on product not on compliance right you want product you want to ship features for your users and when you're selling b2b software to large enterprises well you're going to need to jump through a ton of security and compliance hoops you know that one of those hoops especially for large companies is the customers want to host your software on their cloud and you need to build that out on a per customer basis it's a lot of work right b2b startups constantly face this dilemma do you keep your develops focused on the infrastructure which is going to hurt your product velocity or do you keep them focused on product velocity which is going to delay your ability to close large customers well there is now a solution for you it's called release delivery they automate the creation of enterprise class app delivery for private clouds and single tenant applications this lets you deliver your software seamlessly to any customer environment and this is going to unlock a ton of revenue potential for you basically all the big fish require this you know that release delivery is going to put all the tedious stuff on autopilot for you you can turn your ideas into apps and deploy those apps quickly and flexibly let release show you the power of release delivery and get your first month free at release.com twist that's right they got that

SPEAKER_138: domain name release.com what a great domain name that's up to ten thousand dollars in value at

SPEAKER_00: release.com twist i like the analogy i think maybe one way to look at it is that you move the person like you move somewhere from the stone age to the point in time where they could write books and it's like okay what is writing even and then like 14 seconds later you move them to the time where you could print 10 000 copies and 14 seconds later you move them to the time where there is the internet right but you just keep doing that forever and you can't really ever grasp what's going on that it just keeps getting faster and crazier we are already at the point you know things changing used to take many generations um thousands and thousands of years ago the innovation was so rare um on the time axis that it was normal for a a generate like that many thousands of years ago right it was normal for a generation to just not have to change the paradigm um and even until recently you know the invention of things like the industrial revolution that took forever um if you look at the internet

SPEAKER_144: that did not take forever no i mean we take it that did not take forever take a gen xer right yeah

SPEAKER_05: and compare what they experienced to what their great-great-grandparents did in the 1800s right you know they might have experienced the printing press uh

SPEAKER_14: nuclear power maybe if they made it to the 50s the machine gun uh trains maybe those were the

SPEAKER_05: the biggest things they ever saw oh and the airplane right yeah pretty amazing stack for a hundred year old to live from 1850 to 1950 to have seen they got to witness an airplane a car nuclear bombs

SPEAKER_14: and the machine gun now what is gen x experience the internet dna sequencing uh gps

SPEAKER_06: you know i mean it's now ai right like it's a whole different cohort of and it's not over

SPEAKER_07: well and it seems like it's going faster like what i just described would be like one item right

SPEAKER_06: yeah um yeah exactly oh and self-driving potentially and life extension you know like these things it's just unimaginable what um this could do in the midterm to long term when you saw i

SPEAKER_05: don't know if you saw yesterday ibm's like you know what we're pausing hiring about 30 percent of uh

SPEAKER_06: new jobs thousands of jobs because we actually think they're going to be these back office roles

SPEAKER_12: are just going to be all ai so what's the point of even hiring them now we'll just pause hiring them knowing that ai is coming so quickly now this is ibm that created watson big blue deep blue you know b

SPEAKER_05: kasparov like they know what they're talking about uh you know you can make fun of ibm if you want to but they they actually have been a pretty storied institution uh yeah and for them to just say what's the point of hiring operations people it's a pretty big statement you know i don't know if

SPEAKER_157: they're blaming the ai maybe they're blaming the ai i i don't know about the specific story but i

SPEAKER_38: would i would say that generally you know like people say a lot of things pr whatnot but in

SPEAKER_00: those actions you really see what people believe uh you know a public company allocating funds one way or another uh is usually associated with their beliefs of where the world is going and um and yesterday you know there was an earnings report of a company that lost revenue because of josh pt and um

SPEAKER_47: that that these these things are just going to increase um is the company you're talking about and

SPEAKER_05: as chag is a a student resource for getting homework help to solve problems writing citations you can rent textbooks all of that those study guides exams they said people are just using chat gpt now so

SPEAKER_62: students are going to find the most affordable free in fact solution to their problems that's why they will photocopy a book you know or a chapter of a book or share a book or buy a used book so they're just going to go down like water going down a hill it's going to find the most optimal route uh yeah chag was

SPEAKER_166: the company you're talking about yeah it's crazy for a second yeah um thanks um yeah you seem to have

SPEAKER_00: a positive view of all this well look um i studied game theory for many years this is the field i did research in since i was 15. that's just the thing i kind of i don't know i like decisions humanity is just just a large collection of decisions and the incentives and rules around those decisions um and and sort of a lot of you know kind of group behavior is predictable by those things and a lot of limitations on group behavior are predictable by those things so i'm i'm a fan of that um am i optimistic well i let me be clear i think there is a good path um and that path is mind-blowingly amazing it's an evolutionary time scale thing that we are just immeasurably lucky to be experiencing if it happens we we are like maybe the generation people are going to envy forever um you know the last one to experience non-ai life the first one to experience a life and the whole transition i mean this is probably really the most exciting part of the history to ever live in and maybe forever who knows um just in terms of how exciting it is but at the same time you know humans and ants um you leave them alone when you're hiking in the forest and you're kind of like cute but you don't really care and when they're in your kitchen they're just dead uh like a half a day later because you have some ant poison and then they don't understand what you're doing they don't understand how and poison works so they just they go there the way ant poison works so you have to put a little thing somewhere and then they go there they carry that and poison back and all the ants die they didn't even know what

SPEAKER_38: happened it just right then that's what happens if a dumb species annoys a smart species and if we build ai that is so as much smarter um you know than we are as we are smarter compared to ants and it doesn't like us or someone tells it to kill us all and there's no way to defend against that which is

SPEAKER_00: both a game theory problem and a simple x of a is it even possible except similar to cyber defense where it's just not clear like how hard attacking versus defenses um in some vectors that is real and um

SPEAKER_124: if it's extremely high variance and that i believe there are ways to do this that lead to the good future and there are ways to do this that lead to everyone dying um i i do i think both are possible um i have a lot of thoughts on the split in the three at that

SPEAKER_00: junk track uh there but um i don't know if i'd call it optimistic i'd say this is the most important time and to ever live and i don't know why but glad i have some ability to contribute to hopefully

SPEAKER_47: making it go well um well let's hope it's not a zero sum game you know if if this is a you know like

SPEAKER_14: a giant 10 000 piece puzzle hopefully the ai and humans are solving this puzzle together and that whoever puts the last piece in uh we both benefit from the puzzle being completed the puzzle being

SPEAKER_06: humanity understanding the nature of consciousness the universe the cosmos why we're here what all this is and then having some joy from the experience and you know i i don't actually define it as living forever but hitting 52 uh i'd certainly like to double my lifespan uh and maybe preserve consciousness and understand what the hell the universe is it does feel like this is going to tell us but i love your

SPEAKER_05: analogy of hey let's be thoughtful because this thing could grow so you know exponentially a 16 and 18 second moore's law is like lighting a fuse to something that we don't know the chemical nature of and

SPEAKER_14: it's literally like you could be lighting a fuse to a giant bucket of kerosene underneath you and not know it and by the time you figure out what happened you're incinerated right like that is kind of the analogy here and i think we have to be thoughtful and the fact that you're recognizing that i think are

SPEAKER_05: you for slowing this down is there any guardrail here should people who work on this technology be licensed like doctors are or truck drivers are should people be required if they're using these tools to have you know some uh ownership of what they're building you have any thoughts on when regulation should

SPEAKER_00: come to this space if at all yeah i think regulation is is needed now um i think that would be a good thing um the making it i i do think you need to do this work because it's it's ultimately i i do believe that if something is possible it will happen um if you can't live in a world where all of humanity knows that you know how to build a screwdriver and nobody builds a screwdriver it's just not gonna work right someone's gonna hide in a basement and build it and you know with compute getting cheaper training a gbt4 quality model will get cheaper and you know it's it will eventually happen so we need to make sure that it gets developed safely so both that it a gets developed and be safely in the same set you need to develop it to develop it safely uh otherwise i just can't i can't see a world where we stop it or pause it um and again there is a utopia version that i would certainly love to be in um but yes i do believe you should make it extremely um hard to mess up like

SPEAKER_05: you know life is fascinating yeah we should just not mess it up what what what are the choke points here i

SPEAKER_14: mean one i can think of is access to compute power required um people and then maybe the people who do

SPEAKER_06: control that compute power i know it sounds silly aws nvidia azure they control the means of deployment that seems to be a place where you could be thoughtful just like people who control the precursors to certain chemicals that are dangerous have some controls uh and for people who are in the no control camp go try to buy a bunch of fertilizer after the oklahoma city bombing if you want to go buy a bunch of fertilizer it's tagged it has secret codes inside the fertilizer they report back to the

SPEAKER_183: government when somebody comes in and buys a lot of fertilizer who's not a farmer there are some controls in place because people realize fertilizer bombs can knock a building down it's it's not controversial

SPEAKER_14: to think about regulating you know a certain size of uh gun a certain caliber of gun you can't buy a 50 caliber i don't believe uh you can't buy 50 caliber certainly machine guns automatic machine guns like that's limited to the military but you can buy you know 38s and 45s so there is a certain caliber under which uh you know some regulations occur how do you think about specific regulations i put one

SPEAKER_62: out there which is the deployment uh yeah and access to deployment technology if you

SPEAKER_190: that would be careful um the so first i think training is something that is a huge

SPEAKER_00: job point because it right now requires a whole lot of compute that you can't hide um to really to train these models you need very specific hardware that only a few companies are building so um if you have those companies uh follows very strict procedures and um who gets access and you know what monitoring should look like i think disclosure is important to just get an idea of what's even going on the training is training is one that's easier to regulate this starts to change once you know what happened with this model called llama where the weights just you know where like kind of half open source then they leaked and everyone has them once that happened i think you're in big trouble because inference is not that difficult um even if you like so you know if i downloaded gpt4 on my computer like i could figure out how to serve it without anyone noticing regardless of government constraints it might run extremely slowly but frankly who cares so oh my god my completion takes 15 minutes instead of three seconds like what's 15 minutes nothing right now i can run that i could run that on my cpu with hard drive of offloading which is fancy terms but the point is like you can run this stuff on any hardware it's just going to be extremely slow so that i don't think i don't think you can particularly stop um individual deployment you can you can stop rollout you can stop productization um which i think a lot of the lm companies are doing themselves which is good um from like making sure that you know malicious use cases are not being pursued um they they do that well um but training is the big one and i think i think for that that's why i think regulation should come soon um for that to work it just needs to be done before there are a lot of models like that um and and you need to make sure that as you said i think licensing of uh companies and individuals that are in control of that and then the security procedures to ensure that the weights do not in fact uh get distributed to the wide world um are important you know you bump into many international um issues there like what do you do if other countries don't want to comply with that like do you just slow down as the us i mean i'd understand if the us doesn't want to slow down if other countries don't so this needs to be a multinational thing just like with nukes and there's a long story about how it went with nukes i don't know how much time we have but basically the the whole like we don't use nukes thing fell through because they couldn't detect um i think it was underground tests um back then because they didn't have a you know measurement capabilities that were good enough uh so if you can't detect it you know people don't trust each other governments don't trust each other and then you get everyone doing it anyway um but you know training i think you know i just bioweapons too i mean yeah yeah we're supposed to not be

SPEAKER_05: doing bioweapons research right like i think there's some international treaties on this and it's like is saddam hussein or yeah you know kim jong-un or you know some bad actor are they going to follow it no and and can you detect it no uh can you maybe get spies to tell you what's going on perhaps

SPEAKER_14: can you find some precursors and follow them around or do some lab detection i guess but yeah

SPEAKER_38: brave new world but people should just take no i mean the fact that you know people who are building

SPEAKER_00: this stuff jeffrey hinton the guy who's just a godfather of ai of course but he just he drove this um that he's warning of it and everyone i mean i you know i'm just a tiny fish in the pond uh the the the people who've been working on this for decades are worried uh this is different from when people are worried about you know crypto like or where people are worried about whatever else is the thing they're currently worried about right that the people who have been working on this for ages who have dedicated their careers to it including myself are um saying that this is very much a path with a juncture and and and we are we were supposed to be careful here um yeah and you know we have government authorities checking on development of other potentially dangerous

SPEAKER_201: technologies and tools and we should just have that here um yeah if you don't know about uh

SPEAKER_62: hinton he um he has uh left google with a warning of danger ahead and this is uh a reoccurring theme uh people who uh helped build this saying um this is moving at a pace that's making me uncomfortable

SPEAKER_05: thank you so much for sharing what you're doing i wish you great success and uh we'll have you back

SPEAKER_14: on the program uh after you launch and you get a you know i don't know a couple more moore's law cycles uh learnings so i'll probably have you back on the show in 18 minutes uh to to explain how

SPEAKER_156: this all turned out for humanity uh great having you on the show are you i know you raised a bucket load of capital are you hiring and uh yeah who you're looking for to join the team yeah this is a great

David Friedberg: conversation i was i'll be totally honest i was like wow uh okay this is really simple uh i get it uh totally logical idea for a company it's going to be very successful do a great return for investors and then we started talking about the big picture and you turned out to be a wonderfully thoughtful guest and we had some really great moments here of like thoughtfulness just batting back and

SPEAKER_06: forth um you know some analogies of where this is going and you really expanded my thinking on it and i'm sure the audience will get some great usage out of your thoughts and your thoughtfulness uh where can people find out more about your company sign up for the beta and then uh maybe join your team

SPEAKER_124: because i just really appreciate what you just said i did thank you i'm glad i can contribute some

SPEAKER_00: amount to hopefully you know the wider picture of vgi going well beyond me just pressing buttons on my computer um which i certainly like doing and do a lot of um but you know okay so yes we can find us at magic.dev um dev like you know developer and um joining us yes we're looking for people who care a lot about ai going super well and not going super bad and um who are good at pressing buttons that make computers think uh you know the deal if you if you are one of

SPEAKER_124: those people um so just reach out at eric at magic.dev and uh we would love to hear from you um magic dot dev slash join as well to see what they're looking for and waitlist waitlist and uh yeah no i'm i'm an open inbox guy uh but yeah slash waitlist if you want to use the product um maybe i don't know when

SPEAKER_17: this is maybe when this comes out potentially is already used but probably waitlist we'll see

David Friedberg: all right everybody uh that's eric magic.dev and we'll see you all next time