SPEAKER_00: we also have conversations with people from printing press companies and they're using like really old printing presses and they are touching sensors or printing presses because they want to kind of like you know they're getting old and they need to you know adjust them to for the for their availability and the same problem yeah i have all the sensors around this printing press

SPEAKER_01: can you tell me when things are about to go wrong like predictive maintenance yeah recalibration because time is money for these guys you know every minute machine doesn't work

SPEAKER_07: that's that's money this week in startups is brought to you by dot tech domains don't miss our jam session with jcal contest coming soon to apply and get more details go to jam with jcal dot tech brought to you by dot tech domains open phone create business phone numbers for you and your team that work through an app on your smartphone or desktop twist listeners can get an extra 20 off any plan for your first six months at openphone.com twist and vanta compliance and security shouldn't be a deal breaker for startups to win new business vanta makes it easy for companies to get a sock to report fast twist listeners can get a thousand dollars off for a limited time

SPEAKER_14: at vanta.com twist all right everybody welcome back to this week in startups obviously we're moving into an era of startups and employment and work and life that is going to be driven by absolutely mind-blowing experiences powered by artificial intelligence videos by sora mid journey we've seen all that we're starting to see robots like optimus and figure music generation all of this stuff is incredible and it generally uses text-based prompts but what if ai could understand the real world in real time well archetype ai is here to bridge the physical world with ai and they call it just that physical ai they've created newton it's a first of its kind ai model that understands the physical world okay the innovation allows integration of sensors with machine learning so you can have sensors pull this stuff in here and today we're lucky enough to have the ceo and founder ivan kuparev to explain what they're building and to show it to us if you're not watching us you can go to youtube.com it's a new website that hosts videos uh you're gonna love it by the way there's a lot of

SPEAKER_17: videos up there like thousands of them and uh go to youtube.com and search for this week in startups

SPEAKER_19: you'll find the episode ivan welcome to this week in startups how are you doing good great to see great to be on your show yeah uh i i've seen some demos of what you're building right and it's really

SPEAKER_23: interesting so why don't we get started and we'll just show the audience what you've built how it works how do you like to start do it like you want me to show the demo do you just talk to the demo because

SPEAKER_19: i think it's like one of these things where once you see it you start to understand and you do such a

SPEAKER_17: good job of demoing it and explaining what's happening behind the scenes and the other demos i got to see

SPEAKER_29: online right okay all right so ivan show us how you take motion and and you find some meaning in here

SPEAKER_30: the idea of market api is to build a foundation model which can understand physical world when

SPEAKER_32: you think about the physical world you can think about sensing and sensor data right because you know

SPEAKER_00: human naturally observe physical world throughout and biological sensors but when you go to you know machines and talking to the physical world environment and industries which they run on kind of all kind of sensor data motion data radars you know spectrograms and so forth so let me show the video which is very much inspired by our conversations uh with logistics companies like one of our investors is amazon and how they can track packages through the long logistics supply chain and know what's happening when the package moves because you kind of don't know like you send it somewhere you have no

SPEAKER_01: idea what's happening with the package so how can we get uh sensors to tell you what's happening to

SPEAKER_00: the package so this is an example of of the demo we built so in this case i'm going to pause here in this case you can see there is accelerometer in the box and you can see all the sensor data coming up on

SPEAKER_14: the screen so you can see you have an accelerator box which looks like a you know playing cards a pack of playing cards and uh you have an accelerometer in your phone so you get the idea and then you see like essentially a wave signal of some type three waves a purple a green and a yellow here so you shake it it

SPEAKER_01: moves that's right so you can see the you know the box and she's shaking it moving and now she's

SPEAKER_00: putting the box inside of the inside of the package and what you do is in our interface you can ask newton pretty much tell me tell me the transit status of your package how it moves through this thing so she put you inside of the package another person that's up transit transit status and turns it on and now what what's going to happen with newton is that as she moves the package around the newton

SPEAKER_01: translate this complex sensor data into the very understandable you know message the package is in motion the package is still so you don't need to go and understand what the sensor data means but it's actually you know with ensemble language and now you should change the prompt so we wrote package mishandling so a dropping or shaking and without changing the model without reversing the model without retraining the model the model can understand that sensor data needs to be tracked for

SPEAKER_00: the package dropped and you can see now it's analyzing and see the package drop so this demo

SPEAKER_01: demonstrates that how you can in real time kind of steer the model to look after this particular events

SPEAKER_00: or what they call behaviors in the physical world which demonstrate captured from sensor data something you should naturally cannot understand especially one more demo because and this is a difference between physical ai and and and you know classic lms because physically i is not a chatbot it's not something you are chatting it's something you asking a prompt and then the new model is looking for these behaviors is trying to understand and report these behaviors to you based on what you ask it for and the output doesn't necessarily have to be textual because if you see the imagine a worker at a factory or a doctor in a hospital or anybody who is working physical environment they have to be focused on the physical world they have to be focused on the task at hands so the textual representation is not the most natural for that kind of environment right so the model has to also produce outputs in other formats

SPEAKER_14: in visual so let me show you so this is a dash cam we're going to see now so it's a dash cam recording what's happening in the world it has some sensors i don't know lidar or just video um and then you're

SPEAKER_17: going to translate that into a language model right right but um in this particular case

SPEAKER_43: with a visual model i should say it's a unique model that's not a language model it's a visual model

SPEAKER_01: yeah is the same model newton which can uh translate either a text representation but the same model can render that in a very different representation and later uh during the show i can show you like

SPEAKER_00: a diagram which shows how that's happening and why that sort of translation is possible but in this example on the left side of the screen you see the real video on the right side you can see the overlap visual overlap that newton creates in response to the prompt or response to the question you ask and let's just show how it works right so this particular case you ask a monitor for a car in front and you can see there's a car in front and the newton highlights where is the kind front so obviously when the car crosses the road you can see this case you stop highlighting this and when the car the car passes by it's going to continue highlighting the car in front and the interesting thing here is that you can change the focus you can say stop through the car and look for pedestrians show me how does pedestrian run now you can see on the right side it's pedestrians who is being highlighted to give you know to direct your attention to them and for the same video so you're steering your model to do things which you need by text language here you're asking something to show me crowded areas but the output right now is not the visual overlay but on the right side you can see a heat map on the map based on gps data which shows you crowded areas so you can imagine a very simple use case where you have a fleet of vehicles and it's actually a real use case we're discussing when you have a fleet of vehicles fleet of cars which drives around the town to deliver goods or you

SPEAKER_01: know products you would like to report all other cars what's happening in the city so they don't

SPEAKER_00: get stuck somewhere because of the you know flooding or because of something some other events right so dynamic update of the map based on the semantic understanding of the world delivered by newton and that's one of the many use cases when it comes from understanding physical world

SPEAKER_46: all right you guys know i'm passionate about innovation and tech and i love hearing from founders i've got a crazy exciting opportunity for you to consider i'm hosting something called jam sessions with jcal it's a contest it's powered by my friends over at dot tech domains and over the summer i'm going to have five founders get the chance to do a jam session with me right here on this week in startups it's really simple you tell me in this one-on-one session what you're struggling with as a founder what are your challenges what's your vision for your startup tell me about your product tell me about your customers and we sit there we jam out i deeply listen to you i ask you really deep thoughtful questions you give me deep thoughtful answers and we try to figure out how to grow your business and then we publish it here on this week in startups so everybody gets to learn it's really simple there's only two rules here to get one of these five jam session slots one you got to have under 2 million in funding so this is for new startups and two you just have to have a dot tech domain name so here's what you do if you want to get more information jam with jcal dot tech jam with jcal dot tech that's that dot tech domain name you keep hearing about dot tech domains and i are trying

SPEAKER_48: to find the most innovative founders we have used the dot tech domain for many things and there's tons of people in the industry like rabbit dot tech you know that a really slick ai hardware device aurora dot tech one x dot tech if you're using a dot tech domain name for your startup's website i want to hear about it so apply for jam sessions with jcal jam with jcal dot tech don't wait there's five slots you've got a good chance of getting one if you apply now okay so this model uh that you're

SPEAKER_51: building newton can take any uh sensor data it could be lidar it could be cameras it could be an accelerometer put it into the model and that let you ask questions to i don't know solve problems in

SPEAKER_56: the real world or understand the world better that's exactly correct yes okay so is it an open source model or you is it a closed model right now at this time it's a closed source model we're not opening

SPEAKER_14: sourcing got it and so you're building this model and then you're hoping to get a bunch of training data and then solve problems for businesses and then allow them to the business model here is obviously to make this a hosted services like an amazon web services or something where people can give you sensor data and then query their sensor data and get some output so how are you training this you know

SPEAKER_51: because you show different sensors a camera sensor and then you showed an accelerometer i saw in another demo you gave you showed somebody touching a um a doorknob um and then i think you have other

SPEAKER_60: other models so tell me what sensors what inputs you currently have coming into it so right now at

SPEAKER_00: this point we're focusing uh on four kinds of sensor types you know first of all is the camera so with people using cameras obviously audio time series data and rf which is pretty much radars right this is kind of uh sensors we were focusing right now we're training our model for those sensors the data is coming the way approach we're taking is that very early at the stage of the company we did pretty broad review of the market we went out to literally hundreds of companies we talked to them asking what sensors they're using and what kind of things they want to do with sensors and that's how we selected this group of sensors initially and then very early we started engaging we built design uh partner program instead of engaging with us company to build specific understand how our model can be solving their specific use cases so the training data comes from you know either from partners are giving us data to be able to train our model for specific use cases and with every use cases the model can learn more and more things or for some of our partners we're collecting data ourselves in the physical world specifically let's know it would seem to me the number one use case

SPEAKER_51: here is self-driving and it was in your demo there this is uh i think what elon's gotten to when he shifted hard coding to a language model so is what you're doing essentially a broader version of that

SPEAKER_66: that's available to anybody who wants to use it that's exactly correct we're building architecture

SPEAKER_00: of the newton's architecture the way we're designing newton it designed the way so we can take any kind of sensor within those categories uh with very small amount of modifications sometimes out of the box it can use those sensors to solve their problems it's a very general purpose universal model

SPEAKER_01: for everybody because you know whether you go to the physical world in the physical world businesses

SPEAKER_00: you can't build bespoke solution for every single person or for every single business because it's so diverse and kind of messy the physical world in general yeah so it's universality of the model which is extremely critical for uh you know for being successful in this um in this field do you

SPEAKER_14: believe what elon's done with fsd and making this model and what you're pursuing will solve self-driving and if so when do you think self-driving will get solved because in your model here you're asking it hey tell me where it's congested tell me where there's a car tell me where there's people tell me where there's a cow and obstruction etc so you know one of the core questions is will we solve self-driving in all the edge cases by just watching humans drive and make mistakes and knowing it's a mistake or not so knowing what you know how close is tesla to having perfect driving yeah i or better

SPEAKER_51: than human because you must have used 12.4 12.3 and you're building something similar so just humanity in general yourself tesla let's just brought it out because you obviously don't work there yeah yeah

SPEAKER_00: well i don't work for tesla and uh we don't really focus on self-driving self-driving uh is just one of the use cases we're working with a few companies to help them with self-driving but that's not one of our one now we actually broadly building horizontal model across multiple multiple modalities we're working with a semiconductor company working with a automotive companies obviously but also consumer electronic companies and and construction companies like this is so like we're trying to build a generic model as always it's very hard to predict with anything which happens in the future like self-driving when it's going to be solved but i do believe that be able to understand contextual information beyond what sensing uh you know from the from the direct sensing but understand the context information behavior of the complex system behave with the people around it and using large language models style reasoning about the world around you would definitely bring full self-driving closer to solve all those complex use edge cases so how much of what you're doing is predicated on having a large

SPEAKER_51: data set um you know there are some people who have cameras in entire cities london china or both known you know or different cities in china are known for having massive surveillance systems for safety etc and so they have this massive amount of data if you had access to that man you would understand a large portion of the real world then you have satellite data maps gps data and then i guess people walking around with sensors on them or bicycles riding around with sensors them obviously a tesla or the waymo cars have massive sensor arrays so what what is your training data you

SPEAKER_17: know people look at all the language models using open crawl or reddit data or twitter data or quora data or stack overflow there's all these pools of data and oil what are the ones you're tapping into to

SPEAKER_01: understand the world right so as i mentioned we working uh with uh design partners and depending on

SPEAKER_00: the use case we're trying to solve for them we're tapping their data should they provide to us we're also using obviously a lot of open source data out there all these data sets which are available we're using them to kind of seed our model with the initial understanding of the world and kind of train the

SPEAKER_01: model on those you know like what we found out is that like when you work when you work with very

SPEAKER_00: specific customers the specific customers has a very specific problems you really have to work with

SPEAKER_01: those customers to get this data from them and then you fine-tune the model on their specific use case but we also were quite surprised that in many cases those customers are quite open to let the model to train on the on this data they can write the data later but once the model is trained everybody benefits so got it the approach is to do the piece by piece you're solving for one customer

SPEAKER_51: and that empowers everybody else got it and so factories are places where there's a lot at stake there's a lot moving around there are complex environments there's robots as humans so getting into factories and just understanding what's happening in a factory is is that one of the

SPEAKER_14: early use cases here and then do you need to make more sensors for those factories or they they already

SPEAKER_00: have the sensors in there don't they that's exactly correct you know one of our investors is hitachi right and the hitachi actually were interested in in our company exactly for that reason because they're already storing massive amount of data from the sensors right you know when we talk to them they say like look we have all this data but understanding this data and figuring out how these multiple data streams can be analyzed to understand not what a particular sensor does but how all together they can draw holistic image of the factory that's what we're looking for and the use cases that's just like endless there completely uh infinite amount of the edge

SPEAKER_51: well let's let's double click on that you know you have a factory building i don't know robotic arms right there's a there are factories that have robotic arms building robotic arms quite meta

SPEAKER_14: but let's say you have a factory that builds robotics and you um you know get all the input for the last five years of everything that's occurred in that factory then what would they ask and

SPEAKER_17: what would the benefit be once they have all that data in the language model because we showed very basic proof of concept demos here but in the real world what do you think they would then start

SPEAKER_84: asking it what what could they ask their factory that built i don't know uh cars or televisions

SPEAKER_67: i can give you the real use case okay please example with the real customer we we have so we

SPEAKER_00: having conversations right now with um a very kind of large semiconductor company right so these machines which making chips you know literally they're saying like look we have uh something like a plasma reactor for etching the silicon wafers right and some of these machines have up to 400 sensors inside of the machine and 200 of them are critical which means if they're off you know the value which

SPEAKER_01: is supposed to be it cannot cannot work and what they're saying is the problem there is that you take this machine and literally you move it or you shift it or change it by a meter and because the

SPEAKER_00: precisions and tolerances are so high everything's got out goes out of whack right away this means they all start getting false alarms the machine stops yields drops and then somebody has to come and reset

SPEAKER_01: the entire machine which can take days and a lot of money lost right which is obviously being passed passed to end customer and eventually to us right so the question was like can your model not just like

SPEAKER_00: look for the threshold values of the data but actually understand which data is correct which data is not correct so when you move it and it's actually moved around the model self-adjust to itself so this is one of the very specific use case where the factory is looking for solutions wow yeah that i

SPEAKER_91: mean that's incredible when you think about it these highly precise machines if monitored they have

SPEAKER_51: monitors and sensors already of course this is now the language model is watching it it could tell

SPEAKER_17: you what to fix it could maybe even fix it in real time i don't know if this it can actually adjust those 400 you know nuanced or it can actually do the adjustments with the machine itself i don't know how the machine is configured but at least being able to monitor it's going to save a lot of time and money i'm i was thinking of like a printing press to go old school not that we print much

SPEAKER_106: anymore but you watch those newspaper presses or magazine presses it was a very similar situation

SPEAKER_51: if they were off just a little bit because you see how fast they move exactly exactly the whole thing

SPEAKER_24: is just you're just throwing away a lot of off-printed newspapers yeah exactly it's a lot of loss right

SPEAKER_00: yeah surprising by the way you finally mentioned printing press we we also have conversations with people from printing press companies and they're using like really old printing presses yes and they are touching sensors or printing presses because they want to kind of like you know they're getting old and they need to you know adjust them to for the for their reliability and the same problem i have

SPEAKER_01: all the sensors around this printing press can you tell me when things are about to go wrong like predictive maintenance yeah calibration because time is money for these guys you know every minute

SPEAKER_29: machine doesn't work that's that's money juggling multiple devices and apps to run your business is a mess open phone is here to make it simple by simplifying your business communications with one easy to use app open phone has rethought every detail of what a modern business phone should be and here's the magic it works through a beautiful elegant app on your phone or you can just use it on your desktop making it super easy to get a business phone number for your entire team and you know how brilliant open phone is my teams use it every single day my sales team loves it my ops team they use it all day long and here's the features that we love you can create a shared phone number like customer support with multiple employees fielding all the calls and all the texts to that one number at my investment firm launch we pride ourselves on replying to every single call or email instantly and open phone is the number one rated business phone on g2 for customer satisfaction so here's your call to action super easy open phone is already affordable starts at just 13 bucks a month but twist listeners get an extra 20 off any plan for the first six months at openphone.com twist and if you have existing numbers with other services no problem open phone is going to port them over easy peasy lemon squeezy no extra cost head over to openphone.com twist to start your free trial and get 20 off

SPEAKER_14: we talked about one type of vehicle cars i wonder battleships and airplanes also pretty uh complex and with unlimited sensors my god the sensors in an airplane or a battleship i mean incredible has the military and space you know started to come out and say hey let's just take a look at all the sensors we have i could imagine a spacex rocket or a giant airplane a complex boeing airplane having all these sensors reporting in and then being able to ask questions i wonder if you could avoid accidents or maybe come up

SPEAKER_17: with insights on how to make those products have less drag or you know uh be more efficient in some way

SPEAKER_00: we haven't you have yet from the anybody from our space we have this conversation from aerospace industry or anybody anybody else uh from that but you know of course we open these conversations and love to talk to them i mean if you think about automotive industries as a kind of a proxy for this complex machinery so there's a lot of interest from automotive industry because you know the car generates some just gigantic amount of data i i've just read like maybe yesterday an article that like once ai cars i said bring you know bring to the cars even not full self-driving just like ai you know incorporated cars it's actually like 25 gigabyte data per per hour going to be generated by the car so how even process all

SPEAKER_01: that amount of data and how you can make sense of that how humans can understand that data so you need a sort of something in between which can help you to analyze this data and that's what newton is

SPEAKER_32: newton is look in the physical world that's all this data and gets you help help you to make sense of that world of physical physical data that's kind of our i was just wondering about environmental stuff

SPEAKER_112: you've got the obvious factories that are packed with centers but then we have the real world and we're

SPEAKER_51: very concerned about the rainforest we're concerned about oceans and temperatures and pressures and

SPEAKER_14: you know the amount of sunlight and etc precipitation and those sensors have also been deployed in many cases and those systems you know are i think incredibly complex weather systems come to mind you know global warming and co2 and all all of those have you have you started to think about

SPEAKER_51: how we might be able to use all the global sensors on the planet to maybe understand

SPEAKER_84: uh what's happening to to the ecology of the planet yeah i mean obviously like if you if you be yeah of

SPEAKER_00: course certainly certainly the um decarbonization and and um supporting kind of environmental you know environmental monitoring it's one of the most interesting um uh directions we can take to i'll give you another example which we discussed quite extensively with one of the um uh partners we uh you know in the process in the process of conversation you know the gigantic windows you know these those things you

SPEAKER_01: know that rotates uh like uh offshore so they have a very specific problem is that vibration of of the gearbox

SPEAKER_00: is a prediction of failure and you have basically this uh a windmill farm of you know like dozens of those windmills and this they all vibrate slightly differently to try to understand is this normal

SPEAKER_01: vibration is it the ground vibration is it is it the wind vibration is like what's making sense of

SPEAKER_00: this vibration would allow them to do either predictive maintenance or slightly adjust operation of those windmills to um to optimize their performance so this is exactly this is one of those problems which relates to what you what you're mentioning how to control those gigantic infrastructures which we are building in the physical world and how to use the sensor data to actually predict the future of what's

SPEAKER_56: going to happen with those machines and how do you think about the connection between robots and artificial intelligence obviously um we've got figure and optimists and a bunch of people are

SPEAKER_14: starting to look at this and there's lots of sensors in these robots boston dynamics uh obviously has been been doing this for a while so are are those going to eventually be out there in the world mapping the entire planet earth to give us some more information than we currently have right because what could be unlocked if you had perfect insights into everything occurring in a city everything we

SPEAKER_17: have seen this with perfect mapping right with gps has had a profound impact we don't get lost as a species pretty hard to get lost these days pretty hard to be out of communication with satellites and and you know sms to mobile phones now etc so so if we could with these robots you know if there were a

SPEAKER_14: billion robots on the planet and you had all the sensor data how does life change for humanity uh in

SPEAKER_84: your mind and maybe you talk just generally about robotics and the impact here um robotics is very

SPEAKER_00: interesting again so uh just like with the satellites and space companies we we haven't yet engaged with robotics companies we mostly focusing on our current focus is construction and automotive and you know factories and semiconductor factories particularly like all that stuff help them to solve the world that's our initial set of customers but we're talking to a lot of a lot of people obviously i think the speculation is always you know uh dangerous what's going to happen in the future but what we see from the data coming in one of the most important things people are asking for is some form of prediction and optimization in the way if you can because you know the best uh way to predict the future is actually is to understand the past right so if you have a certain amount of data captured about behavior of the factory or behavior of the building behavior of the ecosystem of such as a city right there is a potential to predict if you can use all those data in the long term just like with large language models by using all this data it is possible to predict potentially what's going to happen tomorrow the death of tomorrow a few days a few days after and by doing this you can optimize your energy consumption you can optimize your infrastructure control you can optimize how people to live better lives just be able to predict what's going to happen just like we predict the weather you know we should be able to give some sort of prediction what's going to happen with your factory what's happened to the city like where what happens to traffic what happens to things so start looking into the future with all this data and making better decisions now to either avoid unnecessary outcomes or prepare for them better that i think would be one opportunity we can see we can see here listen

SPEAKER_119: a strong sales team can make all the difference for a b2b startup but if you're going to hire sharks you need to let them hunt and you can't slow them down with compliance hurdles like sock2 what is sock2 well any company that stores customer data in the cloud needs to be sock2 compliant if you don't have your sock too tight your sales team can't close major deals it's that simple but thankfully vanta makes it real easy to get and renew your sock2 compliance on average vanta customers are compliant in just two to four weeks without vanta it takes three to five months vanta can save you hundreds of hours of work and up to 85 percent on compliance costs and vanta does more than just sock2 they also automate up to 90 compliance for gdpr hipaa and more so here's your call to action stop slowing your sales team down and use vanta get a thousand dollars off at vanta.com twist that's vanta.com

SPEAKER_51: twist for one thousand dollars off your sock2 there are seismic networks and seismographs or whatever those are that are predicting earthquakes right and i don't think we're particularly good at it it sounds like we could be a lot better at it with more sensors and more language models also tsunamis tornadoes so has that come up yet and have you studied those areas we haven't yet no we we try to keep our

SPEAKER_129: of course obviously our aperture as broad as possible and look as many but at the same time we

SPEAKER_24: kind of don't want to boil the ocean right so construction seems like a really good place to do it because there's a lot of construction yeah yeah construction was very interesting actually

SPEAKER_00: we actually very actively engaged with one of the i would say largest construction company they're based in japan and they're building these massive projects you know like you know terraforming style moving the mountains and changing direction of the rivers the project which takes years right and the problem they have is that they would like to opt because the amount of kind of like resources spent to build those projects is just humongous right how can we optimize this process you know going forward for future projects even to understand what's kind of like how the process was working right now looking the data like four years of the data and asking questions like well when this construction period started what's happened then how what was the throughput for that style how many people were engaged in this part of work just asking those questions is allow us to probably dramatically to reduce waste increase speed of building construction projects and reduce the cost of them

SPEAKER_01: so that is one of the uh actually active engagements we are right now pursuing just take a look at the data they have you know um and helping them to figure out what what's actually happened to this construction

SPEAKER_51: period yeah there are some giant construction projects going on obviously in the middle east in dubai in saudi neom and then yeah you have let alone some of these water projects you know whether it's moving water china's got a giant project to move water from i don't know if it's from the north to the south or the south to the north i'm i can't remember but there are some major major you know multi-decade projects in and also you have things like venice or seawall projects in amsterdam and these

SPEAKER_130: are you know these are tens of billions of dollars some of these projects they're that's exactly correct

SPEAKER_00: i think what's happened it's happening is that our ability to kind of like build things dramatically improve right so we can build those gigantic you know constructions we can build very precise chips and very precise technology nanometer and like you know two three nanometers you know parts as the project becoming more and more complex they generate you more and more data and it's not going to decrease like we're going to have more and more data coming in the avalanche of that is not stopping and we can't control those projects and this is going to control those technology without having very precise sensing and very precise understanding so this sort of like we kind of discussion with you too right so you need more data to control those things but you can't analyze those data so that's what we try to help in we're trying to help all those industries to

SPEAKER_51: understand you and i are of a certain age watching the last 30 years of development of sensor technology

SPEAKER_17: which was absolutely catalyzed by creating billions of smartphones the prices went down to nothing then you had storage and fiber and the storage costs have gotten down to you know very much commoditized you have bandwidth very much commoditized and it was just waiting for a technology to help us sort of analyze

SPEAKER_60: this at scale yeah and ai is that technology i completely agree i think when we when we started

SPEAKER_00: the company we were discussing that like you know there's a several building blocks for for the newton to happen for the archetype to happen we need a few building blocks we need first of course is a uh cheap sensing technology and kind of technology an industry to be ready to use sensing technology it has to be sort of mouth sort of like uh penetration of the sensing across the different industries and that happened you know there's a whole industry 4.0 movement the iot wave which happened it wasn't

SPEAKER_19: really successful but it put sensors everywhere everywhere why didn't iot like there was supposed to be this

SPEAKER_14: giant iot iot of everything and it kind of didn't happen except in your smartphone maybe in cameras but

SPEAKER_23: why did that have a false start do you think what was missing because of the problem of analyzing the

SPEAKER_149: data you have a siloed siloed data in a certain device that makes total sense the device by itself

SPEAKER_00: produces some so small amount of data and the value from that just one device is not very high because okay

SPEAKER_01: on off kind of signals how much value they're going to give you so it's okay whatever you know like i know my my fridge is on my fridge is off why do i care right it's when you start connecting different types of data together and then you try to try to place in the context of larger human life

SPEAKER_00: and kind of attach all this meaning of this data which has became possible with this foundational large language model approach and transformers and do this deep prediction that's when suddenly

SPEAKER_01: that's that's becoming possible but something dream of iot was was in the future right so i think this

SPEAKER_00: is sense is one of component bandwidth another component storage another component and of course ai this is fundamentally this kind of this transformer space space model which allows you to kind of like use a huge amount of data to predict you know and understand this uh this the future this is all

SPEAKER_01: components came together and this is sort of like a vision yeah now it's becoming possible it's super exciting for that reason it's almost as if ai was the keystone

SPEAKER_46: in this arch you know like yeah exactly all the bricks got built up and was like boom we should put ai in here and you know we're seeing it inside the human body quality sensors people have continuous glucose monitors heart rate monitors pulse oxygen levels steps uh and then people are getting prenovo full body scans blood work and nobody's put all those together that's the the body have you considered did that come up when you were doing your startup of like hey maybe we should just work on the human body

SPEAKER_14: and somebody should just take all that big data all those sensors and uh put them into some language

SPEAKER_129: model of the human body we will definitely of course we did right actually one of our advisors is uh chief technology officer of the orthopedics department of the ucsf right if you need to have a needed place

SPEAKER_00: he's the guy to go to um so we actually discussed very very deeply with him he has here this kind of this this idea the whole direction the pretty big direction is um um motion as a new vital sign

SPEAKER_01: which is very interesting you're saying that if you understand how people move through space you understand how healthy they are because the goals of the healthcare is to get you moving nobody is you know getting you know like better because just to like lay down on the bed and not do

SPEAKER_00: anything right the goal is to have an active life so by measuring motion we can measure the success of the healthcare so he was one of the first um kind of our advisors in the company and we deeply looked at the at the at the um health space the health space is tricky though right so there's

SPEAKER_156: a lot of regulatory regulatory uh you know yeah of course yeah i mean you can't it's totally different

SPEAKER_51: than somebody's house you know we're starting to see houses and buildings also have this technology where right as but one example we i now have in you know my my house and my ski house um humidity

SPEAKER_14: sensors water sensors temperature gauges that are all remote obviously we have cameras uh around the

SPEAKER_51: houses inside the houses etc and you know when something happens like there's a flood or water

SPEAKER_14: we're now getting a handle on that quicker earlier and then you know avoiding damage right and that's just the tiniest of and you know maybe one of the most common ones but boy it's going to get interesting over time right i think the nest is also doing some interesting things in terms of turning down the temperature or your air conditioning when the grid gets too high so you have two different systems

SPEAKER_00: that are interacting it's really going to be a brave new world right no exactly and again it's like one one of the interesting use cases we have you know like one and like a lot of the stuff um of

SPEAKER_05: archetype was informed by our work at google you know like just to tell me a little bit about the team it was we all worked at google and it's kind of like of building models for sensor data right we kind of try like understanding how to use sensor how to extract meaning from sensor data and actually

SPEAKER_00: put value right and one of the use cases one of the things we built we built this radar solely radar which is a project which actually i talked about it at one of your events a while ago yeah and at launch yeah and so we launched a very first sensor uh and built it kind of invented the whole the first sense which was consumer grade radars tiny radar which you can put in the phone or can put in air conditioning you can put into the into the um i remember yeah the pixel had this right to do the

SPEAKER_05: that's right that's what we that's what we did at google and then at that time we were kind of like

SPEAKER_01: first time look at the greatest radar sensor data and we realized that human cannot extract

SPEAKER_00: information of the sensitive it's impossible it is too complex sensitive signals too complex that was the first time we applied deep neural network to very complex sensor data which humans cannot understand and it was very successfully to the point that uh you know our last product uh at google was shipping a

SPEAKER_165: the uh sleep monitor right which can measure how well you sleep using radar and that's in that google

SPEAKER_46: home device that sits on your side table that's right and it watches you by radar and knows if you're moving around and gives you it's so funny you mentioned i have one of those google things a couple of feet away from me in my office which i used to watch my nest cameras and it i was in the settings page and had turned it on and off exactly because it's not it's in my office not next to my bed but what an incredible concept is that the radar is watching that right and yeah it's that sensitive

SPEAKER_45: enough to to monitor humans in a bed yes also your breathing your heartbeat it's extremely sensitive

SPEAKER_01: it's extremely and it's privacy secure right it doesn't have it's not a camera it doesn't see you

SPEAKER_05: it just see your motion and just how you you know act and then kind of like i think also you were

SPEAKER_70: using i think nest cameras were also using this a little bit or there are some not yet not yet uh not

SPEAKER_48: yes i know that there was talk of using this for um sudden infant death syndrome sids and watching

SPEAKER_17: babies because when you have a baby uh if you're a dad you know like you put the camera in there and once you put a camera in your baby's room now you're being super vigilant and all of a sudden your anxiety goes way up are they breathing or not did they stop breathing i mean it was more good to talk about babies dying but sadly sometimes babies will stop breathing and they roll over a certain way and they could suffocate it's happens in all every species and these these cameras could actually know when that's happening and put an alarm out i guess that's true yeah so there's a there's a couple of companies

SPEAKER_00: where where she used the radar for observing babies yeah there's people who put radars into the um just regular cameras to for the for the power consumption so if nothing happens it's it's radars looking around and when something coming in then the camera turns on so you can extend you extend the power of power life and just improve also false recognition and false alarms and that's it's one of the particular use cases but what i want to say is that so this is the first time we understand the data from from from from from from the radar like that's how you can use deep learning to understand this really

SPEAKER_129: complex sensor data that was one of the inspiration for for the company yeah the company that was doing

SPEAKER_14: this is called outlet uh duo or outlet dream and it is specifically using i believe radar and sleep to watch your baby and just maintain the environment so it's fascinating it's super interesting and i think it also has like a sleeve you can put on the foot yeah it does um so it uses talking about combinations

SPEAKER_46: of sensors you you could put a sensor on the baby itself and now ai is going to be able to tell you what's going on with your baby if your baby's lethargic or maybe it's got an upset stomach maybe

SPEAKER_14: the formula using is uh disrupting its sleep or something like this is incredible what we're on

SPEAKER_46: the precipice of exactly uh what gets you excited you know you're deep in this and you've been deep in it for a long time i do remember you were you were at our launch mobile event and you this is way back in the day when the pixel three or it was a very early pixel that you guys had this sensor stuff pixel 4 yeah it was very early um and so what gets you excited now when you're watching this progress and if you were to talk about the pace of change that's occurring you've been a technologist

SPEAKER_14: for three decades i believe um watching this last three decades talk to the audience just generally about the pace of innovation and what makes you excited today what excites me most is combination

SPEAKER_32: of the sensor data and artificial intelligence right i think that is uh is fascinating and it's

SPEAKER_00: you know like i i i worked i used to work at google i used to work at disney with building the sensors for the for the parks and resorts oh yeah right before that i was at sony and we build like

SPEAKER_93: very first kind of mobile devices and and and again did you work on the magic link project or what was it called general magic and that stuff there's a magic link project well at disney no i wasn't involved in that but i know really well that project no i wasn't part of that but we build the things like you know avatar land you know we build a sensing system there for the avatar land and you know with with all the rides and magic fountains and then you name it right all kind of sensitive technology so much fun yeah it's a lot of fun and it's you know like when you when you work in

SPEAKER_00: this thing you're realizing that the most important thing is is a narrative right so it's all right the narrative and narrative and distance around the magic this is a magic of technology magic of things happening before you anticipate like before they happen but anticipate you these things which guess what you want they can understand your sort of like you what you like to happen and they're

SPEAKER_05: happening for you so that's what kind of like people's really surprised and excited and and happy that's what people makes happy right when you are where our dreams come true right and i feel that

SPEAKER_01: this combination of sort of um sensing and prediction can you know that this is like can

SPEAKER_00: anticipate what people want can solve our problems before we even see them support us before we ask you that this kind of anticipatory interfaces in anticipatory use cases that's on the personal level

SPEAKER_194: i'm still kind of like imagineer and that that makes me super excited you know like that's kind of nice

SPEAKER_14: well i mean just looking at your face and understanding the mood you're in and you know people moving through a city like wow everybody's really depressed everybody's really anxious like what do we do here you're gonna have like this incredible pulse on the world um that we just didn't have insights into and that's what disney does you go to disneyland that's why they say it's the most magical place in the world because they're anticipating you know your experience and then

SPEAKER_46: delighting you with laughter or surprise thrills whatever it is yeah and those are all going to be customized right a certain person might go on a ride and you could actually sense that they want more thrilling or they want more i don't know storytelling or more fun you could actually adapt

SPEAKER_01: the ride to their particular age or desire right while extreme personalization right extreme personalizations of everything like that's you know because we're living we live in in the period of mass production right everything is mass producer things are cheap and we can buy them for the price and they're really really high quality it's amazing like product we can buy right now is amazing but yeah they designed one product fits everybody else yes so can we go back to the you know when you

SPEAKER_00: also have a personalization where every product is separate and works for you and that's something you know

SPEAKER_51: yeah i mean science fiction is just you got to work at disney so you got to see a little bit of this

SPEAKER_14: and a little bit of pixar a little bit of star wars but you know the minority report film uh minority report was just so um so many little items in there because they did go to mit and they a bunch of futurists and technologists contributed to i think that was spielberg who did it yeah and they they contributed to you know the different interfaces in fact i think the the gloves were an mit

SPEAKER_51: specific project that they just extrapolate on but in that film people are walking around and when

SPEAKER_17: they look at a billboard it tailors it to that person so ivan would get one ad i would get another you might like chinese food i might like japanese it's gonna direct us in the mall to the our preference it was and and here we are you know ads on the internet are as customized as they could possibly be in a way that people think it's like listening to our voices uh you know and listening to

SPEAKER_28: our microphones even though it's in most cases not this is amazing uh is it is the api available for hackers to start hacking on yet have you have you made a public or api or developer kit yet

SPEAKER_00: not yet not yet but uh we we are planning to do this uh we you know we're a process of building the core technology first and right now we're focusing on a few as i mentioned before we have a design partner programs and design partners programs open and we kind of inviting companies to join design part of program come to us with your problems and see if there is they're there for us and building technology with them i think once we have a few um kind of pilot cases built and demonstrated to the public and showing the value at the same time preserving generality of the platform we would love of course to open to broader audience and let everybody try with their own sense of data whether it's a mobile phone or from kind of iot device they have some few hacks in your kitchen you know connect to our model

SPEAKER_130: and try it out yourself that's coming got it and it's you can learn more at archetype dot are you io

David Friedberg: archetype yeah archetype ai dot io archetype ai dot io yes so you can understand the real world if if you're

SPEAKER_14: looking um to do a partnership it's interesting to you go over there and i know you're hiring so go to the website and go to the careers page as well uh and continued success with this is kind of

SPEAKER_46: mind-blowing it's very early but i wanted to have you on early because i know next year everybody's going to be talking about what you did and i wanted to put this moment in time in 2024 here because in 2025 2026 this is going to get really interesting so i hope you'll come back next year and uh tell the audience about you know all these incredible use cases that you're kind of stealthily working on and

SPEAKER_14: uh share more updates uh great seeing you again and we'll see you all next time on this week in startups bye-bye thank you