How ChatGPT & AI Embeddings Will Replace Google Search

0:00Incoming transmission from the future Vancouver AI community meetup. Kushaw is coming up here to give a talk about how he thinks about AI. A lot of us think we know what AI is. We even talk about it's like as a person or a thing, but it's this whole series of processes and stuff. And so Kush, I welcome you up here with your laptop to take 10 minutes and blow our minds about AI and what's really going on with next token prediction and how much what's going on under the hood, bro. Uh 15 minutes. Um No, no, no. I haven't been I haven't been I I didn't name myself. Right.

0:36Right. You know, by the way, by the way, since I have time, first debate of the evening. This is not me. I mean, this is Chris's [ __ ] with AI. That's not me. It's mostly you. That is not my hair. Honestly, beyond my hair, there were shoes in order. Those were not my shoe. These are my shoes, you know.

1:05Um, yeah. So, my name is Kush. Um, I'm a software engineer for the most part. Um, not that I'm allowed to be. I didn't study that. Um, my background is film and and like u visual effects and also like um industrial design. So, you know, designing furniture and stuff. Not that I've done that either professionally.

1:21Um, I just learn things and then I, you know, don't go into them as jobs. Uh, you know, and somehow I survive in Vancouver. Not really sure how once in a while someone just comes around and gives me like an app or something to build. Um yeah, and then you know there's still roof over my head still.

1:35So anyway, enough about me. That was supposed to be the about me slides that I do a few things. Um so today I'm here to talk about well uh initially what I was talking about with Chris a few different things I wanted to explore. Uh but one of them that I landed on was uh search and the and like like how embeddings work, right? But when I was talking about uh embeddings to grace as well as thinking about how to explain that, I thought maybe I should go back into the history of search. How has it evolved over the course? I love a history lesson. I love when people sort of give me a low down on how things went

2:06and when, what happened, etc. But then as I was doing that and I was like, oh, search has come so far. I was like, well, you know what? It still sucks. And and that's not like a epiphany. I know I feel it every single day. [ __ ] sucks. Uh and so I was like, you know what? I think I have to also address that even with all this stuff, it still sucks. So the talk is in two parts. First I'm going to talk about the history of search, the evolution of it and then I'm going to talk about why it still sucks.

2:29So all right so let's let's go into it. So the evolution of search big grand title. All right. So uh initially you know let's say even uh let's just say we have this data set right? We have this data set of around 45 movies. I've uh just put them all here. You know some of my some of the movies that I've watched um you know Die Hard is in there. you know, sort of man. Um, and they're like in three genres. Um, dark comedy, action, and sci-fi. Um, and we'll look into why that is in a moment. Um, we wanted to search something in this data set, right? So, let me also get out my

3:04phone so I have some sense of how much time I have so I don't get cheetah out of it as well as not go over it. Um, so I want to search the data set. Um, well, one of the ways that I could do it is perhaps alphabetically sorted. then go, you know, depending on the character Z or D or whatever, try and do that binary search in my mind. Um, how did people figure out how to search items, you know, in libraries and so on long time ago? Well, even like 200 BC, I believe, uh, we had uh people cataloging um, in libraries. Library of Alexandria famously has uh, scrolls categorized based on topics. So even back then and

3:40certainly uh you know even with 1605 uh there's uh I think it's Francis Bacon or whatever it was but there's exploitations of taxonomies etc. And so manual classification one great way of trying to find content in a large catalog. Of course right now I have only have 25 movies or whatever but you can imagine you know 25 million or something. Uh certainly Amazon products or whatever might be a billion um possibly. So okay so let's classify these things. I've now classified them into my database in such a way that um I have three different uh uh genres in which they are u categorized as well as I have a ability to sort it. So that

4:18helps as well if I want to look for the sort of oldest movies within the sci-fi categories that are in my library that I've purchased or something. Um I can go ahead and see Matrixes in there. Maybe I feel like watching that. Maybe I feel like watching Moon. I love Moon. Um so possibly right that that's a good way to explore the data. Um this is some form of search. You're searching for information that you know you want. You may not know exactly what it is, but certainly you have some information about it, genre, etc. Um, okay. Well, time passes by, people realize, well, by the way, there of course the clustering or the categorization of

4:50these genres is still a bit too rigid. U sometimes we may not have a movie that's just a comedy. It might also be a comedy sci-fi or it might be something like that. So, we could of course do multiple, but some torate that idea, I also I explored another vector which is themes. So themes can apply to multiple movies. A movie can have multiple themes. So let's say if I look for movies about greed, we see parasite is in there and we see the big short is in there. These are not perfect labels by the way. Don't come after me. I just, you know, sort of put it together.

5:16However, you know, just for the purpose of a contrived example. Um, and so okay, greed, greed, we have two movies. Let's say greed and finance probably the big short, right? Let's say greed and systemic failure, also the big shot. Perhaps if you go greed and inequality, maybe the parasite comes up. That's sort of the movie uh movies classification in this database. So that helped sure classification but a bit more u deeper on classification topic. Well then you know let's say jump past 2000 3,000 years we get computers and we can now do digital searches. We can do exact string matching. I know the name and even though the database is like 50 million you know entries I could find it very

5:53easily. May take you know the computer room a few you know days to figure it out. Um, so okay, so I have string string matching. Uh, let's say I want to look for a movie that is talk talking about shooting. Is there a movie like that? Sorry. Shooting there. Maybe maybe there's guns involved. Maybe there is revenge involved. Okay. Revenge.

6:13Killable hazard. Mission Impossible has it. Sure. But you saw as I was typing, I was trying to type a type a word, but it wasn't quite working and misspellings were not factored in. So if I say revenge doesn't really work. Well, this is pretty idiotic. Like it's a computer. it should be able to figure out something as you know simple as that and certainly for a while this was the norm but even as far back as 60s in fact a lot of stuff happens in the 60s computers wise um the idea of uh edit distance uh gets introduced which is uh called Levenstein distance by Vladmir Levenstein and the idea there is you can make typos because people of course

6:50started to type in search items in the on computers and this problem is now more prevalent than And so now you can say if I'm looking for let's say tension tension okay well even though I made a mistake in typing I can see that it it still matched it right it didn't just say or if I say tension like let's say whatever so the idea is now you're able to make some errors the UX is much more friendly um and you get to you get to access information that you may even have a misunderstanding about the spelling or simply made a mistake during typing or perhaps much more common the data set itself is flawed. You may very

7:27someone may have made an entry that is incorrectly um you know entered. So over time then people are like okay this is all great. We can we combine different elements together. So boolean search comes around around the same time 65 or 75 so on. Um and the idea there is okay well I'm going to give you multiple pieces of information about the thing that I'm looking for and I want you to find the exact specific thing that I have in mind. So this might be this thing and that thing and that thing if it matches it that's my criteria. So, I'm going to say I'm looking for a movie that is kind of poetic. Uh, okay. I see

7:58two two options, arrival and network. Well, actually I I'm looking for something that is dark comedy. Oh, network. Network. Great. Right. So, now you have two different uh vectors that you've applied on two different uh pieces of information that you're using together. You could of course do not I want this that is not that. But, you know, hey, I built this whole thing in 3 days. So, excuse me for not building that in. Um so all right moving forward from that uh we then get um people you know doing all this stuff and finding it still is realiz making us look for exactly the information and unless we know the data set unless we know what

8:31values are there what genres are there what uh names are even there and what descriptions might contain the words if we don't have the exact word let's say there's a word called global you know global is in here but if we say world is also there doesn't matter we won't really find exactly uh the thing unless the word is is in there in the data set.

8:52So people start to figure out what are ways in which we could go beyond just the exact keyword matching. And by the way, they made a lot of progress on the keyword matching stuff. They there's an algorithm called TF which is an effort to say hey if the if there's certain words that are represented in a given description um more often than not and if they're also common everyone everywhere else like the or and so on, ignore them. the words that are uncommon, prioritize them, and this way you can really search large data sets much faster than having to literally string match one by one by one by one constant search time go much faster. But

9:25even with all that stuff in BM25, another algorithm evolution of TF, people couldn't get quite past the feeling that we could have a better mapping of words, sentences, etc. That's when semantic search comes in, embeddings comes in. I'll I'll do like a little bit of visualization to show you how they work as well. But first, a demonstration. So, same data set, right?

9:45But now I can say, you know what? I'm looking for [ __ ] I'm have to think of examples. Uh, I forgot all the examples I had in mind. Uh, let's say a movie about I don't know, some guy on Wall Street making a lot of money. Okay. Uh, I'm going to hit u, it did not do a good job. This this engine is not very good. Uh, forgive me for that.

10:08Let's say I don't know identity crisis. Now it doesn't really show born identity. Okay, this whole thing is a joke. Doesn't matter. The thing works. Um um I I don't have that in the information in this data set, but let's say movie with a time loop uh where aliens invade and there's a big war going on.

10:34Okay, what the [ __ ] going on? Something go wrong with my search? Doesn't matter. something must certainly have gone wrong cuz it wasn't this bad before. Point is semantic search will match you the thing that you're looking for even if the words don't appear in the items themselves pretty common aspect you do have you have seen experienced this stuff now in the especially in the last few years um something that I want to mention though why am I talking about this stuff maybe I'll go about it later cuz then I'll you know be upset about it um so embeddings how does this semantic search actually works um work um well what's happening the way it's able to match meaning do

11:09something uh that is the same meaning but differently worded is using an embedding model. An embedding model is uh just like one of those lang large language models. It's it's some something similar. It's a neural network and it's trained there's two steps to it. One is a bird step. The other is contrasted learning and so on. But the idea is you you train a model to be able to first understand language to an extent so that it can just say generally this means that or this thing is closer to that. Um and then you then you train it to say this concept and that concept um are similar and now fine-tune your weights fine-tune yourself so that

11:44whenever I give you the string you put it in a point in space for now we're imagining a two dimensional space generally is 3D 4D 5D actually 1,000 dimensions but that means that now I have a function now I have a ability to take any concept any string any paragraph any large piece of text and make a address out of it make a location in space and happens to me the first time I learned about it and I I still don't understand the math uh of most things. I was like how the [ __ ] does this work? Like how can actually a model put two concepts that mean similar things in a similar point in space? This

12:18model is frozen. It's not learning anymore. It's learned and it's done. It's just ready for you to use always frozen in time. Somehow for this model if I put this description in a darkly comedy comedic thriller class warfight warfare the bad side one it shows up there. But then if I put in Fight Club, it shows up somewhere close to it. If I put as the description of Moon the movie, somehow it seems to be much farther away. If I put the description of Die Hard, it seems far away from Parasite and uh the Fight Club and also far away from this. So, it's interesting that it's able to map out. Now, this is kind

12:52of lacking a little bit in information about what have we really clustered. So, I went ahead and actually added colors to this based on the actual Ron genres that we already know. So now you're able to kind of see, oh, you know what? There seems to be already a cluster. The model already seems to have clustered all the dark comedies somewhere on the left, all the action films somewhere together on the right, and all the sci-fi ones somewhere over here. Now, it's very hard to get this chart to look right cuz I'm using something called PCA as a way to reduce the complexity of what this really is from a thousand dimensions into something that our eyes can

13:25understand and our brains can perceive. Um but uh certainly uh now you can really understand that this model is quite good at placing meaning into a point in space. So anyway uh we get semantic search we are able to now for example I built this demo to do something fun but I'm totally blanking out on all the examples but let give someone give me an uh description of a movie that you have in mind of an alien.

13:51First interview of an alien. All right it showed up in the sci-fi area. Right. So, these are all sci-fi movies. Children of men, whatever. Arrival, right? Um, let's say, uh, oh [ __ ] I should have, whatever, doesn't matter. Let's pick another one. I don't know. Yeah. Let's say man goes around killing people for fun. I don't [ __ ] know. Um, whatever. Yeah. Yeah. Also, also sci-fi.

14:18Yeah. Yeah. I don't really know. I I had all these Okay. Epic space battle. Um, you know, I don't know. Cool stuff is happening. Uh it's is also in the sci-fi cluster kind of close to action as well. Um well, I'm really like forgetting all the examples I had in mind. Doesn't matter. Uh but you get that it's it's somehow approximately putting the this new string that I have in mind into a point in space. Now, how could you use it about the brain of an artist? A movie about the brain of an artist.

14:52Okay, this is quite ambiguous. It could be somewhere between this, this, and this. So, I guess that's good enough, you know, like kind of ambiguous where it would be. Would it be I don't know if it's a rock biology. Yeah. Let's see. Uh, beautiful biotech. Uh, okay. You know what? I don't know. But point is you can see in this demo that I built things moving and get a better intuition for how you're embedding models. One thing I will mention is this what this example is not great because I only have a very small data set. If you have a lot bigger data set that you have applied PCI to and store what you call igon

15:22vectors which is how I'm keeping this very stable as I change things then you will have a better ability to see things actually land where where they are but they don't and I didn't have the time to make the example more fun in any case semantic search that's how uh oh sorry one thing I want to mention sorry uh okay this so um what's happening here is this so if for example semantics is how you how we use this you have something in mind a movie that you want to watch.

15:49Let's say the description of it is but you know it's Fight Club right? You can just say I want movies like Fight Club. How would recommendation system benefit from this? It will say simply find this movie's address in that point in latent space. Then walk around, get your nearest neighbors 1 2 3 4 maybe this one as well and get this block back to me.

16:10That could those could be the the recommendations. That's lat space. So lat space is the idea. I don't I don't have an official meaning of it but the idea is that that highdimensional space where these embeddings are being uh these these vectors um put those concepts to be that you can call laten space. You can also use laten space as concept in large language models where you say within the weights the multiplication is happening and changing the vectors the elements response to this or that you generally can refer to the internal representation that the model has as latent space. come to think of it as like the dream space of an AI.

16:47Like all the things that are possible in reality but that maybe haven't been done yet. the the the inner brain possibility space of honestly that's a really great way of thinking about it and I find more intuition for that when you're talking about st like like visual models like uh diffusion models like stable diffusion there it's a lot more obvious cuz you can literally and this is the craziest coolest thing ever when stable diffusion was open sourced you had videos of people talking about how you can simply do ar multiplication and you have like a picture of a ship uh flying in space let's say um and you do ar multiplication towards a certain direction and you'll see the ship

17:20transform form into a like into a plane or something or or you'll see the ship the sky turn turn into rain or something. So this is like just math using math as a way to change the picture and so there it was a lot more obvious that you were actually navigating latent space. Um in this case it's a you can do it there's like ways to visualize it. I have a different talk that I've done about how language models work and then I talk about it. I think I've talked about this too much so I'm going to move on to the the next part which is after okay maybe I'll bring more. So I have so after all this time

17:51all this research we have semantic search we have agentic search as well we can ask a language model to find stuff and if we if it doesn't get right we can give it feedback etc. It still sucks. Many many reasons. Few that I want to get into. Search engine optimization. The the worst [ __ ] thing I can think of. In the last 15 years, Google's monopoly and a bunch of other companies, but certainly Google's monopoly has allowed them to simply dictate whims on what they feel is representable, what they feel should be. I I personally am wondering like what happened like when at what point cuz I wasn't there for it.

18:22Certainly, you know, who allowed a private index to be the de facto thing? Like a private index for the open web? What a brilliant idea. Who the [ __ ] did that, you know, like like I wasn't there to invent it. I'm still not there to invent it. I barely do anything. But what the [ __ ] Like what? Like the de facto index is a private private thing and it'll have ads and it'll have its own profit motives and that's okay with people like no no one's even fighting it. You don't see every year a new uh thing trying to fight it. No, a private index. Brilliant idea guys. Um so next thing is lack of multimodal. This one I

18:55can't really blame on anyone. It's getting better now though which is over the last 10 years especially because it used to be much better search and the open web existed. A lot of the data now is videos, Tik Tok videos, YouTube videos, um images that perhaps are hard to u transcribe etc or uh you know uh caption etc. And as a result all that piece of information is still buried somewhere in some person's channel cannot access it. the information is there. We just simply don't have access to it because no one's built a proper enough index around it. Next thing, and this one really pisses me off, is open communities have gone to [ __ ] because of

19:30policies that these private companies that somehow ran these uh supposedly open uh public spheres uh have have chosen to do. For example, uh Reddit, which pretty much everyone can generally in the last 15 years have assumed it's like a public sphere. You go there, you share knowledge, you learn. Did anyone ever like think when you were using Reddit like oh uh this feels really nice to make another dollar for the CEO of Reddit? No, you were thinking like I really love this topic. I'm engaging with the friend with the people around this who are also engaged in this. I'm learning from them. I'm going to publish contribute here as opposed to publishing a book about it. These are the feelings

20:01that people had as they were talking on Reddit and yet they closed their API. Now, OpenAI and Google made multi, you know, 70 60 billion dollar deals and now they get to only scrape and their model are the only ones that get to have the knowledge that most people using it consider to be public knowledge that they're contributing openly. They don't feel like they're only contributing to a private pool. So, there's that API lockdown. Same thing happened with uh Twitter. Don't even get me started on Twitter. Uh but like like $40,000 per month for for for their API. So, pretty much like a no. Um and similarly decline of um open source open open forums which is you know like Reddit and

20:37so on could be considered that but also there used to be so many real for open forums like 20 years ago and now you have things like discord things like slack where even open source projects local llama a bunch of different like just communities have discords now and stable diffusion has a discord like it's like so you're doing all your conversations on discord non-scrapable non-indexable people are talking there and those things those things just vanish So this means that people are still willing in this current moment to contribute openly to to share the the things that they care about to share knowledge to answer people's questions it just does not end up in a public

21:11space that I can find what that means is gatekeeping which what what not necessarily gatekeeping in this sense there a different word for it but it means that if you are someone who's looking for information and you're not an insider if you don't know where to look if you're not part of that given Discord if you're not part of that given uh community on Slack or on WhatsApp or whatever it may be you simply cannot find that information so that information is opaque to you. Even though the people in that community, a 100,000 person community has stable division, do you think those people are caring about privacy of the knowledge they're sharing? No, they actually want

21:38to contribute publicly. It just happens to be that the medium is broken, right? So, so many things and more contribute uh to this decline in the quality of search because nothing is now publicly index indexable and even if it was the index itself is private and they go motivated. As I talked about, as I looked into this topic more and more, um the theme that emerged was that things were not this bad. They didn't have to be this bad. They became this bad because of choices that people made. And information is the key key thing here.

22:06Information that people want to share, knowledge people want to share that could be ending up in a public index that doesn't anymore because of many reasons. Stack overflow no one goes to anymore cuz they go to a private model whose queries always remain private, etc. And one person that I found inspiration from as I was looking into who was who had a clarity of vision about these things that like like no other uh was Aaron Swartz whose words have been uh behind uh the slide and you can look into it. Um this is his manifesto um the gorilla open access manifesto and he talks about in complete clarity what we should be doing if you

22:39care about this uh mission of open access to knowledge and if you know the story of Aaron Swartz he uh wasounded for downloading a bunch of information off of JTO which is an MIT archive um which was from what I understood from recent researching a bit more an open network. He just went over the limits.

22:57But they made it sound like he made it he he ended up doing a crime and then they pursued him for two years trying to jail him for 35 years and then he killed himself. So that's just one person but his clarity of purpose and his contributions to making RSS to a bunch of other things are significant. So anyway, I felt like as I was doing this talk, I sort of kind of got really nervous. I was like, you know what, I feel like there's it's much bigger than me. It's much bigger this thing. It's even much bigger than semantic search.

23:22There's real things broken in the heart of search. and you know it'll take our efforts to fix it. Thank you. [Applause]

23:37Uh what what was the main what's the main thing you wanted to share with people tonight? Um, initially I wanted to talk about honestly agentic search, semantics or something that I didn't even get to but as I went deeper went deeper into the topic I realized like much more important at least to this audience talking about uh how things work which is cool you'll be curious enough and find it and you can just bug me or you know any kind of uh uh forum online. The more important thing was to kind of just re reign people of the reality of where we are right now which is everything is getting more and more closed. Um and unless we actively

24:13participate in the communities that we build being more open the software that we build having different values etc. We will continue to go down this road that it's not like a you don't just get it for free it's like a the price of freedom or something. That's my first question is is you do get it for free and that's how they made the public web a private index is they gave us a lot of [ __ ] utility for free. They gave us Gmail, Gcal, Google Drive, all these things, right? And we slurp that [ __ ] up and then we end up with, you know, private company owning the index, the whole public internet. So for all of us

24:46like normals, how do you square that circle, man? That's how they ended up where they're at. Like we all still want Gmail, Gcal, Google Drive, all that kind of [ __ ] So, how do we balance that equation? We can't all run those things on our own servers at home. I mean, this the ultimate question is like what are the alternatives and and and if you're willing to put the care and put the effort into actually create the alternative, right? Um, it's not like I have it's not like I've created a social network that's open source and made his own Photoshop. I make stuff and then I can use in a very hacky shy way and if I

25:17can make it useful enough then, you know, I can share with people and that's kind of my approach. But certainly, I'm not solving the problem. So I I do think that there is this philosophy that I've adopted a while ago now which is that generally I had this character thing where like if I can't fix it I'm not going to complain about it a long time ago I change that completely doesn't matter if I can't fix it if I see it wrong I got to here here right and so so I think I think the solution to that I'm an anarchist and they tell me that I'm wrong because I don't have the solution to come what next what comes next and

25:47I'm like I don't need to have the solution my job is to say this is [ __ ] burn it down and then you guys job is to find the solution So go ahead and keep critiquing the stuff that and and and I try in a little bit of like trying to educate be like well here's how you could try and build a system but I certainly am not the kind of person who wants to get some VC money [ __ ] around and like make another thing that again ends up being a purchased private bot thing and I certainly don't have the resources to get a big farm and a farm together to produce and serve off the server like

26:17this and I just don't have this faith in the current sort of incumbents of the current you know in in places of authority to do the right thing if I do approach them from a nonprofit from a not you know profit motivated motive please. Yeah. So I have a question for you. So if we go hit back I'm James Row.

26:37I'm from circles of AI. All right. Um so my question is about uh lessons from history. So before digital all knowledge was in books. books were traditionally controlled by elites and but eventually the governments opened up libraries and that knowledge was made publicly available. Are there lessons to be drawn from this? Um I mean I think that's sort of the trend of you know information become more and more accessible over time and we kind of saw some of the peak of that trend in the last well let's say 10 years ago or whatever people really getting the ability to and then actively sharing uh information on those forums and then it became sort of a regression from it. Uh

27:19I definitely agree like that's how the past has been. information was kind of the domain of the people who were the priests or whatever and everyone else can go pound sand but over time we had more ability to share and more ability to learn I'm not saying it's a given default it is in fact isn't the the default is that people find u actually the default is not that people find that they want to keep things private the default is that the people in power find that they like power and so they keep it I don't find general the general person to want to actually uh attain more power this is not really a trend that I've

27:50seen well and one of the things that came for me around the um lock down APIs, Reddit thing is like it really sucks if you're contributing to what you think is one type of community and they lock it all down and you can't access it anymore. That's [ __ ] cheating, right? Like who thinks about that? Does anyone really ever truly think when they're on Reddit that oh like actually this is not public? Like this is the three people who are going to read this comment. It's not public, but you're contributing to their proprietary data set gold mine that they're going to exploit the [ __ ] out of. Like you're actually enriching them. It's not just that you lost access

28:16to using it yourself via your own software. That was that was exactly the last thing that I skipped which is that closed source AI that's why I keep going on about it cuz I don't find that open that open source or close source things in AI same as in software it's about information actually closed source AI is like literally information being logged down cuz an open source model is trillions of tokens or hundreds of gigabytes or whatever of information compressed and compressed from form such that it can even answer questions in an instruct model that is massively useful imagine so all the all the information that now charge GPT gets because of all the people going to that model or

28:48anthropic is because people are going to that model as opposed to stack overflow was the same people who were using stack overflow before. So it's not like the field of software engineering disappeared is that the entire information was now going into a hole. Same as Reddit people are still using it actively. There's not like I know plenty of communities that strike and they you know went dark and so on but plenty of people are still using it like it's it's still a thing and that information will never see light of day in an open source model and it'll just be in a clos and then you people wonder how come closed source company are frontier. Yeah, they

29:16definitely have. It's because of for smarter scientists you Yeah, we have time for one more question for Kush Niels has one. Does anyone else have one? I was going to do battle royal. I was going to make you figure out who wanted to ask the question more. So yesterday for some of the web summit attendees, we had a speech by the CEO of Blue Sky who's kind of creating an open-source uh social media platform where other people it it seems very uh counter to what some of the other things are doing. I was wondering if you've looked into Blue Sky at all and maybe what your take is on what they're doing.

29:52Are they actually doing trying to combat this or or is that just marketing? Yeah. So I mean this is actually one of the questions that's harder for me to answer because yes I I know a bunch of people around me who are using blue sky even as developers and and the some of the protocols around them and and how to use the fire hose to create your own apps etc. But I myself have not really engaged with it as much because it's like you know I'll tell you this I am not like a smart person. I'm almost certainly like not smart enough to figure out things. What I find is I try to train my my my

30:26[ __ ] sense um kind of like being inspired from Carl think from from Jordan George Carlin uh you know sort of tradition like I don't need to know everything to sort of try and sus out the bad actors or people that are you know cynical about these things and I just don't have a general inclination to lean in full heart you know full fully into something that came out of a CEO that already built a different kind world did not put it into action. They say that yeah we we built this protocol at Twitter because you we wanted to put it into production. Okay. What what took you so long? What happened? Oh yeah. And

30:59then you sold the company that was okay with you. So there there's so many things that they say. Yeah. It's kind of like you know when Google brought project uh Ara it was called. There was a video I I'll never forget this. There was a video inspired the [ __ ] out of me.

31:11I was studying at the time phone blocks. It was this guy talking about a modular phone and it was going to be this is a pretty big digression but modular phone that you could switch out the camera, switch out the screen, you know, awesome stuff and Google's like, "Oh, we're going to hire the guy. We're going to make this product." They hired the guy and it was very inspired. They called the project a shut it down 2 years later. No one will ever ever attempt it again. For all I know, it's because of, you know, some kind of rights or IP stuff that they made happen. But I don't see anyone trying that stuff again. Is

31:37it because that idea is [ __ ] Is it because everyone learned that lesson that modular phones are not a thing? No, it's because this is what happens, you know? Yeah, I don't just I just don't buy that an entity that made billions trillions that thrive off of being private is going to suddenly produce something that is open source. Doesn't mean the people working there don't have those values. I just not going to lean into that. There's it's a protocol.

31:55There's sure hundreds more. Yeah. Thank you, Kash. We appreciate it, man. I encourage you to check out his YouTube channel. He's got a lot of great videos that he's made explaining this stuff that he then went and edited in an open source video editor tool that he created himself from scratch. So, uh, he's the real deal. Thanks, Kush. Seriously, bro.