This is basically what Microsoft wants to do with Recall and they got slammed for it. Which drives me nuts because it's the only feature from the recent AI hype wave that excites me because it's the only thing so far that sounds like it will actually make my life better. But then I thought about it a bit more and I realized that what I really want is not AI, I just want my computer to have a detailed local history and search functionality.
My computer should remember everything I did on it, period. It should remember every website I visited, exactly how far down I scrolled on each page, every thought I typed and subsequently deleted before posting... And it should have total recall! I should be able to rewind back to any point in time and track exactly what happened, because it's a computer. I already have a lossy memory of stuff that happened yesterday and that's inside my head. The whole point of having my computer remember stuff for me is that it's supposed to do it better than me.
And I want the search to be deterministic. I want to be able to input precise timestamps and include boolean operators. Yes, it would be helpful to have fuzzy matches, recommendations and a natural language processing layer too, but Lucene et al already did that acceptably well for local datasets 20+ years ago. It's great we have a common corpus, but I don't care about getting tokenized prose from the corpus, I care about the stuff I did on my own computer!
From my perspective LLMs don't bring much value on the personalized search front. The way I understand it, the nature of their encoding makes it impossible to get back the data you were actually looking for unless that data was also stored and indexed the traditional way, in which case you could have just skipped the layer of indirection and queried the source data in the first place.
I am also curious to see how all of this develops. I get a sense that the current trend of injecting LLMs everywhere is a temporary stop-gap measure used to give people the illusion of a computer that knows everything because researchers haven't yet figured out how to actually index "everything" in a performant way. But for the use case of personalized search, the computer doesn't actually need to know "everything", it only needs to know about text that was visible on-screen, plus a bit of metadata (time period, cursor position, clipboard, URL etc). If we currently still need an LLM to index that because snapshotting the actual text and throwing it into a traditional index requires too much disk space, okay, but then what's next? Because just being able to have a vague conversation about a thing I kindasorta maybe was doing yesterday is not it. Total recall is it.