jerryliu12

Personally feel like it could be more ambitious with what it creates.

Woah didn't know about VNRecognizeTextRequest, that's super cool thanks for flagging!

Looks awesome, but a 30B model is too big. Vast majority of people probably have 32GB of RAM or less unfortunately.

Yep, you figured out how it works! That was the easiest solution I could come up with. I'm sure theres additional context on other screens but this was a good 90/10 solution.

Thanks, yeah I do need to flesh out the debugging options. In the menu bar you can click the Dayflow icon which should allow you to view the recordings folder. The sqlite db is in that folder too if you want to poke around there as well.

Thanks! Wanted to build something I'd personally be comfortable using.

Gemini 2.5 Pro is pretty expensive, mostly because videos take up a lot of tokens. It's roughly 1 million input tokens/hr, with a relatively insignificant amount of output tokens. Fortunately, Gemini has a very generous free tier, which is more than enough to cover daily usage. If you set up one paid project (and just don't consume any tokens), you can still use the free tier on a different project, and they can't train on your data.

If I had to put a grade on my own experience and evals, Gemini 2.5 pro produces A- results and qwen2.5vl is maybe like B-/C+. Obviously everything's nondetermistic, so it's hard to guarantee a level of quality.

I'm reading through papers that suggest it should be possible to get SOTA performance on local models via distillation, and that's what I'll experiment with next.

You definitely could! I think it would just be harder to get good semantic understanding of what you did during a segment of time without LLMs.

Yea, honestly I would hate if people used this to track _other_ people, especially bosses. I wanted to build something that gave people more agency to do more with their precious time, but there definitely is a fine line here.

Yep, helping people understand their distraction patterns would be an amazing feature. I find myself doing the same thing, funnily enough I also have that same Youtube extension.

Wow, yeah that's clever I hadn't thought of that. Will add as an advanced setting.

Yep! Have tested it out on Qwen 2.5VL 3B and it works reasonably well on my 16GB Macbook Air. The only thing I will say is that I don't think it's a great idea to run local models on laptop battery, since it's quite compute intensive and drains kinda quickly. Have tested with Ollama and LMStudio, but you should be able to use any OpenAI compatible local server.

Thanks! Between my friends and I, it's about a 50/50 split between local and cloud. I think it's great to be able to pick the tradeoff between quality/privacy based on your own privacy preferences.

That would be really cool, but for the foreseeable future there's still a lot of room to improve how screen data is used so I'll mostly be focused on that.

Recall (and Rewind) are similar in the sense that they both use screen data, but it's designed for retrieving specific things you saw, not semantically summarizing your time. My opinion is that they're completely different feature sets.

For some reason I thought this would be a semantic map - that would be super cool to see as well.

My main concern with running LLMs locally so far is that it absolutely kills your battery if you're constantly inferencing.

Have to say that I'm shocked that the local government hasn't cracked down on the Mario Kart stuff.

How did you get the compensation information?

Have you guys done any benchmarking to see which LLMs perform best?

Been using Cohere for a while now. Really impressed with how easy it is to set up!