The inference machine is a basic headless gaming PC with a second hand RTX3090 and 64GB RAM.
The solar panels clock 5kW which is mostly fed back into the grid. It's sunny a lot here. The maximum consumption of the PC is 700W and it's mostly idle. Broad strokes we're net positive.
- Running queries on bank transaction data to figure out our home budget, savings and spending rates, etc. I don't want this data in the hands of AI companies.
- Joplin note-taking app LLM integration for searching, creating, summarizing notes. This is my private notebook and I don't want it leaking.
- Termux based phone assistant. I can use voice to ask it to add something to the shopping list or my notebook. Much faster than pulling up the app. Especially useful when out walking. Like a low-key home-grown Siri. I'm adding more skills to this as I need them.
- Building a little robot character in our home that people can use to ask weather etc. This is mostly an experiment so far.
- Various other one-off use-cases and experiments. Recently I've been testing out Clef which is like a self-hosted Jev that can read images. Response time of 300ms it's wild! Haven't found a use for it yet.