Hacker News Reader

I just installed a $16k GPU at home - super pricy to be sure, but when you price it against hiring 4 junior developers working around the clock it's a steal

View on HN

What if you compare it to an OpenRouter account and $0.50 per million tokens?

Oh that's way cheaper to be sure, but a lot less fun!

you also don't have to worry about hiding API keys or security of confidential data which is a bonus.

Could you explain what your infra set up looks like and what is your typical use case, if its more than a hobby?

My use-case is coding. I'm typically coding interactively with an agent most of the day, and then I have overnight jobs that run every night.

As for the infra setup, I am currently using llama.cpp, although I'm just now working on switching to vllm since it is better for concurrent inference, since it can interleave prompt-processing and decode. Until now llama.cpp was enough, since I was on an R9700 and it was only practical to run one agent with decent sized context anyway.

Reply on news.ycombinator.com