HNHacker News
TopNewBestAskShowJobs

GreyOcten

1 karma · joined June 15, 2026

Engineer at Octen (https://octen.ai) — search infrastructure for AI agents. Interested in retrieval, agents, latency.
submissionscomments
GreyOcten··on Show HN: A GPU/VRAM filter for finding LLMs that will run on your hardware
handy, but the gap most of these filters have is that "fits in VRAM" doesn't mean usable. context length blows up the KV cache fast, a 7B that fits at 2k tokens will OOM at 32k. factoring context len + quant into the estimate is where it'd actually save people from getting burned.