1,075 karma · joined October 15, 2024
What started as all-you-can-eat $50 buffet has quietly become a $6k bill, frontier models that don’t ship your codebase to Beijing don’t come cheap anymore.
Full agent coding however is the complete opposite, you’re in constant damage control of a junior who moves fast and breaks everything. They’ve got better but still do dumb mistakes.
lot of engineers are discovering firsthand what it’s like to manage a team of eager but useless employees. Not fun at all
This was a colleague favourite for flow state time
Yes nuclear option, but I’ll take an hour down time over a $100k unexpected bill
Now it’s owned by corporates and everyone is using bloated JS frameworks.
Anthropic when their own source code ends up on GitHub: Real s**
Shows the “compliance theatre” of what SOC2 has become
Interesting to use MCP for GRC but probably the most risk adverse to MCP segment to build in!
The security ratings in the readme are hilariously wrong
brew install llama.cpp
use the inbuilt CLI, Server or Chat interface. + Hook it up to any other app
AMD are doing gods work here
Llama.cpp already has KV compression and one of the turbo quant PRs will get merged at some point.
If you don’t care about the fancy 3 bit, the q8 KV compression is good enough! Don’t bother with q4
./build/bin/llama-server -m model.gguf \ --cache-type-k q8_0 \ --cache-type-v q8_0 \ -c 65536
Etc
Lemonade or even llama.cpp are much better optimised and arguably just as easy to use.
db-f1-micro is about $10pm inc storage for something that just works and can scale, be shifted on prem etc. you can run all your vibe coded slop on one instance.