967 karma · joined August 19, 2015
One year ago I already ran qwen2.5-coder 7B locally for pretty decent autocomplete. And I still use it today as I haven't found anything better, having tried plenty of alternatives.
Today I let LLM agents write probably 60-80% of the code, but I frequently have to steer and correct it and that final 20% still takes 80% of the time.
- Tool calling doesn't work properly with OpenCode
- It repeats itself very quickly. This is addressed in the Unsloth guide and can be "fixed" by setting --dry-multiplier to 1.1 or higher
- It makes a lot of spelling errors such as replacing class/file name characters with "1". Or when I asked it to check AGENTS.md it tried to open AGANTS.md
I tried both the Q4_K_XL and Q5_K_XL quantizations and they both suffer from these issues.
The flash model in this thread is more than 10x smaller (30B).
Codex is notably higher quality but also has me waiting forever. Hopefully these small models get better and better, not just at benchmarks.
Both Python and Ruby (the languages themselves) only specify the type hint syntax, but neither specifies anything about checking the actual types. That exercise is left for the implementations of third party type checkers.
The kind of messy code base that results from (large) numbers of (mediocre) developers hastily implementing hacky bug fixes and (incomplete) specifications under time pressure isn't necessarily solved by any technical solution such as type hints.
In other words, I found that the resulting code often looked more like Java but with weaker guarantees about types and much worse performance.
Adding static typing to a dynamic language mostly gives you the disadvantages of both, without a lot of benefits. It's better to stick to languages that were designed with static types from the start.
I love programming in Ruby, having to worry about type annotations and the additional constraints that come with them would take a lot of the fun out of that.
Support for operating systems, compilers, programming languages, etc.
This is why a Raspberry Pi is still so popular even though there are a lot of cheaper alternatives with theoretically better performance. The software support is often just not as good.
For the rest you'd still use non-GC rust.
https://sma-partner.com/storage/app/media/Dokumente/Netzgraf...
It's based on qwen 2.5 coder 7B and you should be able to run it locally quite easily since it would only require about 8 GB of VRAM for the 8-bit version. Not sure if Zed supports this though, I'm not a Zed user myself.
Biggest limitations are the memory bandwidth which limits token generation and the fact it's not a CUDA chip, meaning longer time until first token for theoretically similar hardware specifications.
Any model bigger than what fits in 32 GB VRAM is - in my opinion - currently unusable on "consumer" hardware. Perhaps a tinybox with 144 GB of VRAM and close to 6 TB/s memory bandwidth will get you a nice experience for consumer grade hardware but it's quite the investment (and power draw)