HNHacker News
TopNewBestAskShowJobs

sleepyeldrazi

91 karma · joined June 3, 2021

Personal projects at https://git.kokoham.com/sleepy . Trying to optimize LLM inference frameworks for local models in speed, and explore interesting quants for size.
submissionscomments
sleepyeldrazi··on Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
I have been getting good results with IQ4_NL and TurboQuant at 4bits on 24gb (3090). It easily fits 256k with that setup, but it starts slowing down quite a bit after 80-100k. Quality in my testing is also still good:

- Coding task test: https://github.com/sleepyeldrazi/llm_programming_tests/ - Design task test: https://github.com/sleepyeldrazi/llm-design-showcase

Coding was against minimax-m2.7 and glm-5, and the design against other small models

sleepyeldrazi··on Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
I ran 3 prompts (short versions, full version in the repo):

- Implement a numerically stable backward pass for layer normalization from scratch in NumPy.

- Design and implement a high-performance fused softmax + top-k kernel in CUDA (or CUDA-like pseudocode).

- Implement an efficient KV-cache system for autoregressive transformer inference from scratch.

and tested Qwen3.6-27B (IQ4_NL on a 3090) against MiniMax-M2.7 and GLM-5 with kimi k2.6 as the judge (imperfect, i know, it was 2AM). Qwen surpassed minimax and won 2/3 of the implementations again GLM-5 according to kimi k2.6, which still sounds insane to me. The env was a pi-mono with basic tools + a websearch tool pointing to my searxng (i dont think any of the models used it), with a slightly customized shorter system prompt. TurboQuant was at 4bit during all qwen tests. Full results https://github.com/sleepyeldrazi/llm_programming_tests.

I am also periodically testing small models in a https://www.whichai.dev style task to see their designs, and qwen3.6 27B also obliterated (imo) the other ones I tested https://github.com/sleepyeldrazi/llm-design-showcase .

Needless to say those tests are non-exhaustive and have flaws, but the trend from the official benchmarks looks like is being confirmed in my testing. If only it were a little faster on my 3090, we'll see how it performs once a DFlash for it drops.

sleepyeldrazi··on Xbox co-pilot mode changed disabled sister’s life
If he can comfortably move his head, might I suggest something VR related? Most of the first VR games that came out for mobile (i.e. using your phone as a VR screen) couldn't utilise a controller so they used head movement for controlling things. Usually you had e small crosshair in the middle with which you can select things by simply holding a position. I sadly can't remember the names of the games I tried but you might find them (or newer ones for newer headsets) through google. I know it's not a complete substitute but if they miss mountain walks, there are VR experiences (often videos) which simulate that. Might be interesting to try. I'm not aware of it but there might even be multiplayer ones, so that his wife can join him.
← PreviousPage 2 of 2