freerunnering

For some reason every time I type "Unsloth" macOS auto corrects it to "unclothe". It did it now, writing this reply. It's really annoying!

I was benchmarking different models, different engines, and different draft models, I posted a video on twitter, and people started asking about the setup in the final screen recording. So the blog post isn't so much "how a beginner should setup something" it's "here's the setup I posted in the video".

Original video: https://x.com/Freerunnering/status/2065275403548168398

And in the blog post there is a table showing the different speeds I got from different engines.

Slowest combo was 38.1 tk/s, and the fastest was 72.2 tk/s. All from "the same" model.

The full video is on Twitter: https://x.com/Freerunnering/status/2065275403548168398

Plus a followup one where you see me type the question in and press enter (though that video is with Qwen 3.6, not Gemma 4) https://x.com/Freerunnering/status/2065354101878055038

I do a lot of fine tuning and development with small models themselves (not just using an LLM over a HTTP API). So downloading the models directly and running them from the CLI was natural for me, so that's what I reached for when I wanted to play around with this.

The video is stuck in an `<img>` tag so you need to wait for it to load. On a slow connection it might just not show for a while. Though the video is only 1MB so should load in if you wait.

> I appreciate the author for sharing their experience, but for beginners this might not be the best guide to use.

Yeah, I didn't write this as a proper developer guide. My screen recording started getting loads of favourites and I started getting messages asking about how I set it up, so just through up a quick rundown of how I setup this test.

I little just saw the Unclothe announcement about "Double the speed" and thought "Ha. I wonder if that will get it fast enough I'd actually be prepared to use it" and had a go at setting it up.

I'd done tests before last year with things like Devstral, but they were always both so slow and dumb, I didn't want to bother.

This finally hit the "wow, this is useable" level of both speed and intelligence.