vardalab

Yeah, flash models, DeepSeek, MiMo, GLM, I love those things. For simple tasks like a daily routine shit, just setting up stuff and then doing the hard stuff in Claude/Codex, that's a reasonable approach for someone like me, a "gentleman code farmer", lol. And even lower tier stuff, I have the local models taking care of. Now that Jev is out I can finally have a true AI sysadmins managing my "cloud in the basement" homelab at the cost of electricity, which is not cheap btw

Well, if you are serious about it and you have Strix Halo, there are better ways of getting more context and capability and speed. Lookup halogen for Strix The most cost-effective local option right now, I think, is dual R9700. You can run a 27B dense Qwen at FP8 around with a full context and 2-3 concurrent sessions of 260K context. If you go down to an MXFP4, you get 4 to 5 concurrent sessions. And speed is on par with anything you'll get from hosted providers. You're getting between 60-80 for FP8 and 150+ tokens per second speed for MXFP4 and pp is 4K+. Lookup vllm radiance There is also a lot of progress in running Qwen 3.8 next flash with dual R9700. Obviously one gets less context and speed is a little bit less, but it's still very acceptable. Better than what you're getting with Llama on Strix Halo, that's for sure.

These are small dense models,meaning Qwen, have gotten capable and fast. And there's been a lot of progress in the area. So sticking with Llama you are not taking advantage of the hardware you have. And yeah, for Strix Halo, you should just look into halogen and you shouldn't be just using 96 gigs for the VRAM. You should give it most of the VRAM to the inference and connect to it from your laptop or something. People ar egetting 1000+ pp with qwen 3.8 Next Flash

I don’t even think that regular Astra medium is anywhere close to 100 for tg

Just ask your freaking agent to read it for you and extract the information. That's what it's all about. Why would I be reading these articles other than information?

Who would you blame it on? Physics, gravity, destiny?

Technitium is also really good as a performant local blocker and recursive dns server. I have been running mine for years now and it is fast as well.

When I was doing research (physical electronics, lasers, fiberoptics and sensors stuff), lot of time was spent just writing all sorts of DAQ and processing code. So all this LLM stuff would have been really useful. There's a lot of data collection, data processing in the lab that require all sorts of ad hoc scripts and stuff. That was many years ago, but I would be surprised if the current crop of researchers are not using these things. And if they are not, then they are just not serious.

Yeah, it's like day and night. It used to be really unpleasant to interact with early codex versions. Even 5.3 wasn't great. Now, I go to Sol if I need to discuss anything. I don't even bother with Opus because I know that it's going to give me a headache.

try using pi harness, hae not encountered these sort of problem myself also yuou can ask codex to look at the transcript and figure out the solutions to tool call failures that way

try ninfer once you get your 5090 https://github.com/Neroued/ninfer

Good 60-70 tg and 2K pp Qwen 3.8 27B FP8 can be had for about 5-6K (2xR9700 + PC) Gives about 3-4 concurrent sessions with full 262K

Fast 150+ tg and 2-8K pp Qwen 3.8 27B nvfp4 is about 8K (5090 +PC) Gives really only one concurrent session that flies because kv caching is not perfect for ninfer https://github.com/Neroued/ninfer

both are very serviceable, I prefer FP8 on 2xR9700

But, yes it doesn't scale that well but in 5 years the same hardware should still be very capable of running some great MoE models, for example Qwen 3.6 35BA3B on 5090 can fly at 600 tg

you can get 4xGB10 for <20K so that gets you about the same tg and pp will be probably better. Power consumption though will be something like 200W idle so that's a bummer. And you get VLLM and SGLANG unlike them mac where one has to use oMLX (nice but not the same concurrency or cache performance) I have 128GB M5 Max laptop and sill prefer to run things on other boxes in basement because its no fun to have the primary device being overloaded.

Nice! I will need to get this on my mikrotik router.

Using Sol XHigh or even High will deplete the Pro sub pretty fast in my experience if one is running any sort of automations in their harnesses. Sol-medium lets me squeek by with it using lesser subagents. Using ninfer on 5090 and 35BA3B qwen 3.6 is also kind of cool to get a local cerebras experience at 600 tk/, it does make errors so 27B is actually faster at the end at 140-150 tk/s. 35B is great though at digging through session logs and such at high speed.

How would this work on the dual monitors? Is there an option to have a secondary monitor have just one workspace while main monitor has multiple workspaces?

I actually changed the output style for Claude Code to use ASD-STE100 and it still doesn't help that much. It still comes up with a lot of stupid words like this gem "Standing where it stood"

Opus yesterday produced this gem: "Standing where it stood." I replied, "never stop stopping!" and we had a stalemate, lol

Opus yesterday produced this gem: "Standing where it stood."

I replied, "never stop stopping!" and we had a stalemate, lol

Yes, but we still have a leed in freedom according to the lore from our fearless leadership.

That will be 1100 bucks, lol

Sol 5.6 loves seams. I never heard the word mentioned until Sol 5.6 before that it never came up. Now everything is the seam, lol. At least it doesn't bear any load.

So, I grew up in the former Soviet Union. Starting with ninth grade entrance entrance and exit exams were like that. I remember I had to take university physics exam just to be able to qualify for a consideration. It was fairly rough, but I think in a way, good system because it sifted out the wannabes early on.

Yeah, I told it to save in its memory that I don't want to have any more word salad!

Says who? I kind of like it.

It's at least 2.5x that speed for dual sparks and prefill is good as well. Basically going on vibes it is faster seeming than what one gets by default with openAI or Anthropic.

My takeaway from this was that Claude was being completely useless as a helper Thanks to all the safety safeguards and that nonsense.

Yeah, but maybe Kimi doesn't pretend to know better than I do. Like, I just had a task today where I had a silly form where I wanted to copy in a signature from one document to another document. And, you know, I could have just used some other dumb tools, but I decided, hey, let me us ask Codex to do it. But it started lecturing me on how we can't do this kind of stuff. And it was just pure silliness. So hopefully Kimi Work doesn't try to pretend that it knows better and lets me decide on ethics.

This blog post here summed it up the best. These are tools, so let them be tools.

https://geohot.github.io/blog/jekyll/update/2026/07/11/ai-20...

One thing I find that's missing a lot or at least I haven't come across other than commercial offerings like AquaVoice is a decent injected technical vocabulary so that the initial transcript requires minimum cleanup afterwards. Because I mostly use these tools to essentially ramble at the command line with coding agents. So there's a lot of technical terms that don't translate well. Like OpenBao comes out as open bowel sometimes,lol. That necessitates significant cleanup prompt or background text available to the cleanup llm, usually in the form of screenshot or something that gets converted to text but that in turn requires good hw for speed to be almost imperceptible. For example m5 max turns cleanup into a noticeable delay while 5090 is decent.

Only way I have found that's relatively easy to inject technical vocab is to use whisper, but limited, I think to about 220 or so tokens. Whisper has sort of like a priming prompt where one can put in a bunch of technical words and it will try to recognize those. But again, that's limited to small number tokens. And that limits one use a relatively slow, by today's standards, whisper.cpp.

I benchmarked it across a bunch of different hardware that I have available, and Whisper gives decent performance as far as speed goes only on a pretty top-end GPU, such as a 5090 or 4070, like for example on Strix Halo, it's still relatively slow for longer transcriptions because I prefer just a stream of consciousness ramblings for minutes and then that being transcribed and cleaned up versus short sentences. So in that scenario something like 5090 really is good because the cleanup prompt runs fast using usually Qwen 3.6 MOE model. Whisper on 4070 itself is about 0.7 seconds for two or three minute transcription. So the total wait time for a three-minute transcription is roughly a second, or a little bit more than a second, so totally acceptable. But it does take decent hardware, and it grows to be double that on if running totally local. Well, in my case, it's all local, but it's my own hardware all over the place, but truly running on laptop, it's much faster using Parakeet, but then the cleanup is the bottleneck.

Anyway, it's just my experience messing around with this for the last year. I did start using AquaVoice, but their speed was exceptional, and tech vocab was exceptional, but they would have some annoying delays occasionally, and I didn't like paying the money and sending sensitive topics and screenshots into the cloud, and I had hardware, so my local solution is basically almost as good as commercial one. But I think they train their own model. So what I'm doing is I collect all the samples of my transcriptions, and I am slowly building my own data set that hopefully at some point when I get energy I will find some way to fine tune something.

now it's back? i saw it was gone from the usage but then reappeared about 10 min later, lol

A lot of stuff that has to do with VLLM and troubleshooting and compiling and building VLLM, compiling kernel, or just dealing with setting up eval for local models, it punts to Opus 4.8 on a regular basis. To the point that I have given up on using it for that purpose.