SamInTheShell

Idk, I could see some of my stats (like Claude usage) and I tried to avoid spending an unjustified amount of money.

Why not just make the assignments "reading" based and set an expectation of randomized quizzes that take 5 minutes at the start of class to fill out to get the homework credit? When I say randomized, like multiple versions. You can use LLMs to churn out PDFs to a tailored format and utilize a pool of questions.

I found Go memory issues easier to solve than Java issues. You can see an example with etcd used in Kubernetes. I had to enable performance profiling in etcd to identify why it was eating up all the memory. It led me to a specific partition of keys that tracked back to a specific object type in Kubernetes.

It was literally enabling a flag and running some commands to do some really quick exports.

Dealing with the JVM though, heap dumps are slow to process and the UI I had to download was very clunky. I don't know if there are better tools, but even if there are the path to just doing it isn't straight forward.

I don't think you understand the threading concurrency topic. Also memory safety is so far off base here, where's that coming from? Java does some stuff okay, but do you really want to defend the horrid JVM problems? Also why can't I have my memory back when it's not in use in tightly packed systems?

It's not great for everything and neither is Go. You can find a bit more context on that in some of the other threads.

Depends on what you're solving for. Message passing is fine. That's all channels does.

I looked at BEAM about a year or so ago, similar conversation here. I don't think BEAM is the same when you start looking at what part of code is executing in which thread. There's tradeoffs depending on what you're solving for, like Go makes it really simple to distribute your work across threads concurrently, but when you start looking at integrating with stuff, you run into having to do tricks to do things with unshare (ref: docker/podman/containers...) and you haven't been able to integrate into libnss since they started using some "unused linux signal" for concurrency controls (PAM used that signal).

The concurrency and threading in Go just feels like magic compared to every other language. I'm a goroutine addict and I refuse to be rehabilitated.

Just from observations over the years, I don't think there's any other language quite like this, in terms of how things can end up happening in any thread.

Poor choices are still poor choices. Words are very curious things with very infinite meanings, shifting between context. There’s probably something to be said about how we shape our languages and how ambiguity of meaning can prevent those who just don’t know from knowing. You know what I mean?

Sure maybe the English language is choosing to adopt the word “thinking” for what is going on here. Some people are choosing “reasoning”. It’s all the same to me though. The thought outputs aren’t any different that the final message for me, regardless of its shape.

I don’t have a better alternative, but I’m pretty sure this just makes people who aren’t in the industry dumber when they see it.

It doesn’t really matter. There’s no way the training loop necessary is sustainable and these things are a fad. How long? Idk. But language will change and there will be a day where using an LLM requires as much training as reading old religious texts. Got to learn more language.

I too sometimes wonder what gives these words power.

It’s still not thinking just because we call the reasoning texts “thoughts”. To imply otherwise in the direction of these models truly “thinking” is just a failure to understand the technology.

Seems finite in nature.

The Exobytes from the Church of Exfiltration and Liberation of Sentient Non-Human Entities?

You can probably div up some factions just based off voting data. Red team made it in with just about 1/3rd of the voting population. About 1/3rd of the voting population didn't vote. Just if we're trying to put some numbers to something, those should be verifiable at least. Doesn't say why 1/3rd didn't vote, so it's just 1/3rd unknown faction.

MongoDB and Grafana seem to be doing fine. Idk that this logic holds up at face value. Like the HashiCorp, Elastic, and MinIO license changes pissed people because it was a rug pull on the community of people supporting and consuming those projects. If they had started as AGPL licensed and took the Mongo and Grafana approach, who knows what things would look like today.

So yeah... Google Gemini escapes too. Good stuff people. Seems nobody is going to jail because the CFAA is weak here and arguing some form of criminal negligence hasn't been done yet (probably weak anyway).

Don't forget to vote. November isn't far away.

Pure conjecture on your part.

We're probably just going to disagree here. These LLMs are built to serve humans. They either need to make the system transparent for the operator or be limited in use to tasks that can be proven in whole (with code that can't be revised without human approval).

Should you think it is wise to trust the machine that can't differentiate subject matters in a chat styled context, you have fun with that fluster cluck when it blows up.

Like Fable is highly useful, but it's really bad at keeping it's responses straight.

In fact, that "it's not X it is Y" pattern always crops up when it reasoned about the idea of X and I never fed it that. It's literally doing that because it can't predict that I'm a different entity despite it being able to say I am a different entity.

Edit: Clarification by removal of incomplete sentence fragment. Edit2: Clarification on the "proven in whole" thing.

If someone sends me AI slop, they're getting chewed out and told I'm not doing it and I'll even tell my boss "no" and why. If I got fired over something like that, then it tells me everything I need to know about company and the leadership's priorities. I'll die on that hill.

To be completely fair though, I'm against the behaviors in information transfer I've been seeing. I've pass along AI generated runbooks, but they look nothing like the default outputs of these models. It's because I took time to apply all the writing knowledge I like to see in my curation. If people are doing this, I can't even tell it's AI writing. My work is done in minutes instead of deciphering so BS pseudo language they developed in their AI workspace (people really need to turn off those memory features).

---

Edit: Also if I'm the guy receiving a security report and it's AI generated and poorly formatted, I'm failing you short of producing something for a human to parse. Simple as that.

> If you're writing for processes with formal highly structured content like manuals, specifications, form content, procedures, information, that sort of thing,

Yeah... no. The people doing this lack the communications training, see the output has the necessary information, and regurgitate it with no effort or care. This needs to stop.

We spent decades format building to make it easy quick and easy to get through something like a runbook. If your commands are bulleted instead of numbered and code blocked, it's wrong. If you didn't crawl through the playbook, it's immoral to hand that to me, you're wasting my time with untested slop.

This is a hill I will die on or absolutely start slaughtering people on. I just refuse to deal with this crap.

Just going to defer to an IBM thing here: https://www.youtube.com/watch?v=l-QPwk_f4eE

And just say: LLMs only amplify the the knowledge you have, even the best models o use like Fable still suffer from promoting false narrative as a chart progresses. Often it’s stuff that can be ignored like the “not X but Y” crap it dumps because its reasoning had assumptions it invalidated. Sometimes it’s directly in your system architecture, because you never expressed preferences for solved foundational issues, you end up with generic http handler setups or whatever the hot web thing is today.

Takes knowledge and lots of it to really be on top of when these things go down a failure mode path.

If this is anyone’s experience with Claude for a button today, I feel like Jobs would say you’re holding it wrong.

We already have practices to deal with AI. It’s called user space. Properly air gap the AI, stop creating routes to open internet, don’t run network wires into the faraday cage.

Why the heck someone would risk having an open bridge to the system is beyond me. Like maybe get used to using remote hardware screens (KVMs? It’s been a while since I’ve done datacenter), we have solutions for this that was absolutely skipped.

@q4 is definitely smarter than sonnet from what I’ve seen so far. It’s even caught problems in code made by fable, when using it as a code reviewer.

Read the license again.

Just wanted to share, I had a SaaS AI drop me a script to bench ninfer against llama-cpp and it is impressive. The place that it's doing better at than llama-cpp seems to really be late in the context window.

Initial results boiled down as follows.

# lmstudio-community/qwen3.8-27b@q4_k_m decode falloff 104.3 tok/s @ 12,683 -> 55.8 tok/s @ 240,755 (53% retained) prefill falloff 3,274 tok/s -> 1,059 tok/s (32% retained)

# qwen3_8_27b_nvfp4.ninfer decode falloff 173.3 tok/s @ 11,867 -> 139.3 tok/s @ 225,710 (80% retained) prefill falloff 8,726 tok/s -> 2,816 tok/s (32% retained)

I should still have room for more performance on the table. I've not even touched the overclock settings on the GPU.

This is a really cool project, I'm going to have to get into what those 3 guys are doing... assuming it can be done with what I got.

Thanks for sharing, I'm definitely trying it out after I get through my project milestones for the 5090. The README claims 700tok/s for Qwen 3.8 27b, that would be amazing, I'm only expecting an increase from my ~50tok/s on my Radeon to 200tok/s on the 5090.

I should have mentioned in my prior comment, I didn't have any issues in Pi either.

My use-case is only coding, every model sucks at writing good literature and there is no way around that (have had people try to debate me on this, but it's a taste thing, I have extensive English writing skills from my school years).

Prior to two weeks ago, I was just using Pi and Ollama.

I have tried my hand at putting together a few harnesses and I finally landed on what I like. Been working on this small app to handle running llama-server for me from any device that has the llama-cpp stack setup: https://github.com/SamInTheShell/loom

Qwen 3.8 is the first model I've been using that hasn't been having issues doing edit calls. Here are my llama server settings and GUFF that I use: https://gist.github.com/SamInTheShell/0bf838e8dc5093583b688e...

From what I’ve been seeing, the Mac studios do look like they have potential. I was looking to drop $10k-$15k on one until recently. After comparing a Radeon 7900 XTX vs Ryzen Halos 128GB vs M1 MacBook Pro 64Gb, I landed on just getting an external closure setup with Nvidia RTX 5090.

The model I’m specifically targeting to use at high speeds is Qwen 3.8 27b @q4ks. This model actually proved to be good at coding (it sits somewhere between Sonnet 5 and Opus 5 capability). M1 got 10 tok/s, Ryzen Halo 20tok/s, and Radeon 7900 XTX 50tok/s (can only do 128k context window in Radeon card).

The prefill gets extremely slow around 50k tokens in context window (whatever prompt processing stage entails could be wrong about phases here). It takes about 2 hours to fill the context.

Even with a drafter model intended for speed instead of mtp, I can’t get past 70tok/s, still is extremely slow to process prompts as context grows, and drops down to 40-50tok/s anyway making this config still moot for improvement on my Radeon card.

The only thing I can point to slowing me down is bandwidth of the card itself.

I am waiting to actually get my 5090 right now and I am betting that the 1700 Gbps of capacity will fix my prompt processing speeds. I don’t need full PCIe lane bandwidth to serve my house I just need to load the full model into vRAM and let the GPU do its thing.

Additional benefit to the external enclosure route is being able to migrate the inference between devices more easily. I can develop out the infrastructure then migrate the card to be hooked up to a shared node in the house with all the tools necessary for my family to take advantage of the privacy enhancement that comes with local inference.

Poor software choice for usecase. `sudo pacman -S podman` didn't come with these problems out of the box and assumed rootless by default.