boutell

I'm not sure how long ago you tried it, but look at Qwen 3.5 32b on a fast machine. Usually best to shut off thinking if you're not doing tool use.

I've been trying to get speech to text to work with a reasonable vocabulary on pis for a while. It's tough. All the modern models just need more GPU than is available

Only if you insist on running the standalone slack app for some reason. Why run one instance of Chrome when you can pay for two?

Patricia Anthony published a novel about this in '93. Cold Allies. It's good military sci fi. Doesn't pretend to offer answers.

Sure, hiring is expensive, but firing is really expensive! I salute your efforts to get it right on the first try.

Because you can buy it with 32GB of unified RAM, the MBP is now actually the cheapest device for something... useful local AI models!

Same as I feel when AWS goes down... "this sucks, but it's not like I'd do better" (at devops)

Ads might be financially viable for low quality AI. The cost of really good models is another thing entirely. There are people on Reddit asking when they will get ad supported Claude Code! The answer to that one is never.

"Failed to fetch."

It's missile command. But it's pretty enjoyable missile command.

Can confirm. I gave it a variant of the car wash question on a MacBook M4 with 32 GB of RAM. It produced output at a conversational speed, sure, but that started with 6 minutes of thinking output. 6 minutes.

On the plus side, it did figure out the question even without the first sentence that's intended as a bit of a giveaway.

The pivot to MTC is a big change in the infrastructure of https. I wish other browsers were at least mentioned in this blog post. I'm curious about the future of letsencrypt as well.

Later the vimkings pushed them out. Then they were suppressed by the vs-igoths, but modern day enthusiasts still call themselves neo-vimkings

The speed is ridiunkulous. No doubt.

The quantization looks pretty severe, which could make the comparison chart misleading. But I tried a trick question suggested by Claude and got nearly identical results in regular ollama and with the chatbot. And quantization to 3 or 4 bits still would not get you that HOLY CRAP WTF speed on other hardware!

This is a very impressive proof of concept. If they can deliver that medium-sized model they're talking about... if they can mass produce these... I notice you can't order one, so far.

Strong meme game. I'm on an older release and now I'm reluctant to update. In my current release, the verbosity is just where I want it and control-o is there when I really need it.

Flawless logic!

I have a spoiler-tastic fan theory about the movie Marty Supreme that is apropos here.

This stuff is cool. So is whisper. But I keep hoping for something that can run close to real time on a Raspberry Pi Zero 2 with a reasonable English vocabulary.

Right now everything is either archaic or requires too much RAM. CPU isn't as big of an issue as you'd think because the pi zero 2 is comparable to a pi 3.

Thank you. The waste heat problem is so bad but no one gets around to mentioning the fact that you can't have AI grade chips and space at the same time.

That's a solution for minuscule amounts of heat that nevertheless disturb extremely sensitive scientific experiments. Using gold, no less. This does not scale to a crapton of GPU waste heat.

How do you cool them? Getting rid of heat is one of the number one challenges on the ISS.

The most important benchmark:

https://boutell.dev/misc/qwen3-max-pelican.svg

I used Simon Willison's usual prompt.

It thought for over 2 minutes (free account). The commentary was even more glowing than the image.

It has a certain charm.

I feel like the time for this was two years ago, and LLMs are now less bothered by remembering syntax than I am. It's a nice lisp-y syntax though.

For those who would like to try it for real and not wait in a queue. It's a very fast model, even with your CPU it doesn't take very long.

This repo is a convenience wrapper that sets up Python just so, runs ml-sharp, and then builds a static WebXR website so you can view the results on a Quest headset or similar.

I find it is most interesting for indoor scenes with people in them. It's a bit like a rag doll version of everyone. Outdoor scenes with nature tend to generate complex topologies of polygons that kill the headset's frame rate.

I've cleaned this up since my earlier submission. It should now work on any Mac or Linux environment.

Knowing all of these is exactly what a developer shouldn't need to do. Fix "big O" problems in your own code. And be aware of a few exceptionally weird counterintuitive things if it matters on a "big O" level — like "you think this common operation is O(1) but it's actually O(N^2)". If there actually are any of those. And just get stuff done.

I guess you could find yourself in a situation where a 2X speedup is make or break and you're not a week away from needing 4X, etc. But not very often.

I like Claude but this is an absolutely tone deaf thing on Anthropic's part.

I've been pondering that given what the inputs are, llms should really be public domain. I don't necessarily mean legally, I know about transformative works and all that stuff. I'm thinking more on an ethical level.

I love programming in listicle:

HERE ARE THE TOP 10 fibbonacci_numbers:

YOU WON'T BELIEVE n := 6

He lost me a bit at the end talking about running chat bots on CPUs. I know it's possible, but it's inherently parallel computing isn't it? Would that ever really make sense? I expected to hear something more like low end consumer gpus.

Recent generation llms do seem to have some significant efficiency gains. And routers to decide if you really need all of their power on a given question. And Google is building their custom tpus. So I'm not sure if I buy the idea that everyone ignores efficiency.

Google owns 14% of Anthropic and Anthropic is using Google TPUs, as well as AWS Trainium and of course GPUs. It isn't necessary for one company to create both the winning hardware and the winning software to be part of the solution. In fact with the close race in software hardware seems like the better bet.

https://www.anthropic.com/news/expanding-our-use-of-google-c...

Had me going for a minute there.

This is absolutely the first thing I looked for too. They just barely mentioned thermal management at all. Maybe they know something I don't, but I know from past posts here that many people share this concern. Very strange that they didn't go there, or maybe they didn't go there because they have no solution and this is just greenwashing for the costs of AI.