andy99

I don’t understand any of them. Normally even when I’m not really familiar with a tool, I have enough background knowledge to understand why it’s funny e.g. Scheme and Haskell jokes or something. I do use basic git regularly and all of this is over my head. I don’t know if that speaks to how complicated and unintuitive some of the advanced stuff is?

If an AI researcher was going to pelicanmaxx, they would almost certainly apply the augmentations mentioned in the article during training, e.g. randomly selecting animals and conveyances. You’d want a model that generalizes well, just sfting in that specific prompt would be pretty bush league for a frontier lab.

I don’t have any reason to believe they are gaming the benchmark, just saying. I do find the idea of a data labeller having to generate thousands of svgs of different animals on different modes of transportation quite funny though.

This is more a statement of how awful Canadian banks are than anything else. For anyone unaware we have an oligopoly of five identical banks all of which treat their customers like shit and effectively extract tax from the Canadian population while providing nothing.

This AI written article seems to be substituting ethics for “taste” and making similar arguments to those from the past. Choosing what to do is more important than doing it is a taste problem, of which ethics is an aspect but one of many.

Real morality doesn’t pay, shallow “ethics” for business, tech, etc. has a whole industry that pays very all.

To go off topic a bit further, I recently went into Walgreens to buy some bottled water, and other than Evian, it was all advertised as alkaline. Is that just a trend, was it already alkaline and now that’s just in fashion? Personally I see it and worry I’ll get an upset stomach if it start messing with my pH.

It’s the Strix halo (AMD) with 128 GB shared memory. The 4bit quant is ~75GB.

Unfortunately I don’t know about the best way of running on an Nvidia gpu, you could try llama.cpp and offloading as many layers as possible into the gpu and using RAM for the rest, not sure if that would slow it down too much though.

Replying to myself, seems this PR was merged into main and it the model does work with a Vulkan backend on my Framework desktop, I’m getting about 220 tok/s prompt processing and 21 tok/s output on the 4-bit quant. This is really a sweet spot imo on this machine between maximizing ram use and still having decent speed due to the expert size. This looks really promising.

These articles are propaganda, it’s not an independent journalist writing it, they’re doing it at the behest of Anthropic or someone like them that’s pushing for regulatory capture.

This appears to be mostly due to fringe / activist views about copyright, rather than anything to do with quality or principle. If it was the latter I could get on board, as in instituting some standards against slop. But in reality it’s just letting activists hijack with their agenda.

Seems it’s not fully supported in mainline llama.cpp yet https://github.com/ggml-org/llama.cpp/pull/25165

In the huggingface link they mention building for CPU and for CUDA, does anyone know if that means it wouldn’t be possible to build targeting Vulkan?

This was posted earlier but didn’t get traction, and I made the following comment:

Id want to know if “AI” makes a material difference vs just having access to the wrong answer. Like someone could be given search access that successfully retrieved wrong answers to questions, would that give the same results. How much do uniquely AI characteristics, like sycophancy or the conversational aspect play into this, vs people just being willing to believe what they read?

Right, and there are two parallel tracks. First is the “every crack and crevice” part - “ summarize with AI”, “re write with AI”, “help me write”, “analyze with AI”, basically useless features being splattered everywhere in the name of incorporating AI while annoying the F out of everyone.

The second is the AI race / existential risk / pretending we’re Oppenheimer track which I think is the larpiest, basically pretending that LLM chatbots are some kind of super weapon and role playing accordingly.

None of this means they’re useful, AI coding, for one, obviously is a valuable tool (though there remains a lot to sort out). But the usefulness is unfortunately being overshadowed by, as you say, people not resetting expectations and instead wanting to play terminator or neuromamcer or whatever (I’m sure there are more relevant sci fi).

Id want to know if “AI” makes a material difference vs just having access to the wrong answer. Like someone could be given search access that successfully retrieved wrong answers to questions, would that give the same results. How much do uniquely AI characteristics, like sycophancy or the conversational aspect play into this, vs people just being willing to believe what they read?

Also, eating say a Big Mac and fries isn’t that unhealthy when done occasionally, there’s a lot of salt and fat but nothing horrendous. Compared to the load on your liver and pancreas etc of consuming literally about 1/2 pound of fructose that’s in a big soda. The soft drinks are by far the least healthy part.

I get about 12 tok/s with 27B 8 bit, 50 with 35B A3B 8 bit, and 12 with 3.5 122B A10B 4 bit. The latter is about 80 GB iirc. it feels like the best balance between using as much memory as I can and still having a smaller expert model for inference to give decent speed, but I haven’t actually rigorously compared the performance of the three models.

Edit: that’s for one machine, would be interested to know if the upstream commenter with two has them networked to run bigger models? If I had two I might be inclined to have them running in parallel, the obvious limitation I’ve found with a single machine is that I can’t parallelize any tasks and I think I’d get more use out of the extra speed vs a bigger model (there’s nothing I’m too excited about in the say 200B range that having 256GB memory would unlock). But am very curious what others do

Qwen 3.5 to 3.6 was a big jump for the same size, e.g. 29 to 32 on artificial analysis intelligence for the 35BA3B models. Although I don’t think anyone has released a better model of that size since.

I would love to see something like a 90B A6B model that is optimized for 128GB machines e.g. strix halo, I haven’t seen anything really targeting the combination of RAM and compute these machines have, but I’m biased because I have one.

2022, and pretty clear why.

Here is the discussion from the time: https://news.ycombinator.com/item?id=30977147

They did it interactively with Claude, it’s possible that it played up the significance and humor of the findings in a way that the interaction left the user feeling like they were really on to something.

People ask the same question of why YC funds yet another Uber for dogs or a button on the Touch Bar that cost $10/mo to help join a meeting faster.

They invest in people more than ideas, so you’ve got, at least in many cases, people with good pedigree and skills (age adjusted anyway) building on stupid ideas, but that are eminently employable.

Obviously there are other factors, I’m not really trying to defend anything but just point out that there are legit reasons why someone impressive enough to get into YC would also be impressive enough to get a good job. It’s not like it’s random founders off the street.

Interesting, I didn’t know this existed, do you think it’s competitive with Qwen 3.6 35B A3B which seems to be the closest comparator?

It’s 20 vs 32 in favor of Qwen on artificial analysis intelligence index (cohere isn’t benchmarked on the coding index)

I’ve done some of what I think this is, working on prem with customers, and I find it funny when I see jobs for FDEs that are somehow all in-office in San Francisco. The whole idea of being forward deployed I take to mean actually deployed.

I had a ~2008 vintage Samsung phone with a fingerprint sensor that gave your blood oxygen level (SpO2). One day it told me something similar, I had to agree to send them data or I couldn’t use it. So I never used it again, but yeah they have been abusing their costumers a long time.

This is they same company whose tvs take pictures of what you are watching and send them back to Samsung.

I’ve been using local models on a Framework desktop. I get > 50 tokens/s on Qwen 3.6 35B A3B, and I find that speed ok for coding (though the model not quite). If machine could do a SOTA model with say 0.5-2T parameters like GLM 5.2 at 100 tok/s, it would be very useable.

In 2 years presumably the SOTA models will be much bigger, but I won’t be surprised if there’s a fable level sub 50B parameter model by that point either*, in which case 1000 tok/s may be reasonable.

* on artificial analysis intelligence index, GPT 4o is 12 and Qwen 35B A3B is 32.

The funny thing about Fable, is we all but know it will be obsolete in a month or two. Between their embargo shenanigans (which IMO they could have avoided just by not pretending it was dangerous) and continuing to give access, whatever marginal advantage it had was essentially wasted.

It would have been an interesting experiment to charge more for it right away and see what the market would bear, rather than tease it for long enough for it to be presumably superseded any time now by whatever is next.

He says he might have been too harsh in his “eternal sloptember” post from may: https://geohot.github.io/blog/jekyll/update/2026/05/24/the-e...

I wonder what he thinks was too harsh, still seems pretty bang on, I think it’s going to age well.

The secret sauce is training data. They’re not just taking advantage of more compute (which obviously is necessary but as mentions basically a commodity). They are paying billions to data labelers and making judgements about the nature of the training data they best need to make the product they want. This seems to get pushed aside as a minor point but it’s the primary differentiator of the big labs.

I read before that iirc because of waning protection it’s better not to get it too early. It’s not clear to me why you can’t get it twice, but what I read (and it was some online discussion so could be wrong) was that someone had been specifically told by their doctor to wait to 50 as the best spot to get it. I’d like to know more, I’m in my 40s and would be happy to get it now too but not if it was going to be worse overall for some reason.

I’m assuming this person did ask in the second way, it’s hard to imagine someone working through a problem that has already tried a bunch of stuff just going in cold and not providing any context and saying “How do I do X?”

Good advice obviously if it’s not being followed already but also likely over-simplifying the problem. Also a normal person on the receiving end would probe a bit about what has already been tried. Which to be fair makes the whole thing a bit weird and does sound more like she’s being brushed off.

(in mice) - I get it if it’s some new experimental drug, a mouse probably makes sense to test first. With exercise you’d think they could go straight to humans? Seems like it would be more effort getting mice to lift weights than it’s worth.