andy99

I thin HF holds huge power in how they’ve consolidated all the open models and data, but monopoly feels like a stretch here. They are in reality a fairly small startup with lots of alternatives (all of which are much worse at this point granted) and are mostly a loss leading part of the open source ecosystem. I think it would be very hard to make a case against this on monopoly grounds, though I wouldn’t be opposed to someone trying if they thought it was possible.

The open weights AI ecosystem is way too concentrated with HF. That said, it was clear they were going to either get bought or aggressively monetized at some point, Nvidia seems like it could be one of the more aligned acquirers.

I do worry about any sort of crowding out or downplaying non Nvidia-relevant quants, and about changing rules to crack down on models or datasets that for one reason or another “don’t align with their corporate values” - uncensored etc. Someone mentioned Microsoft and GitHub, they appear to me anyway to have been very hands off, I hope it’s the same model.

Is there reason to believe that outcomes would be better with many small players? Obviously the “blast radius” is bigger when it’s a big company with a huge footprint, but are incidents rarer and quality control better?

Completely anecdotal, I see lots or recalls from smaller producers I’ve never heard of, I can imagine they have less scrutiny and oversight.

Maybe I’m interpreting this differently but to me modern LLM+full text search means “agentic” - LLM gets to pick the search terms and iterate on them. The underlying LLM does know synonyms etc, better and more flexibly than an embedding model, and gets explainable feedback from failed searches.

> Qwen3.8-Flash-Next features a 125B-parameter main model, supplemented by an additional 51B N-gram embeddings, with 6B parameters activated per token.

Didn’t see this mentioned yet. I wonder what this means for the effective size. It’s evidently ~176B paramètres, but how does that get quantized. A 4-bit quant under 100GB seems unlikely, I’m suspecting this won’t run in 128GB unified memory

In principle I like the idea of trading more memory for compute though, even if there’s a memory shortage right now

I only see a 1-bit quant posted on unsloth HF and it’s 72.5 GB. Is that what you mean? That’s much bigger than I expected. If you can’t run a 4 bit quant in on Strix Halo it becomes a lot less interesting. https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF

The watermarking is an inextricable part of the token generation, it just using a known pseudo random sequence for the sampling. It’s not a transform that can be applied later.

These things are all pretences to track and control “normal” people anyway though. It’s not really about protecting children or about reducing porn access or whatever, it’s about control. It’s less important that the people that really want to do this stuff slip through, the key is to tighten the screws on the population generally. This is why there are apparent inconsistencies, which disappear once you realize what’s actually happening.

What is the problem with replying instantly? If you can give a reply immediately why wouldn’t you? If it’s low priority work that takes time, sure, but a typical yes/no, I’m available at 5, whatever, why not just send it?

If you’ve already read the email it’s less efficient to come back to it later

I wonder how they will use the data. If it was me I’d try to build a simulation of an airline, and then use it as an agent training environment. It really depends on the exact nature of the data what kinds of agents you could train, but maybe customer support (imo the worst AI use case) that are more empowered to make changes, or something for making more autonomous calls when recovering from irrops? Could be some cool’s stuff if a little niche, I hope they share / publish something and it doesn’t just disappear into a void.

Savings from back office automation go entirely into administrative bloat. I find it even less likely such efficiency would lead to breakthroughs than I do LLMs becoming super intelligent and doing it themselves, which I don’t find likely at all.

The big problem with overthinking on a dense model is obviously the speed hit you take. Going from Qwen 35BA3B to 27B for me is about 7-8x slower (should be ~9x?). This makes me a lot less patient for useless thinking tokens.

I’d want to compare this to the new Muse 30B model which is super terse and has a whole different way of thinking (no “Wait,”) and in my experiments was way more token efficient to the point that the absolute tok / s didn’t really matter.

I should have read that, it’s actually quite reasonable and I don’t really understand the objections in TFA having read it.

> One of my fundamental problem with this is that no two synonyms carry the exact same meaning. “He leaped at the chance” and “He jumped at the opportunity” are very similar sentences expressing the same general sentiment, but they are not the same. The exact words we choose when writing matter.

Doesn’t make sense at all in light of the actual approach, they’re just choosing a different RNG. It’s not like they’re corrupting it by flipping words.

Should add I don’t support the watermarking and requiring it is idiotic.

I don’t understand how this works for anything but prose. Is that the point? In any code or structured output, there just isn’t the flexibility, and depending on how the user requests the output be constrained there is even less (“answer only True or False”). So is it just chat responses? If I ask the API to tell me a story about Alice and Bob then it watermarks it, but when I ask it some implausibly constrained thing like write a story about Alice and Bob with each word starting in rotation with the letters alicebob, does it try to do so and hope there are roughly équiprobable tokens regularly?

This is astroturfing it looks like, or reddit style marketing, whatever you call it. A controversial sounding post linking to a company someone is trying to promote. Please join me in flagging.

worse, both mirror and normal bacteria have taken to wearing tiny chin-covering scarves so there’s no way to even tell which is which at a glance.

As a bonus, you’ve got a story for the “tell me about a time you hacked a non computer system” question on the YC application

What does the demand side look like? I understand why people have these tokens they want to sell, who wants to buy them?

> Software development was one of the last ‘guild fields’ and now we are seeing its industrialisation occur.

Doesn’t really support the rest of the argument. If you’re a skeptic and think it’s all hype, there’s nothing to worry about.

Just read the beginning but I feel like there are obvious counter examples. Basically lots of academic environments. I personally (and somewhat sadly) did my best work in grad school, thanks entirely to the environment. All of us in the lab I studied in benefitted from being around each other every day. From what I’ve seen, this is very common throughout history in academia.

Maybe it’s pressure that kills ideas, or competition, or premature desire to commercialize, or just the kind of people that become founders; it’s definitely not co-working in and of itself.

Should add I agree that solitary reflection is important, just not that being surrounded by like minded people is detrimental.

Think of it as a mad lib, it’s populating a template, and seeing the same template filled over and over gets tiring.

Needs to autoplay a video which when dismissed just moves to another window and continues playing. With sound.

They have to be careful. There’s not a lot that separates the top models anymore. Look at Grok which has almost no market share or credibility because of Musk, Despite being close to the top performance wise they are behind the Chinese models in adoption for example, which themselves are mostly behind the big two because of trust. It would not be hard to push users away, especially if a less unpalatable player ever emerged.

It’s always been interesting to see how similar output is across ostensibly very different models. I remember testing short story writing in the early days and having the models all choose the same niche topic across e.g GPT, Llama, Phi, Claude, etc

They also don’t appear to consider false positives.

Pretty famously, false positives make some other Anthropic products unusable. If you treat them as zero cost it’s easy to drive down miss rate.

People who work with AI all the time should be the most finely tuned in to how sloppy it looks and when to use it and when not.

I think we’re going to see a lot more “product“ focus in the future with deliberate attention paid to these kind of properties. Historically though there are some obvious differences, the focus has been on benchmark maximizing. As that saturates, I expect more interesting choices about writing and thinking style designed to be differentiators instead of a side effect. Kudos to the PM here for taking it in a different directions, there’s obviously been thought put into it.

Having spent a good part of the day with it, glimmer reminds me of Rorschach from The Watchmen. No unessential parts of speech, action oriented, brief and to the point. From a token perspective anyway it’s great, and it seems to hold its own well against more verbose models.

I really do feel like it’s effective tok / s is way higher because it doesn’t waste them.

It’s hard to know in retrospect what was strategy and what was dumb luck. This was in the midst of hysterical calls to limit access by “AI researchers” and safety/ethics types, when very facile takes still has a lot of sway (I think we’ll feel the same in three years about the current Fable stuff). It may have been hard for Meta to just release it outright.

What ended up happening was fairly limited gating followed by a “leaked” magnet link and llama.cpp which really brought a whole revolution in open use and changed the conversation completely.

I have no idea what role Meta played here, it may have been nothing, but they certainly could have been more guarded if they were really worried about the leak. The result was a big change in the trajectory of personal and open source AI use and even the dialog about it. Whatever the exact intentions, they were a key player.

It’s “I’m losing so I’ll try something else”. It’s what any rational actor in a market economy would do, and it should be encouraged. “I’m winning so I’ll pull up the ladder behind me” is what we want to avoid.