kamranjon

There is a really interesting startup in Prague that is doing just that. They fine-tuned Qwen 3.6 27b to have 46% fewer reasoning tokens while maintaining most of the performance characteristics. I'm interested to see if they continue down this path of optimizing reasoning for other models.

https://bottlecapai.com/post/thinkingcap-qwen3-6-27b/

This does seem like a prett niche solution because the electricity is converted into heat and then just used directly as heating - which removes a lot of the utility of electricity.

It is a great solution for places where heating is the primary use of electricity and I hope it finds broader applications.

So while SSD streaming is interesting I'm not sure it's exactly the same thing as the per-layer embedding that is being utilized in tandem with streaming here. To utilize per-layer embedding, it would have had to be trained that way, which GLM 5.2 was not.

Pretty incredible performance for the footprint - really interested to see what could be done on slightly more powerful SBCs like some that have been mentioned in this thread.

Interesting to place that level of trust in the providers, but I guess that’s the best you can do with closed models. Makes me wonder if Opus 5 could have been trained on data they promised they weren’t training on? One of the interesting things about LLMs is how opaque they are from the outside, even with open weights, it’s very difficult to know if a model incorporated benchmark data in their training.

You keep saying open weights - but you aren't sharing any information on which models you are using. What benefit does using open weights models provide to the end user if there is zero transparency?

Wow I never knew this - what a useful bit of information. Makes me want to try a fountain pen, I do like writing in cursive but I often get hand cramps doing it for extended periods.

I’ve recently moved to sourcehut and really enjoy it so far! While I think the creator has strong personal opinions about LLMs - I don’t think they have any intentions of banning LLM generated projects. Anyhow just wanted to share because I think it’s a really nice product that isn’t overbloated with JS and just gets the thing done.

Is it a sandbox if the machines hosting your mock package servers have access to the internet? This just seems like a huge failure on the part of OpenAI to secure their environment during testing.

No benchmarks, no info on which models are used, ai generated video, just a signup page with nothing else.

Anyhow, this kinda reminds me of that quote about architecture: "We replaced our monolith with micro services so that every outage could be more like a murder mystery."

So here is an important question I think.

If LLM outputs aren't copywriteable and you create your own synthetic training set using Fable and share it publicly on huggingface, and someone else uses that training set to fine-tune a model, would this be considered illegal?

I ask because this happens all the time, synthetic datasets have basically become a key aspect of training a model at this point. I even generated a synthetic set from DeepSeek v4 to aid in fine-tuning a classifier just a few weeks ago.

So I just wonder on what grounds any of this makes sense, I wouldn't be surprised if some of these American labs were using open models on their own self hosted infrastructure to generate training data, but by nature of them being open nobody has to know.

I'll make a prediction: I don't think we will ever see any of the evidence of this "distillation" before they end up implementing some type of ban.

"It went from the start of training to launch in under nine weeks..."

This is pretty impressive.

What quant are you using?

Whoa whoa whoa, 118b params, 8b active MOE, long context reasoning, open weights - music to my ears. Hadn't heard of this lab before but I am very excited, will definitely try this out tomorrow - this is a real sweet spot I think in terms of model size and performance.

What I think is really incredible about this is that they are selling a ton of subscriptions, both personal and business subscriptions. So the fact that they are doing advertising on top of this suggests to me that they are in a deep deep hole, much deeper than he probably would have imagined when making this statement. If your revenue is 25 billion and you still need ads, your costs must be running you down at a pace you really didn't anticipate.

Pretty bold move to release this at the peak of the Open Models vs Proprietary Models debate.

Feels like a really easy decision when it's framed this way.

I don't really think the author is misrepresenting the quote in any way, it just seems like many people are reading it as a hard statistic instead of a reference to a properly attributed quote.

Just anecdotally though, my company is not a startup, well established and well known and has already started investigating, purely for dev purposes (not product), using Chinese models - this was spurred by costs rising much faster than expected.

So while I agree that I don't think it's anywhere near the 80% level across the board - I wouldn't be surprised if it starts moving that way.

"It’s obvious to me that there are ecosystem benefits throughout China, from manufacturing to scientific research; every sector can just plug in these models."

I agree with this sentiment and think it's echoed in Fareed Zakarias take here: https://youtu.be/VBblUjLw5lE

China seems to perceive AI as a much more sensible technology than the US and seems to be integrating it in far more industries than the US.

I'm not sure the American mind can understand the distributed benefits afforded to the Chinese economy from opening their AI models, I think it's pretty reductive to assume it's purely a strategy of undercutting American frontier labs.

DeepSeek V4 Flash

You can run Wordpress locally in your own sandbox to test these vulnerabilities, you don’t have to break any laws.

So I am likely coming from a completely different world because I use entirely self hosted models - but is it really common in your workflows to just wait for 12 minutes without any feedback? I am constantly watching my model and following the path it’s taking and cutting it off or steering it if it’s headed down a dead path. I was just curious if this is really common to just have no feedback for 12 minutes?

Unfortunately it is becoming more and more common for US alleys to use the pins with cords systems. Many lanes will actually upgrade to this system and I feel it is a mini tragedy in a way. I am very happy to see this because it gives me hope that there might be alternatives for bowling alleys who are considering the string based systems.

I wouldn’t say I’ve backtracked- I think I’ve been incredibly consistent here. Chinese labs are releasing open weight models, research and analysis. Anthropic is not. They haven’t produced any actual evidence of distillation themselves and what they have presented is tenuous at best.

By evidence I mean logs, I mean IP addresses, I mean timestamps. They claim millions of requests, let’s see literally any of them?

I don’t consider a tweet by Denise Wu, who works at Anthropic, to be reproducible evidence.

I don’t consider “Caveat: fully AI-generated research.” To be someone taking time to analyze anything in great detail.

Because two AI models produce vaguely similar front-end styles when generating similar prompts I also do not consider to be of much value?

I think this is what I mean when I say the U.S. has its head in the sand. The Chinese labs are releasing ~60 page research reports with citations and analyses and evidence and Anthropic is throwing up defensive blog posts with zilch. I’ve seen more detail in a tech blog from Uber than anything I’ve seen from Anthropic.

It’s so funny to me that Anthropic can make claims like this one with zero evidence provided.

DeepSeek and others like Minimax are publishing deep research on Multi-Head Latent Attention and Mixture of Experts, Multi-Token Prediction, novel Sparse Attention approaches, I mean they trained long context models on a fraction of the resources and gave everyone the recipe.

Chinese labs might not have the funding of labs like Anthropic, but at least they provide the receipts.

The fact that API based distillation is even a conversation right now makes me feel like the U.S. has their heads so far in the sand that it’s not really excusable.

These Chinese labs are producing novel models, publishing their techniques and sharing their open weights and the first topic of conversation is how they stole from U.S. AI labs.

Setting aside the fact that it doesn’t make any feasible sense to do API distillation, these models are outperforming frontier models on a number of benchmarks, and often times run more efficiently by several orders of magnitude.

We have to stop crying distillation, it’s getting embarrassing and at this point feels even a bit delusional.

If you look at this chart here it seems the tiny model has a WER of ~12%… not sure about the micro model:

https://github.com/moonshine-ai/moonshine#when-should-you-ch...

Allen AI is fighting the good fight with their Olmo models - I’m hoping for a new release soon. Also huggingface has released some pretty nice smaller models with open pipelines like Smollm.

The difference is that because they are open weights, it’s trivial to decensor the models with abliteration and you’ll generally see decensored versions within days of new releases - something you just can’t do with proprietary models from frontier labs.

I also think Allen AI with their Olmo models are doing great stuff by releasing full training data and pipelines and checkpoints which I’m not sure any other US company is really coming close to transparency-wise.