You can also just acknowledge that we don't know what they did (and that is because they chose not to tell us). Apparently, until not so long ago Jev would spell out its identity as Q-w-e-n if asked, so we might as well assume that what they did is at least similar to what this blog poster did (who chose to tell us).
Extremely fast hype and a very short-lived moat, those are the parallels that I see. The OpenClaw guy was smart to cash out at peak hype, and chances are Jev will be overrun by big players and open-weights pretty soon too. I really love the idea, but just don't see any sustainable moat here. People are already coming knocking [0], it should only be a matter of time until we get something like Qwen3.9-Classify (or a classify mode just baked into a multimodal LLM).
Technically, Q is picked because it has the highest probability of all letters. But it makes a difference whether the probability for Q is barely above a uniform 1/26~3.8% or whether that one letter concentrates >50%. What I remember from reading the docs is that Jev gives you the full probabilities (and the confidence, which is something like normalized entropy).
But in general, we might be reading too much into this. If I were to build something like this, a Qwen model would be among the first things I'd reach for too. Initially just prompted inside a little harness to guarantee you get the desired output. Next step would be finetuning, finally training your own foundation model, if you can muster the funding. In this fast-moving space, I think it's quite understandable that they'd go public with an MVP asap, so likely not much training on their own. And even if they're finetuning, Qwen's baked-in answer (through Alibaba's finetuning) seems likely to survive unless it was explicitly overridden.
Once the first letter is Q, the rest is probably pretty determined. Can you see the confidence for the first letter (don't want to accept the ToS to follow your link)?
I mean, there's centuries' worth of mathematical prose to train on. But that's presumably already in the training data, so if it isn't good enough today, it might not get better fast enough to keep track with how fast they'll get better by training on formally verified math. But then again, the prose in Terry's conversation I linked above seemed pretty useful. But it's also a problem requiring famously little advanced mathematics.
> Separately, our group has been exploring approaches along these lines to make such evaluations more scalable
Actually, that sounds like an interesting idea for peer review in general, to include an interview between referees and authors. If it saves one round of rebuttals/reactions, it needn't even consume a lot more of everyone's time if you're doing those things properly. What it would undermine would be blindness, but something's gotta give, and it was already on its way out.
> we might imagine what it could look like to have an analog of the Millennium Prize Problems for open exposition problems
The core idea seems to me that we should shift the standards for professional evaluation from generating proofs to generating explanations. Makes sense that such a proposal would come from the 3B1B guy, and I actually agree with it, irrespective of AI. But what eludes me is how that could be a defensive mechanism against AI automating humans out of mathematics. AI is likely no less good at producing natural language explanations as it is at generating rigorous proofs. It's telling that even Terrence Tao turned to AI to understand AI-generated results [0]. It seems that the essay doesn't address that issue at all.
Well, I hope you're going to be right. My gut feeling, however, is that putting all the little bits that make up the AVP experience together would already have been too much to ask from other manufacturers in the absence of any patents. Just like how we can't seem to get Mac-quality hardware from other manufacturers even at Apple's (somewhat ridiculous) price points (and I say that as someone who doesn't like Apple for its proprietary practices). So the chances that someone is going to deliver a similar quality product, or arguably we'd require something even better (less bulky), in the hope of some gray-zone playbook to work out, don't seem great.
> We measure the actual symbol distribution of 29 ternary LLM models and find that zeros account for up to 51.5% of all weights. Motivated by this finding, we introduce BITCOS, a simple distribution-adaptive layout
I honestly assumed that's how they already work. I have to admit that I even explained it like that to a friend. Why on earth wouldn't you design it like that from the start (talking about the adaptive, not the measure part; just sacrifice a few bits to clarify your encoding and save a ton of bits)?
In this case however, many of the patents have a hardware element to them. Gaze+tap is powered by an inside camera to track eye movements and a Lidar sensor at the bottom of the rim for motion detection. Not sure your playbook would work out there.
afaik the AVP has a dedicated Lidar sensor for gesture detection. I'd say Apple has managed to patent more basic stuff than that (pinch-to-zoom, rounded corners,...)
The thing is, the AVP has a ton of quality-of-life features, eg relating to gestures, and most of them locked behind patents. Looks like we'll have to wait 20 years before other devices will be able to match it (or whatever the expiry date is on those patents).
Being able to comfortably sit back / lie down and work is an interesting proposition. The Apple Vision Pro is too bulky and expensive to carry that whole category on its own, they need an 'Android equivalent'. But I think Apple messed this up by creating an absolute patent minefield (some 5k patents for AVP features iirc).
With formalized math, you only need to validate the problem statement (in theory, in practice agents have already managed to exploit Lean compiler bugs, but the incidence of those should decrease enough to be practically lusable for 'blind' validation of AI proofs in the foreseeable future).
I remember first hearing about NMS' concept and wishing that they would take the general idea but downsize it by at least 10 orders of magnitude. A universe with 1-100 millions of planets would still feel absolutely massive and way more than individual players would be able to exhaust, yet it would allow the creators to do a bit better curation, allow players to actually meet, maybe add more hand-crafted experiences. Basically, give up on technical purity in favor of fun.
I don't think that follows from the published results. Would have been an interesting hypothesis to add though, and quite easy. Just throw the same setup at some benchmarks.
I wasn't aware that we have access to raw reasoning tokens? I thought what you get is a kind of summary. Does the author have some kind of privileged access or was my assumption wrong?
But for the question studied here it probably doesn't matter - overlaps in the publicly available output may be indicative of distillation (or not), regardless of what it is. I would just find it surprising that the Chinese labs would use it so trustingly. The publicly released reasoning trace is the first place where I would suspect some distillation poisoning to be injected.
The article clearly distinguishes between open source and open weights models, and the headline and parts of the article state that it's open source models that are having a moment. But it doesn't list any examples. The only model families explicitly named are open weights only.
Could someone clarify whether there are actual open source models that are competitive with the likes of Gemma (mentioned in the article), or is the headline just wrong?
Frankly, that's the one part I would have done differently. Mixing font sizes rarely looks good, but that's just my personal opinion. If you want to visually indicate intensity, I'd have used color, or rather saturation. Not the text itself, maybe a small pill next to it. But I don't think it's necessary, the order already conveys some sort of ranking, I don't think cardinal information adds much here.
Congratulations! I was going to comment on the scroll field in particular when I saw this. I didn't even realize you had to hand-craft the component, but it's such a nice UI idea in general, the way the scrolling works and how the content above changes.
You must be referring to SEPA Direct Debit, which would be very risky to use to siphon funds. There's an 8 week no-questions-asked refund policy, 13 months for unauthorized transactions, if the payment fails (eg due to insufficient balance), the payee is charged a non-refundable cancelation fee. [0] And that's all separate from any legal troubles for fraudulent charges. I don't know why Germans use PayPal over SEPA, but I'd be surprised if SEPA Direct Debit was the reason.
If you're mad at Bluesky for doing this, I think you're mad at the wrong party. Modifying intent is problematic and can be abused (although I think there's nothing particularly problematic about this particular instance). The real problem is that you have next to no possibility of modifying the behavior. On an OS that respects user freedom, there would by a myriad ways of intercepting what the software is doing to get your desired result. On iOS, you're at the mercy of Apple. I think we should be mad at Apple for normalizing a computing culture that views user freedom as a security risk, potentially something that should be outlawed (thinking of age verification).
It's a noble cause, but there are probably bigger levers to pull than the model size if you care about environmental impact.
If you're running Qwen3.8-27B on energy-efficient hardware like a Mac or a DGX Spark instead of an API (likely running on H100s), I'm sure you're having much more of an impact than you would by switching to, say, a 9B coding-only model on the same hardware. The thing is, I think you won't be able to go orders of magnitude smaller, because a lot of the usefulness of LLMs comes from emergent smartness, and you typically need a minimum amount of complexity to see such emergent phenomena (and I think we're pretty far from understanding this kind of emergence, much further than from the next model generation that annihilates the current one on benchmarks yet again).
> I understand that to mean that they are the same architecture, just one version has a massive training set and the other has a very small subset.
No. It means that the one model has 2.4 trillion parameters while the other has only 27 billion. I don't know the details about their architecture or training, but presumably they used the same or similar training sets for both and a conceptually similar architecture, scaled down. I'd guess they also have some techniques to re-use some of the work done for the big model for the smaller versions (if anyone knows more about this I'd be interested). The architectures cannot be identical by definition because then the parameter count would be the same. Subsetting the data to such narrow fields as you describe could risk losing some edge, there are a lot of emergent capabilities in those models and I don't think that emergence is fully understood yet. There are subject-specific models, but for far broader subject areas than you suggested, like coding or math or prose.
I'm sure composability is possible in principle, I'm just sceptical that it'll be a good long-term solution, for my originally stated reason. It's basically just The Bitter Lesson again, we may gain some short-lived edge by putting more domain knowledge into the algorithm, but ultimately (these days often: surprisingly quickly) it'll be outgunned by something that just leverages raw computation better.
At a more technical level, what do you suggest? Training a small LLM on Python code exclusively? And then one on general CS/algorithms, which you'll also need? I don't think the current transformer architectures would compose as you suggest.