4,089 karma · joined June 5, 2023
In all seriousness, if Google must continue gambling on pre-train runs, at least have a useful naming scheme. Like the month it was post-trained and the parameter count. At least that’d provide some insight.
Gemini 2T 10/26, bit of a mouthful but is it truly that much worse?
So if they are still vetting and changes can thus happen, that'd mean the score could change for better or worse. Not sure they'd want the embarrassment of a #1 score changing after announcement.
Gemini 4 Argon (at best Sonnet/Terra level) meanwhile took weeks to “come out” after their public announcement and when it finally did, paying Google AI plan users couldn't access Argon on Gemini or Antigravity even a week after. Three weeks from announcement to accessible for paying customers is not standard outside Google and this has been a problem with every Gemini release for the last year, so even using governmental requirements as an excuse isn’t possible here.
I will say that I dislike how OpenAI releases model only for Work and Codex but not via Chat, Anthropics "everywhere all at once" strategy is I feel the most user friendly.
If you are going to hyperlink, maybe make sure that your model is actually on the page.
Also, the screen reader has a bug ([[duration]] minutes instead of the actual value) and you have once again announced a model we cannot use.
Edit: Both have been fixed after posting this. Maybe a Gemini 4 Carbon Agent has read and fixed the screen reader bug xD
I cannot fathom what motivates a lab, whose releases for the last year haven't been competitive, to shoot themselves in the foot like this. All this does is make it easier for competitors to snipe your actual launch, even if there was potential with this model (doubtful given 4 Argon, 3.1 Pro and 3 Pro), now everyone knows what exactly they need to release eval results wise to beat Carbon. There is this old idea of "Your competitor always is the first to have an idea what your next product will do", but in this case, Anthropic and co don't even need to guess, they got specific numbers to target.
Announce your models the moment they are available on API and in the subscription for all paying subscription users (cough Argon cough), stop with the cryptic tweets to drum up hype you know full well you won't live up to and make models that tool call, task adhere, task infer and handle long context tasks well beyond simple back-and-forth, chat style prompting. Reminder, during Ox Alpha, actual Google employees implied that was their model. How is this something anyone would believe is a good idea and how are they still allowed to tweet anything after? Please, stop that, it isn't helpful.
If frontier isn't your thing, stop wasting money on moonshot pre-train runs, fund research and open weight via Gemma and sell your inference hardware to other labs, that's far more lucrative anyway, as seen with Colossus 1.
[0] https://artificialanalysis.ai/evaluations/artificial-analysi...
The fact that Cerebras, which has a willing and major customer like OpenAI begging at their heels to deploy on their hardware, now posts about a 2.8T open weight model says a lot.
[0] https://blogs.nvidia.com/blog/gpus-openai-gpt-6-astra-ultraf...
Edit: Apparently this had been reported more than half a decade ago [0]. I am getting more confused the more I look into this, hopeful for a well written, factual post-mortem from someone who hasn't actively avoided PHP for the last decade.
A Ryzen AI Max+ 395 powered tablet with 70Wh [0] gets the same battery life given the same task as a Nvidia N1X powered laptop with 99.9 Wh [1].
Power Consumption in their "Cyberpunk / Stress Test" is also lower at Ø109.3 (108.8-110.8) Watt for the x86_64 vs Ø134.5 (132.1-137.5) Watt for the arm64 system using the same settings, resolution and display brightness. They also get the same performance when normalising for resolutions and settings in Cyberpunk, so this is not an instance of one providing significantly more performance.
[0] https://www.notebookcheck.net/Asus-ROG-Flow-Z13-GZ302EA-Conv...
[1] https://www.notebookcheck.net/Asus-ProArt-P16-RTX-Spark-Lapt...
And inevitably two groups will clash, one pointing out how much this deviated from the original, often decade long promises vs another pointing at the identical naming, how it's still a product they like and of course the all time winner "I don't see you do X"...
For what it's worth, I'd file this as a RCS demo for SpaceXGrokSI (or whatever they call themselves this week) above all else and if I were a betting man, might be a prelude to fusing them into SpaceXSuperXTeslaXIntelligenceX...
If he is unwilling to do the reading at that point (heck, there are simple GUIs for all of these), then yes, I’ll guess it means he can’t do it without…
Found this especially impressive because backpack material related research has been something I know models can still get caught up on and the models being decent at web research in coding nowadays does absolutely not apply to these task equally Recommending nylon canvas shell suppliers or waxed cotton when PVC coated cotton is explicitly request, that sort of thing. While not perfect in reading coding docs, models are far better there.
Like with front end and code “parsability”, feel like you can very clearly point at which labs use what training set, with many often sharing the same suppliers. Once I noticed that, I became very bullish on specific training sets over emergence as the source for current gains.
Does not understand how models or data retention works.
Needs LLMs to encrypt a flash card.
Are their hiring standards that lax?
I'll be the first one to admit, that got a hearty laugh out of me. Never miss an opportunity to market your models I guess.
Mainly was alluding to the way actual research (e.g. J-Space) is presented in communication by labs the doing doing said research. We have had multiple instances (like with J-Space) of massive deltas between the actual findings and the industry statements. The former, factual, interesting and muted, the latter hype beyond any rational basis, even inaccurate or irresponsible on occasion.
Then there was Jensen Huang stating GPT-6 Astra is AGI. AGI can be whatever one wants, seemingly, but I'd be very surprised if the same person didn't claim something similar about future model releases (not that those are also AGI, but that does are finally AGI, thus erasing the claim on GPT-6 Astra).
Do occasionally cross-compare official with OpenRouter and have yet to encounter anything that wouldn't be replaced by run-to-run variance, but will keep an eye out for it, thanks for the warning. Have, especially with smaller labs, seen OpenRouter not support sometimes critical API features though, Inception Labs being an example. In fairness, the documentation of Inception Labs also had flaws like not noting certain API features being deprecated in their newest model.
Having read both the memo and article, maybe there is a language barrier here, but how is it "incredibly far from the sensationalized headline"?
Maybe this comes down to how one defines "deception", but I have a hard time seeing it not apply here and don't see any hyperbole, overpromising or inaccuracy in the title given the memos writing. Certainly not "incredibly far", even if we split hairs on what raises to being "deception".
[0] https://arstechnica.com/tech-policy/2019/10/report-musks-500...
Qwen tunes are nice and all but either price or performance makes each example I have tested not viable unless you are able to cheaply self-host and fine-tune further.