Alpha3031

Depends on what you want to do. Some task specific models can be trained with a few ten or hundred thousand training examples so you can use a bigger model to produce synthetic training examples and then fine tune a smaller student model. I think that's the usual process. Whether you'd get acceptable performance this way depends, as mentioned, on what you're trying to do and what you'd consider acceptable.

You said it yourself, model size and hardware. Big models cost more (good optimisation reduces things slightly, but they still need the hardware).

If someone says just the word, it's hard to say if they mean just artificial selection, which can and is done to some extent today via genetic counselling by medical geneticists, coercive selection at a society-wide level advocated for by a certain social movement, and the social movement itself, which certainly held some pseudoscientific beliefs regarding how if we just did this and that, we could achieve genetic perfection and cure all social ills and all that. All of those things get called eugenics.

It's like how there's a difference between the medical study of nutrition and how some social media influencers will tell you if you eat X, Y and Z these are superfoods that will mean you will never get a cold for the rest of your life and it will cure cancer or some other nonsense.

The 13,800 number seem like it would be quoted from their annoucement of the $830 million funding round they had in March.

Here's CNBC's article on it: https://www.cnbc.com/2026/03/30/mistral-ai-paris-data-center...

Not sure if they would have received the full number yet, but it's been a few months so they certainly could have. Bit of a moot point when the comparison was against Poolside's Laguna which isn't really "general" SOTA but SOTA-for-the-size, and Mistral is clearly capable of training 700B or 120B models that are that when released considering they have done that... A 2-3T model is probably possible with the GPUs they have but they would need to spend most of their resources on it, and it's not clear why they would want to.

Yes for NAND, and I suppose nobody is using mechanical hard drives for this.

Didn't the US stop new aid since 2025? Sanctions against Russia sure (at least until they decided the spat with Iran was more important) but I feel like that moral high ground is little enough that proposing to "ahem remove"...

Actually on second thought even if the US is doing everything perfectly re Ukraine and Iran and whatever and general being the paragon of morality it really feels like jimbob45's proposal would be a speedrun of losing any possible moral high ground. The only thing that would do it faster is a first strike, and probably only by a couple of days or so. We're really discussing that as if "ahem removing" the leadership of a P5 (i.e. has enough nukes to end the world) country is a sane and normal thing to suggest?

Besides which, I doubt China has sufficient influence over Iran to restrain them, considering the US "ahem removed" the all the people that could plausibly considered moderates. I wonder how reasonable China would feel if the US really went ahead and "ahem removed" a thousand of the top political leadership. My bet, well, it's certainly possible they are correct on the "wouldn't be able to continue their population repression" bit... considering we'll all be too busy dying of nuclear winter.

Imagine things happening other way around and China deciding to "ahem remove", not even a thousand, maybe a dozen people from the US government. Odds on that not ending in a nuclear exchange, anyone? I would take that bet at basically any odds, only I probably wouldn't be alive to spend it if I won.

Per the OSD definition of source code, "the source code must be the preferred form in which a programmer would modify the program." which means an argument could be made (as charcircuit is making) that the weights, being the preferred form to modify, are the source.

I do prefer open weights as being more precise (like, is it even really software that has source code in the first place?) but I feel like at this point the ship has sailed somewhat (though if this is something you're willing to spend your time arguing then... moral support I guess?)

Jiang et al. (2021) TransGAN: https://dl.acm.org/doi/10.5555/3540261.3541391

In general probably not much of a stretch to get rid of convolutions or recurrence by replacing with attention and see if it works hence the title of the original transformers paper.

Aren't RLVR signals typically based off formal systems not natural phenomena? The formal sciences are certainly useful for producing tools used in natural science, but it's not entirely clear they alone are sufficient to associate text to natural (real-world) phenomena. Honestly, the pretraining and RLHF are probably more tied to the world than RLVR, for all that RLVR might be useful (maybe even more useful) for making the model perform better in certain tasks (such as working with formally defined systems).

Microsoft surely has enough resources to keep their fork going if they choose to, so I see no reason why Google would try that given they would likely end up with less control over the web.

There are a lot of applications like that though, only they're all being rewritten for electron.

It's more typical to use parsecs in practice for astronomy.

Disney is able to jackboot anyone violating their IP because the DMCA imposes liability on big tech companies if they do not comply with a valid request. The comment you initially replied to proposes imposing such liability for fraud also, which you appear to be adamantly opposed to.

I want the mob boss to face legal consequences.

If they choose to get out of the business of facilitating crime because of that then so be it.

Normally, if there is sufficient evidence, both the mob boss and the ground level gangster are liable and criminally responsible for their crimes. You are free to disagree, but I don't see any reason why big tech companies should be completely immune from any and all responsibility for facilitating and taking a cut of the criminal activity taken on their territory.

You are also free to call it censorship, but I am not aware of any jurisdiction where fraud is considered protected speech, so as far as I'm concerned, censor away baby.

That's very interesting. Does that mean you can reduce say, a 30B class Q8 from ~30 GB down to 10 GB or less?

People's right to make stupid decisions end when they reach public roads and/or borrow children to make a point. That's why we have things like driver's licences.

As I understand it, it is the position of several DPAs that denial of access entirely (i.e. "consent or pay") could contravene the "freely given" requirement of GDPR in most cases, though this has thus far not been tested in court.

(see e.g. https://iapp.org/news/a/cjeu-clarifies-cookie-consent-requir... https://www.edpb.europa.eu/news/edpb-consent-or-pay-models-s... )

Lots of species are r-selected, I don't see why that would be considered inefficient. In fact I think there are probably more r-strategists than K-strategists.

A lawsuit is not the same as a criminal prosecution. One is supposed to remedy damages, the other is supposed to be because it is in the public interest that people follow laws and we don't devolve into anarchy (or more dangerously, some might-makes-right society where it is more beneficial to appease the powerful than it is to pursue what is just).

It is possible for a prosecutor to decide it is not in the public interest to persue a prosecution (or that there isn't sufficient evidence to prove a criminal action beyond reasonable doubt), and certainly the victim's opinion could be considered, but ultimately whether criminal charges should be pursued is and ought to be based on a different test to civil matters, one focused on the public interest rather than mere restoration.

Basically, they shot someone in public to promote their cool new gun.

Previous changes to society were also made by people within society so I don't see how that follows.

> escaped the desires of its human creators

I wonder if we will ever solve the corporate-superorganism-alignment problem and why the AI alignment people rarely (as far as I can tell anyway, though I admit I'm not in the field) talk about it considering it (complicated multi-human systems) does seem to be about as good as a model organism as we can get before we actually have AI to align.

As we all know, the US is entirely free of geopolitical interests, which is why there was a US–Japan trade war in the late 80s.

I was under the impression Axios did that kind of shit even before LLMs, though I admit I haven't refreshed my memory on older Axios articles before this statement.

There's an Merriam-Webster article on it: https://www.merriam-webster.com/wordplay/lite-word-history

Well, if you're happy with around (as in within an order of magnitude or two of) 0.1 tokens per second... I believe that's around what people are getting when loading MoE weights from NVMe.

To be fair, the flagship Xiaomi, Vivo, Oppo etc are comparable in price to other flagship devices. You very much pay a premium for the large camera sensors they put in these devices, amongst other things.

I don't think the timeline is plausible for Moonshot to have done any distillation from Fable, which means any distillation training data would have been from Opus.