jononor

The cards are also coming... Some of the very first are already out - but it will take a couple of years before they hit their stride in full. And cards will likely be primarily for Chinese companies initially, and then go for exports later. So it will go under the radar for many western techies for a while. Seeing what China has managed with PCB production, low to mid-end semiconductors, phone manufacturing and electric I would hesitate to bet against them.

Nvidia is now lobbying to remove the export ban, to try to reduce the incentives for developing their own high-end LLM training/serving (and cash in on current demand).

Even in a fine tuning scenario, the preferred form for making modifications is the weights plus the training pipelines, including evaluation protocols, tooling, etc.

Not comparable today. But in 10 or 20 years the situation might be considerably different. Hopefully closer to a everyone-can-build-using-hardware-they-already-have.

Does anyone have a good article on the topic? This one seems rather LLM generated. Not sure the specifics can be trusted... Overall thesis is maybe right, but would like to understand this in a bit more detail.

Capability per GB and per watt has also been going up lot. This will continue in the future as well (not necessary as the same rate as last years). But enough that I think Opus 4.8 level is reachable on consumer PCs within 10 years from its release. Say at the price point of 2000 USD in 2025 dollars.

Yeah China has a huge (and growing) advantage in power generation. And they have been looking to break into high-end chip manufacturing. The latter has high cost of entry and needs large scale to become viable. AI inference on own hardware would allow them to bootstrap the chip demand. Both for memory and accelerators.

Writing a Python extension would a good way to dip your toes into Rust, and also add a useful skill to Python programming (writing performant extensions). PyO3 is the main project, and the topic has been covered in several talks at Python conferences (check Youtube).

There are no limits to what you can imagine yourself hearing... And appreciation is a extremely complex subjective feeling, one could argue most of that is way beyond audio and acoustics, and even beyond psychacoustics into plain psychology. When talking about the sensation, it is hard to differentiate between "really hear" versus "tricking oneself" - for most intents and purposes this is one and the same..

On the other hand, one can conduct blind tests that show that many phenomena cannot be reliably differentiated by a listener. Different listeners also have different levels of ability to discriminate sounds and sound quality. So when testing audio equipment using listening tests, one needs to consider the panel of listeners that one uses. Typically there are trained panels of listeners (who have received basic training and shown statistically an ok ability to discriminate) and "consumer" panels which are just random people off the street. The two groups will give very different ratings -especially for "medium" sound quality equipment.

For info on the former, see any decent book on psychoacoustics. And for the latter see for example Sensory Evaluation of Sound (Nick Zacharov et al).

Or "this code (which you happened to write) is bad" vs "you are a shitty programmer".

I believe the proposed system is to run the containers on dedicated rented server(s). Instead of having the containers/VMs share underlying CPU/RAM with others.

Also, many of the successful European companies are bought up by US companies.

GB200 and GB300 are decent for training LLMs? H100 can be used also? Though the current European deployments are rather small, even the biggest are just some thousand GPUs.

EU already has supercomputers and data centers. Some examples: https://www.eurohpc-ju.europa.eu/supercomputers/our-supercom...

The goal is to triple the number of data centers over the next years. It is not like there is nothing going on...

You also need a shitton of data. People are saying that synthetic data is widely used, but pretty sure that is on top of the organic data. Both for pretraining and posttraining. That said, I do think we will see more open and collaborative approaches over time.

Prior to AI making a PR involved considerable effort from a human. So the default position for many open source projects was that it deserved some level of attention for the effort. Even if many projects in practice would struggle to review every PR. But with AI tools this dynamic has shifted dramatically - many PRs have basically zero effort been put into it. Additionally there are many more of them, and often way bigger also.

Many people in LocalLLaMA Reddit community has been reporting the same, that 3.5 122B-A10B is on par or slightly better. And a 3.6 or 3.7 od the 122B is one of the models people want to see the most.

Not only that, but such a node is fully booked with existing orders, many which are long term commitments with penalties if they fail to deliver.

Dual 5060ti 16gb does over 100 tok/s on 35B A3B. Even with PCIE Gen 4 x4, which quite a lot of motherboards can do. Though Gen 4 x8 or Gen 5 x4 is slightly faster. Misc working notes on this hardware combo here, https://github.com/jonnor/embeddedml/tree/master/handson/mic...

Dual color filaments exist, and they do not mix at all... It gives the objects a nice transition when rotated. But indicates that color mixing in the nozzle is probably pretty difficult?

That OpenAI was in the wrong when they ignored everyone copyright, does not make it right to ignore their ToU. If a one wants IP and rule of law (incl contracts) to be respected, one should not violate others rights when it is convenient.

On a more risk-strategy level there is the size of their legal team, general endowment, and supplier and political connections to consider.

Everyone is free to ignore their ToU, but I can understand why a company would avoid it...

A 10'000 hour entry fee does rule out a fair bunch of people though, in practice. While there are few artificial barriers to learning to code, there still are some natural ones, like time.

I do not know which is easier. I am not sure that is even well established in research for generative text tasks whether a translation-first or native-language-first is the most sample efficient?

But for a national lab I think it is money well spent to figure out the possibilities and limitations of a native-language LLMs for languages with order of 5M-10M speakers.

These models will never compete with frontier models and do not need to - it is about hitting a good-enough, not being the best. Behind the frontier, getting to a certain performance level, is getting easier over time - both sample and compute efficiency is going up.

Furthermore one can reuse investments in data (both agreements, infrastructure and datasets), compute (GPUs, servers) and know-how (training scripts, experienced engineers).

It would require an investment, but those will pay dividends later, as it becomes easier to train LLMs on/for Norwegian. If we need to translate everything to English we might as well just drop using Norwegian altogether. Practically everyone speaks English fluently already...

WebSerial in Firefox?! Finally! One of the very few things I use chrome for.

Yeqh that is a challenge. DDR5 and LPDDR5X are both manufacturable with DUV. So let's hope they still get access to that...

As a precondition I think we have to assume that the person in question 1) wants to learn and 2) is smart enough to absorb new info and apply it and 3) reflects enough to adjust their approach when hitting bottlenecks or making mistakes 4) has a drive to create. Without these, self driven learning is not viable - and that has very little to do with AI.

For such a person, I believe AI can be very empowering for learning. Like Google, wikipedia and stack overflow, Arxiv before it - AI tools give access to a lot of information. It allows to quickly dig deep into any topic you can imagine. And yes, the quality is variable - so one needs to find ways to filter and synthesize from imperfect info. But that was also the case before. Furthermore AI tools can be used to find holes in arguments or a paper. And by coding one can use it to test out things in practice. These are also powerful (albeit imperfect) learning tools. But they will not apply themselves.

You are correct that bandwidth requirements depends a lot on the exact workload. And that in specific cases, it might be doable to have AM5 for multiple RTX6000Pro. The parent mentioned workloads that are general, and broader than inference-only. In that case I would consider spending a bit extra on the motherboard to ensure that PCIE bandwidth is not an issue.

Foe multi GPU make sure you have enough PCIE lanes! That rules out consumer grade sockets like AM5, you would need Threadripper or EPYC.

There are likely _many_ paths to sustainable business models based on AI tech, that will come to fruition over the next decades. However whether they might not be as profitable as OpenAI and Anthropic are gambling on, is more uncertain.