I'm not saying that it can't run in Linux, I'm saying there is no native binary for Linux.
They have bash scripts that starts the windows executables in wine.
You can see that when you read the scripts or in htop.
375 karma · joined December 5, 2022
I'm not saying that it can't run in Linux, I'm saying there is no native binary for Linux.
They have bash scripts that starts the windows executables in wine.
You can see that when you read the scripts or in htop.
Graphics cards with decent amount of memory are still massively overpriced (even used), big, noisy and draw a lot of energy.
If you go this route, you’ve got to build out your own stack for security, global delivery, databases, storage, orchestration, networking ... the whole deal. That means juggling a bunch of different tools, patching stuff, fixing breakage at 3 a.m., and scaling it all when things grow. Pretty soon you need way more engineers, and the “cheap” servers don’t feel so cheap anymore.
If you need multiple devices, as mentioned in the article, you can even stack the laptops and build a small tower of "MiniPCs" all with different purposes.
Another advantage is that they already come with a built-in screen and keyboard allowing for quick debugging, without needing to connect external peripherals.
So this is how it often is in practice. Then there is the possibility to pin down every version, and also some large language models support temperature 0. This is more in the realm of determinism.
You can print the time, cpu and memory usage per request and build metrics based on these outputs. This can be done in any major cloud, with open source tooling and even with simple linux coretuils. One tool to visualize such metrics is Grafana.
But only Meta was kind of not so smart to publicly admit it.
I would like to have a detailed total cost of ownership calculation. Of course, renting hardware - let's say in Hetzner or some other cheap provider - is always cheaper than AWS or any other major cloud provider in simple 1:1 pricing comparison. So that is nothing new. But the pure CPU/RAM/SSD costs is just a fraction of what it really costs to run such infrastructure.