Imagine not reinventing the wheel for no reason.
But don't worry, I imagine that modern prompting techniques will be used such as "do not make mistakes" so everything will be fine (/s)
13 karma · joined February 7, 2025
It probably needs to store the full F16 version for each model variant, even if the variants share the same base model
Yet it's only useful if one wants to have both the F16 original model plus all the other GGUF quants at a time and doesn't mind spending compute every time they need a specific quant again.
It doesn't do much for a consumer who only needs one or a few specific quants that work on their hardware and I'd wager it's not something that multi billion dollar companies need. At least as far as I'm aware major llm providers aren't using gguf and llama.cpp to serve their inference
SMR drives are now what you find on most consumer drives between with capacities 1<x<8 tb (higher capacities too, but depends on the manufacturer) , they have a CMR area of the platter as a sort of write cache (like slc cache in ssds), while the rest of the platter will be really slow to write to. The write head is wider than the read head, so to overwrite something the drive has to first read and copy somewhere else the data on the track(s) that would be overwritten. This makes whole drive writes really slow and can kill raid 5/6 since resilvers would take very long, possibly even a month, instead of a few days.
Besides the recording technology, the color of the label and the product line name are mostly marketing and won't make too much of a difference for simple usage.
This would be terrible, I thought rust was a safe language...
I know that it talks about only 1 and not 3 points, but it seems close enoguh
doing this should allow you to use as many optimized packages as possible while still being able to install packages not supported by the alhp