400 karma · joined January 14, 2024
ROCm promises a 30-50% prompt processing speedup. This is REALLY important for my workflow so I've been trying to get this shit to work for months. But no release before v10 worked well enough with any engine for it to matter.
The llama.cpp release binaries for ROCm (10) FINALLY work on gfx1501 and its relatives (with the correct shell variables), but the prompt processing boost doesn't materialize and the token generation speed decreases.
There continues to be a chronic problem across all engines with the ROCm integration for UMA devices. The good news is that some improvements have been made to that end for Vulkan, so more recent llama.cpp Vulkan binaries are now faster.
This framing is confusing because OPcache has for the past 15 years taken care of exactly this bootstrapping you're worried about. In a competing share-nothing setup you merely come in at a tie for performance (barring first run of `require`). I do not see 10-50ms latencies in any of my deploys which is why I'm skeptical.
That said, the big sell of your project for me might be memory management - less memory means more workers which is always a good thing if it comes at no cost to latency.
What is the setup/benchmark that caused you to see this?
You're probably asking if they're able contribute anymore?
This is a very easy sell for me because everything these days is doing the infuriating gray text on gray background.
> Neobrutalism is a mix of regular brutalism in web design and more modern typography, illustration, and animation standards. Neobrutalism refuses the usual components of UX-UI design and embraces uncomfortable design elements, and it is more fearless to use distinctive color palettes.
Bad managers hollow out companies. It's quite devastating.
EDIT: To be clear, my employer still uses CF and I don't see any evidence that it's gotten worse.
I can really only remember one catastrophic outage. It involved some part of CF that had no redundancy, and the incident was followed by a pretty whiny blog about a data center.
That's pretty much it. There were other minor outages but I'm not aware that it affected business.
Despite the disclaimer I think the author should explain precisely how "AI product mindset" or whatever is actually translating to bad outcomes.
By contrast, several US states like Vermont and Alaska have GDP in the double digit billions, while others like South Dakota and Nebraska shrunk.
At the heart of it, self-hosting liberates your use cases from all the horribly opaque configuration, shadow prompting, etc. And local models are only getting better and more diverse every month.
Just be sure to name the members of Soundgarden on the paper.
Was NOT expecting a happy ending.
I don't know if the part of MSI Center with the pipe vulnerability is automatically installed on desktops but this is the terribly written software that you need to turn off all the obnoxious lights on your MB and DRAM.