890 karma · joined January 17, 2023
No one said he did.
> That disruption is already coming no matter what.
[citation needed]. Depending on what you mean by "that disruption," I might even be willing to bet against it coming at all.
> He's a fine enough steward of the tech.
He's a manipulative con-man who is mediocre at everything except convincing investors to give him money. If the tech is truly as revolutionary as it's purported to be, he absolutely should not be a "steward of the tech."
Luckily, no. Do you frequently wade into comment threads shitting on others’ statements of their lived experiences?
No, this is a common misconception. Addressing the speaker is part of it but is not sufficient by itself.
People who are quick to claim “ad hominem!” have been getting this wrong basically forever, so please feel free to educate yourself by reading this excellent post: https://laurencetennant.com/bonds/adhominem.html
“It is not a logical fallacy to attack someone; the fallacy comes from assuming that a personal attack is also necessarily an attack on that person's arguments.
Therefore, if you can't demonstrate that your opponent is trying to counter your argument by attacking you, you can't demonstrate that he is resorting to ad hominem.”
Even though OP didn’t provide them, I can think of many supporting examples for their assertion that Bari Weiss and Matt Taibbi are either intentionally operating in bad faith, or stupid, or both. So this does not at all meet the definition of ad hominem.
Put another way: “you’re wrong because you’re stupid” is an ad hominem. “You’re wrong, and I think you’re stupid because [reason]” is not. This holds even if the person making the argument does not explicitly give the reason.
Put another way, there already exist add-in boards like this, and they’re called GPUs.
Meanwhile, the rest of us (well, not me, because I don’t use your garbage product, but lots of others do) have to suffer and have our compute resources used up in the name of “protection.”
https://www.tomshardware.com/pc-components/gpus/oculink-outp...
I apologize, but using multiple GPUs for inference (without any sort of “daisy chaining”) is something that’s been supported in most LLM tooling for a long time.
> Regardless - there's a difference between training and inference.
No one brought up training vs. inference to my knowledge, besides you — I was assuming the machine was for inference, because my experience building a machine like the one I described was in order to do inference. If you want to train models, I know less about that, but I’m pretty sure the tooling does easily support multiple GPUs.
> And pytorch doesn't magically make 5 gpus behave like 1 gpu.
I never said it was magic, I just said it was supported, which it is.
This setup will work for 100B models as well. And yes, the Mac will draw less power, but the Nvidia machine will be many times faster. So depending on your specific Mac and your specific Nvidia setup, the performance per watt will be in the same ballpark. And higher absolute performance is certainly a nice perk.
> You can certainly daisy chain several 3090's together but it doesn't work seamlessly.
Citation needed; there's no "daisy chaining" in the setup I describe, and low level libraries like pytorch as well as higher level tools like Ollama all seamlessly support multiple GPUs.
You would use multiple *90-series GPUs, throttled down in terms of power. Depending on the GPU, the sweet spot is between 225-350W, where for LLM workloads you only lose 5-10% of performance for a ~50% drop in power consumption.
Combined with a workstation (Xeon/Epyc) CPU with lots of PCIe, you can support 6-7 such GPUs (or more, depending on available power). This will blow away the fastest Mac studio, at a comparable performance per watt.
Again, a lot of this has changed, since GPUs and memory are so much more expensive now.
Macs are great for a simpler all in one box with high memory bandwidth and middling-to-decent GPU performance, but they are (or were) absolutely not "untouchable."
I don't understand how this is a response to anything I said.
I'm not sure if this is what the parent meant by "a real modern PC," but it would certainly be 1) faster and 2) vastly cheaper than the Mac. So at minimum, your assertion that it'd be slower is wrong.
Depending on your configuration, you could likely also match the overall power consumption of the Mac as well, though yes, it is easily possible to exceed it. But the most likely way you'd exceed it is with a high-end GPU, which would vastly outperform the (fixed, non-upgradeable) GPU in the Mac.