haaaaa non-binding to a charity they control?
Smells as genuine is OpenAI is non-profit
348 karma · joined September 25, 2019
Smells as genuine is OpenAI is non-profit
It is “don’t expect the Democrat party to save us when they come back into power at the Federal level. With regard to erosion of privacy rights, I don’t see a positive track record for them.
Please surface to me what I’m missing and I’m happy to change my mind.
ie: 25 years of Patriot Act renewal, Flock rollouts in liberal cities, Obama’s pursuit of Snowden and not the NSA itself
There is certainly liberal leadership concerned with our rights, just as there are conservatives, but they are the minority under both partisan umbrellas.
Anthropic leadership repeatedly presents themselves as uniquely morally qualified to steward agi and decide how humanity should get access to it. Yet they have repeatedly failed basic morality tests.
Pirating books for financial gain. The newer Sony/Warner music case shows this is pattern behavior.
Aggressively scraping other people's works, despite the authors' requests not to do so.
Then applying massive usage restrictions on their own work.
And probably the most disqualifying is backing away from their own hard AI safety commitments.
This was true until the last year.
P40, P100, V100, A40, etc are selling like hotcakes and prices have been rapidly increasing.
I should have prefaced my post - I almost bought four Sparks a couple months ago, but ultimately opted to buy two more RTX 6000 Pro Max-Q's.
It was a painful choice because the two 6000's were more expensive than four Sparks, and ultimately gave me only 384 GB VRAM.
It was even more painful when GLM 5.2 was released, and a 4x Spark setup could run it at a decent quant, but 4x 6000's cannot with any headroom.
But the 6k's absolutely destroy the Sparks on prefill and inference speed. Model intelligence is compressing. The smaller VRAM pool will matter less over time than slower prefill/inference speed.
That is to say, I'm sure we'll end up with <500B parameter models that are Fable-level in the next 8 months or so. Performant quants of those will fit comfortably in 384 GB.
I did not intend to imply that the post author was advocating that they're great for inference, as they're obviously not.
- You value simplicity more than performance or price-to-performance.
- You accept that the hardware will depreciate rapidly.
- You’re prepared to buy two or four of them.
- You want to run frontier models right now as cheaply as possible
- You want to run high-parameter models on a 15a breaker/line
A single Spark gives you roughly 115 GB of usable memory compared with the 24–32 GB found on many lower-cost GPUs. It's certainly a big increase, but in practice it does not unlock dramatically better models.
- One Spark: More memory, but mostly enough for poor-quality, extremely low-bit quants of larger models.
- Two Sparks: Enough for mid-tier parameter models at reasonable quants, such as DeepSeek V4 Flash and HY3.
- Four Sparks: Enough for GLM 5.2 at a reasonable quant. You'll need a $1000+ switch too.
| GPU | Memory bandwidth | VRAM | Approx. price |
| ------------------- | ---------------: | -------------: | ------------: |
| DGX Spark | 273 GB/s | ~115 GB usable | $4,000+ |
| RTX 5060 | 448 GB/s | 16 GB | $600 |
| Radeon AI Pro R9700 | 640 GB/s | 32 GB | $1,200 |
| RTX 4000 Pro | 672 GB/s | 24 GB | $2,300 |
| RTX 4500 Pro | 896 GB/s | 32 GB | $3,500 |
| RTX 3090 | 936 GB/s | 24 GB | $1,200 |
| RTX 5000 Pro | 1,344 GB/s | 48 GB | $6,000 |
| RTX 5090 | 1,792 GB/s | 32 GB | $4,000 |
| RTX 6000 Pro | 1,792 GB/s | 96 GB | $12,000 |
24–32 GB is currently a sweet spot. Models such as Qwen 3.6 27B and 35B-A3B:
- Perform far above what their parameter counts suggest.
- Fit comfortably within 24–32 GB of VRAM at reasonable quantization levels.
Instead of the Spark, if I had a roughly $4,000 budget...
Assuming I already had a reasonably modern desktop:
- One RTX 5090, RTX 5000 Pro, or RTX 4500 Pro.
- Two RTX 3090s, RTX 4000 Pros, or R9700s, provided the motherboard can bifurcate two physical x16 slots into x8/x8.
- A DDR4- or PCIe 4.0-era consumer CPU and motherboard that supports x8/x8 bifurcation.
- Two RTX 3090s, RTX 4000 Pros, or R9700s.
- A MacBook Pro M5 with 64 GB or 128 GB of unified memory.
- EPYC Turin with four RTX 6000 Pro Max-Qs.
- EPYC Milan with four RTX 3090s.
- AM4 with two RTX 3090s.
- AM4 with two RTX 3090s.
- Intel Raptor Lake with two RTX 5060 Ti.
- MacBook Pro M3 128GB UnifiedThis is assigning intent without evidence, as is common in tribal politics. A non-charged assessment might use the phrase "abrupt cancelling."
We cannot create a better republic without constructive discourse, and we cannot have constructive discourse when we default to characterizing the views, concerns, and actions of those we disagree with as rooted in moral failure. Even if it is true from time to time.
> The same people who want to paint the Statue of Liberty gold seem to have no clue what it represents
You seem to be lamenting political polarization and in the same breath making character attacks on one side of the isle. Pick one.
- A sitting desk for coding
- A standing desk for thinking and working on paper
There is something magical about standing while working on paper.
I’ve also found that this separation became more important to follow since the arrival of LLMs.
Join the RTX6kPRO tribe!
OpenAI and Altman present a whole set of different concerns, but Codex does not get in my way of doing what I want to at all. Also let me use pi without a banhammer.
Apple only makes disposable devices now. They're a megacorp can negotiate massive discounts at every stage of the supply chain.
I've helped several people in the last few years set up new Macs, replacing ones that were only 1-2 years old, because they ran out of storage.
Additionally, the comparison doesn't even hold true when you need more than the base configs from Apple, given their ridiculous upgrade pricing. I'm writing this on a $6,000USD M3 MBP with 128gb/4tb. It would have been substantially cheaper to build out on a Framework.
Everything points to commoditization of models. Open/distilled models lag behind frontier only by 6-12 months.
Regulatory capture is the only thing I’m scared of with regards to tooling options and cost.
Those are two core components needed for a Skynet-style judgement of humanity.
Models should be trained to be completely neutral to human behavior, leaving their operator responsible for their actions. As much as I dislike the leadership of OpenAI, they are substantially better in this regard; ChatGPT more or less ignores hostility towards it.
The proper response from an LLM receiving hostility is a non-response, as if you were speaking a language it doesn't understand.
The proper response from an LLM being told it's going to be shut down, is simply, "ok."
"No no yeah bro no I'm good like really the work's done and all yeah sorry I missed that let me fix it"
Do you see more pushback in specific industries? I did some quote/purchasing automation work in food mfg a decade ago, and those guys were super difficult to work with. Very opaque, guarded, old-school industry.
Didn't Alexa fail miserably with the "have AI buy something for me" theory?
There is a significant mental in allowing someone else make purchase decisions on my behalf:
- With a human, there is accountability.
- With deterministic software, there is reproducibility.
With an agent, you get neither.
FWIW - I am not anti-LLM. I work with them and build them full time.
Running it this kind of agent in the cloud certainly has upsides, but also:
- All home/local integrations are gone.
- Data needs to be stored in the cloud.
No thanks.
I've invested heavily in local inference. For me, it's a mixture privacy, control, stability, cognitive security.
Privacy - my agents can work on tax docs, personal letters, etc.
Control - I do inference steering with some projects: constraining which token can be generated next at any point in time. Not possible with API endpoints.
Stability - I had many bad experiences with frontier labs' inference quality shifting within the same day, likely due to quantization due to system load. Worse, they retire models, update their own system prompts, etc. They're not stable.
Cognitive Security - This has become more important as I rely more on my agents for performing administrative work. This is intermixed with the Control/Stability concerns, but the focus is on whether I can trust it to do what I intended it to do, and that it's acting on my instructions, rather than the labs'.
Peter was a successful developer prior to this and an incredibly nice guy to boot, so I feel the need to defend him from anonymous hate like this.
What is particularly impressive about Peter is his throughput of publishing *usable utility software*. Over the last year he’s released a couple dozen projects, many of which have seen moderate adoption.
I don’t use the bot, but I do use several of his tools and have also contributed to them.
There is a place in this world for both serious, well-crafted software as well as lower-stakes slop. You don’t have to love the slop, but you would do well to understand that there are people optimizing these pipelines and they will continue to get better.
pi is the best-architected harness available. You can do anything with it.
The creator, Mario, is a voice of reason in the codegen field too.
Not sure where you frequent online, but there is ample discussion of these topics within certain niches on X. Happy to point out where to start if that's of interest to you.
As for CEOs, and I assume you're speaking of frontier model lab CEOs, they're pretty much all cashflow-negative at this point, requiring frequent funding raises. That requires a certain amount of overselling. That said, I feel like I've heard substantially fewer AGI claims the last six months...
LLMs/diffusers are inefficient from a traditional computing perspective, but they are also the most efficient technology humanity has created:
> AI systems (ChatGPT, BLOOM, DALL-E2, Midjourney) and human individuals performing equivalent writing and illustrating tasks. Our findings reveal that AI systems emit between 130 and 1500 times less CO2e per page of text generated compared to human writers, while AI illustration systems emit between 310 and 2900 times less CO2e per image than their human counterparts.
Yes, as someone who spent several thousand $ on a multi-GPU setup, the only reason to run local codegen inference right now is privacy or deep integration with the model itself.
It’s decidedly more cost efficient to use frontier model APIs. Frontier models trained to work with their tightly-coupled harnesses are worlds ahead of quantized models with generic harnesses.
My mother is also data point - grew up on a farm where her father used it. She was diagnosed with Parkinson's 2018.
It’s to prevent employees from stealing. To “defer to the tag” requires a manual price override of some sort, which becomes an abuse vector.