3,333 karma · joined January 29, 2016
[0] https://hexdocs.pm/usage_rules/readme.html
[1] https://github.com/phoenixframework/phoenix/tree/main/usage-...
https://ground.news/article/judge-rejects-unsealing-ghislain...
> Native MXFP4 quantization: The models are trained with native MXFP4 precision for the MoE layer, making gpt-oss-120b run on a single 80GB GPU (like NVIDIA H100 or AMD MI300X) and the gpt-oss-20b model run within 16GB of memory.
If you _could_ fit it within ~60GB VRAM, the variability of the amount of VRAM required for certain context lengths and prompt sizes would OOM pretty quickly.
edit: Ah and MXFP4 in itself is a quantization, just supposedly closer to the original FP16 than the rest with a smaller VRAM requirement.
edit: And yeah, this is all anecdotal for me. No clue how much nicotine you actually intake via these methods.
But please don't start vaping if you don't already smoke. There's also the discussion of tobacco-less nicotine pouches that have hit the scene; on average they have higher nicotine content than regular tobacco pouches. And the same issue applies, as it's also acceptable to partake of them anywhere at anytime.
Of course, the line will always be pushed back as frontier models incrementally improve, but the quality is night and day between these open models consumers can feasibly run versus even the cheaper frontier models.
That said, I too have no interest in this if local models aren't supported and hope that's down the pipeline just so I can try tinkering with it. Though it looks like it utilizes multiple models for various tasks (planner, programmer, reviewer, router, and summarizer) so that only adds to the difficulty of the VRAM bottleneck if you'd like to load different models per task. So I think it makes sense for them to focus on just Claude for now to prove the concept.
edit: I personally use Qwen3 Coder 30B 4bit for both autocomplete and talking to an agent, and switch to a frontier model for the agent when Qwen3 starts running in circles.
https://grapheneos.org/articles/attestation-compatibility-gu...
The article would actually be interesting if they tried either of those with the servers they found.
edit: The only leaking information in this case are what operations you'd like to perform on the encrypted data. e.g., you now know that you've incremented the password by x amount, but you don't know what the plaintext was before, the plaintext after, _or_ infer what the value is by knowing you've modified the encrypted data by x amount.
edit2: Now I think I understand the question. The encryption is technically weakened because you can know what operations are made on the underlying data. Though it's still an advancing field, and there are promising developments with what's called circuit privacy[0] to prevent the server knowing the operations made as well.
edit: This is detailed in the article, but leaving this here for those like me who first jump to the HN comments