HNHacker News
TopNewBestAskShowJobs

threeducks

420 karma · joined January 10, 2025

submissionscomments
threeducks··on Claude Opus 4.5
Good observation!

56.37 tps / 19.34 tps ≈ 2.9

This explains why Opus 4.1 is 3 times the price of Opus 4.5.

threeducks··on Claude Opus 4.5

    > (~1TB / SSD_read_speed + computation_time_per_chunk_in_RAM) = a few minutes per ~word or punctuation.
 
You have to divide SSD read speed by the size of the active parameters (~16GB at 4 bit quantization) instead of the entire model size. If you are lucky, you might get around one token per second with speculative decoding, but I agree with the general point that it will be very slow.
threeducks··on Claude Opus 4.5
Rough ballpark estimate:

- Amazon Bedrock serves Claude Opus 4.5 at 57.37 tokens per second: https://openrouter.ai/anthropic/claude-opus-4.5

- Amazon Bedrock serves gpt-oss-120b at 1748 tokens per second: https://openrouter.ai/openai/gpt-oss-120b

- gpt-oss-120b has 5.1B active parameters at approximately 4 bits per parameter: https://huggingface.co/openai/gpt-oss-120b

To generate one token, all active parameters must pass from memory to the processor (disregarding tricks like speculative decoding)

Multiplying 1748 tokens per second with the 5.1B parameters and 4 bits per parameter gives us a memory bandwidth of 4457 GB/sec (probably more, since small models are more difficult to optimize).

If we divide the memory bandwidth by the 57.37 tokens per second for Claude Opus 4.5, we get about 80 GB of active parameters.

With speculative decoding, the numbers might change by maybe a factor of two or so. One could test this by measuring whether it is faster to generate predictable text.

Of course, this does not tell us anything about the number of total parameters. The ratio of total parameters to active parameters can vary wildly from around 10 to over 30:

    120 : 5.1 for gpt-oss-120b
    30 : 3 for Qwen3-30B-A3B
    1000 : 32 for Kimi K2
    671 : 37 for DeepSeek V3
Even with the lower bound of 10, you'd have about 800 GB of total parameters, which does not fit into the 512 GB RAM of the M3 Ultra (you could chain multiple, at the cost of buying multiple).

But you can fit a 3 bit quantization of Kimi K2 Thinking, which is also a great model. HuggingFace has a nice table of quantization vs required memory https://huggingface.co/unsloth/Kimi-K2-Thinking-GGUF

threeducks··on CUDA Ontology
Why should the author use the registered trademark symbol?
threeducks··on CUDA Ontology
After being once again frustrated by the CUDA installation experience, I thought that I should give those containers a try. Unfortunately, my computer did not boot anymore after following the installation instructions for the NVIDIA container toolkit as outlined on the NVIDIA website. Reinstalling everything and following the instructions from some random blog post made it work, but I then found that the container with the CUDA version that I needed had been deprecated.

There were other problems, such as the research cluster of my university not having Docker, but that is a different issue.

threeducks··on Comparing Integers and Doubles
What exactly do you say is not commutative? This Wikipedia article claims that at least floating-point addition and multiplication are both commutative:

https://en.wikipedia.org/wiki/Floating-point_arithmetic#Accu...

threeducks··on The internet is no longer a safe haven

    > Fail2ban was struggling to keep up: it ingests the Nginx access.log file to apply its rules but if the files keep on exploding…
    > [...]
    > But I don’t want to fiddle with even more moving components and configuration
You can configure nginx to do rate-limiting directly. Blog post with more details: https://blog.nginx.org/blog/rate-limiting-nginx
threeducks··on Zig / C++ Interop
Oh, that explains the seemingly useless movs. Thank you! I simply thought that it was a bug, since clang did not emit them.
threeducks··on iPhone Pocket
Prices fall when supply exceeds demand.
threeducks··on Zig / C++ Interop
Yes. For example consider this function to add two 2D points, which accepts and returns all variables entirely in xmm registers: https://gcc.godbolt.org/z/hPGKrh6W4 (surprisingly, gcc generates some fairly odd assembly code here)
threeducks··on Firefox Forcing LLM Features
Ah, that went fast. I just got the update. Thanks for pointing it out!
threeducks··on Firefox Forcing LLM Features
I think that (optional) AI features are fine, but the model selection only shows the big providers by default. You can enable local LLM support under about:config -> browser.ml.chat.hideLocalhost (set to false). It will check localhost:8080, which is not a great choice for a fixed port. I'd prefer if I could just point it at any OpenAI-compatible endpoint. Either way, seems to work well when serving a local model with llama.cpp.
threeducks··on Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
> but we literally have no way to prove they're right

Of course we do. Just run a benchmark with Cerebras/Groq and compare to the results produced in a trusted environment. If the scores are equal, the model is is either unquantized, or quantized so well that we can not tell, in which case it does not matter.

For example, here is a comparison of different providers for gpt-oss-120b, with differences of over 10 % for best and worst provider.

https://artificialanalysis.ai/models/gpt-oss-120b/providers#...

threeducks··on We built a cloud GPU notebook that boots in seconds
It's the OS. About 10 years ago, I had an Asus EeePC, which was an underpowered piece of trash with a 32 bit Intel Atom CPU, but it cold-booted in less than 3 seconds. And by "booted", I mean completely booted, i.e. not like Android, where you have to wait a few more minutes until all the background services settle and the UI stops lagging.

Unfortunately, the MeGoo OS was discontinued shortly after. https://en.wikipedia.org/wiki/MeeGo

threeducks··on I was right about dishwasher pods and now I can prove it [video]
While attempting to write my own script, I found that there are many websites which offer YouTube summaries, which are probably an easier solution. For example (not affiliated) https://www.easemate.ai/video-summary It even allows you to ask questions about the transcript.

I also found a Python library for fetching YouTube video transcripts, but some issue mentioned that they got banned, so out of caution, I implemented my summary script as a JavaScript bookmarklet instead. It will probably break on the next YouTube update, so I am not sure how useful it is. Also, you have to set your own API key (and maybe URL). I used Groq (not to be confused with Grok), because it is free and very fast.

    javascript:(function(){
    var GROQ_API_KEY = "YOUR_API_KEY_HERE";

    var btn = [...document.querySelectorAll('button')].find(b => b.textContent.trim() === 'Show transcript');
    btn.click();

    function checkTranscriptAvailable(){
        var transcript = document.querySelector('[target-id="engagement-panel-searchable-transcript"]').innerText;
        console.log("transcript:", transcript.slice(0, 50));
        var length = transcript.replace(/\s/g, '').length;
        if (length > 100){
            fetch("https://api.groq.com/openai/v1/chat/completions", {
              method: "POST",
              headers: {
                "Authorization": "Bearer " + GROQ_API_KEY,
                "Content-Type": "application/json"
              },
              body: JSON.stringify({
                "model": "openai/gpt-oss-120b",
                "messages": [
                  {
                    "role": "user",
                    "content": [
                      {
                        "type": "text",
                        "text": "Briefly summarize this transcript:\n\n" + transcript,
                      },
                    ]
                  }
                ]
              })
            })
                .then(res => res.json())
                .then(data => alert(data.choices[0].message.content))
                .catch(err => alert(err));
        }else{
            setTimeout(checkTranscriptAvailable, 1000);
        }
    };

    checkTranscriptAvailable();

    })();
threeducks··on Server DRAM prices surge 50% as AI-induced memory shortage hits hyperscalers
536 € seems expensive for March 2024, but either way, the price dropped a lot over the last one and a half years, only to surge in the last two months.
threeducks··on From 400 Mbps to 1.7 Gbps: A WiFi 7 Debugging Journey
A few things come to mind:

- Games (400GB for Ark, 235GB for Call of Duty, 190GB for God of War)

- LLMs (e.g. DeepSeek-V3.2-Exp at 690GB or Kimi-K2 at 1030GB unquantized)

- Blockchains (Bitcoin blockchain approaching 700GB)

- Deep learning datasets (1.1PB for Anna's Archive, 240TB for LAION-5B at low resolution)

- Backups

- Online video processing/storage

- Piracy (Torrenting)

Of course you can download those things on a slower connection, but I imagine that it would be a lot nicer if it went faster.

threeducks··on Debug like a boss: 10 debugging hacks for developers, quality engineers, testers
Yea, it's full of AI slop like "It's not X, it's Y!":

> Half the bugs you chase aren’t in your code. They’re in your head.

> You’re not asking for their input, necessarily. You’re asking to have someone listen to you so you can think straight.

> The truth is in the path, not the punchline.

> The log is not your diary. It’s your surveillance system.

> Debugging isn’t just thinking. It’s re-thinking

> These 10 tips aren’t a checklist: they are a starter kit.

> Logs are your sidekick, not your saviour.

threeducks··on Uv is the best thing to happen to the Python ecosystem in a decade
A malicious server could detect whether the user is actually running "curl | sh" instead of just "curl" and only serve a malicious shell script when the code is executed blindly. See this thread for reference: https://news.ycombinator.com/item?id=17636032
threeducks··on D2: Diagram Scripting Language
That's okay. The models already know how to do operator overloading. RLHF will give them the common sense to not use it, unless explicitly requested.
threeducks··on D2: Diagram Scripting Language
If you are okay with the edges going from right to left instead of left to right, you can do it like this in Python:

    def main():
        a, b, c = Node("a"), Node("b"), Node("c")

        edges = [
            a <- b | "first edge",
            b <- c | "second edge",
            c <- a | "third edge",
        ]

        print("Edges:")
        for edge in edges:
            print(edge)

    class Node:
        def __init__(self, name):
            self.name = name

        def __lt__(self, other):
            return Edge(self, other)

        def __neg__(self):
            return self

        def __or__(self, label):
            self.label = label
            return self

    class Edge:
        def __init__(self, a, b):
            self.a = a
            self.b = b

        def __repr__(self):
            return f"{self.a.name} <- {self.b.name}: {self.b.label}"

    main()
It works like this:

The '-' sign is interpreted as a unary minus, which is simply discarded.

The '<' symbol is handled by the '__lt__' overload of the Node class.

The '|' operator has precedence over '<' in Python, so the edge label is stored in the right-hand side node.

That being said, I will haunt you if you actually use this in production.

threeducks··on NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
More precisely, the RTX 5090 has a memory bandwidth of 1792 GB/s, while the DGX Spark only has 273 GB/s, which is about 1/6.5.

For inference, the DGX Spark does not look like a good choice, as there are cheaper alternatives with better performance.

threeducks··on LLMs are mortally terrified of exceptions
Or -Inf, depending on the sign of the zero, which might catch some programmers by surprise, but is of course the correct thing to do.
threeducks··on Fire destroys S. Korean government's cloud storage system, no backups available
This reads very similar to the Turbo Encabulator video.
threeducks··on Solveit – A course and platform for solving problems with code
That testimonial page looks super fake. Why does the same Mathew Miller have 7 testimonials? Similarly, Pol Alvarez Vecino, Pierre Porcher and Duane Milne all have 6 testimonials. And almost all other users also have multiple testimonials. This can't be organic.
threeducks··on Solveit – A course and platform for solving problems with code
@dang Looks like astroturfing. There are several comments by users with very little history, all mentioning that they took the course before.

https://news.ycombinator.com/item?id=45456502 (0 karma)

https://news.ycombinator.com/item?id=45460800 (1 karma)

https://news.ycombinator.com/item?id=45456941 (4 karma)

https://news.ycombinator.com/item?id=45459683 (3 karma)

https://news.ycombinator.com/item?id=45460321 (1 karma)

threeducks··on How AWS S3 serves 1 petabyte per second on top of slow HDDs
A certain data center in Utah might top that, assuming that they have upgraded their hardware since 2013.

https://www.forbes.com/sites/kashmirhill/2013/07/24/blueprin...

threeducks··on Qwen3-Omni: Native Omni AI model for text, image and video
> So did you run the model offline on your own computer and get realtime audio?

At the top of the README of the GitHub repository, there are a few links to demos where you can try the model.

> It seems it needs around a $2,500 GPU

You can get a used RTX 3090 for about $700, which has the same amount of VRAM as the RTX 4090 in your ChatGPT response.

But as far as I can tell, quantized inference implementations for this model do not exist yet.

threeducks··on OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems
Last quarter, NVIDIA reported data-center revenue of $30.8 billion and gaming revenue of only $3.3 billion.

https://nvidianews.nvidia.com/news/nvidia-announces-financia...

This also explains why NVIDIA will not sell high VRAM consumer GPUs: it would cannibalize on their exorbitant data-center profits.

threeducks··on WASM 3.0 Completed
> Just might not be able to save...

Oops, I did not read that before going ham in the editor. It seems that the files are stored inside the emscripten file system, so they are not lost. I could download my exported 'test.stl' with the following JavaScript code:

    var data = FS.readFile('test.stl');
    var blob = new Blob([data], { type: 'application/octet-stream' });
    var a = document.createElement('a');
    a.href = URL.createObjectURL(blob);
    a.download = 'test.stl';
    a.click();
← PreviousPage 2 of 5Next →