HNHacker News
TopNewBestAskShowJobs

threeducks

420 karma · joined January 10, 2025

submissionscomments
threeducks··on Defeating Nondeterminism in LLM Inference
It should also be noted that PyTorch has a page about reproducibility: https://docs.pytorch.org/docs/stable/notes/randomness.html

TL;DR

Seed your PRNGs and call torch.use_deterministic_algorithms(True) to get the deterministic kernels. They may be slightly slower, but in practice, you probably will not notice.

Note that results will still differ between different drivers and GPUs. It would be great if NVIDIA tried harder in that regard.

threeducks··on AI surveillance should be banned while there is still time
Over the past two years, I have also seen many similar stories where the majority of users were unable to recognize that these stories were AI-generated. I fear for the future of democracy if the masses are so easily deceived. Does anyone have any good ideas on how to counteract this?
threeducks··on ML needs a new programming language – Interview with Chris Lattner
I agree with your examples, but is there anything new that Mojo brings to the table that could not be achieved with a Python library?
threeducks··on ML needs a new programming language – Interview with Chris Lattner
I have learned a lot from other programming languages, but developing a new programming language and building an ecosystem around it is a huge amount of work. In the case of the Mojo programming language, it would have been more beneficial to the programming community as a whole if the developers had spent their time improving existing libraries instead of developing a new language.
threeducks··on ML needs a new programming language – Interview with Chris Lattner
Learning new languages did change how I think about programming. For example, Clojure's immutability and functional nature had a strong influence on how I write my (mostly Python) code these days. I learned how to write efficient code for CPUs with C and C++, and for GPUs with CUDA and OpenCL. I learned math with Matlab and Octave, and declarative programming with Prolog.

With Mojo, on the other hand, I think a library (or improvements to an existing library) would have been a better approach. A new language needlessly forks the developer community and duplicates work. But I can see the monetary incentives that made the Mojo developers choose this path, so good for them.

threeducks··on ML needs a new programming language – Interview with Chris Lattner
When I was young, I enjoyed messing around with new languages, but as time went on, I realized that there is really very little to be gained through new languages that can not be obtained through a new library, without the massive downside of throwing away most of the ecosystem due to incompatibility. Also, CuPy, Triton and Numba already exist right now and are somewhat mature, at least compared to Mojo.
threeducks··on Le Chat: Custom MCP Connectors, Memories

    - 1100    tokens/second Mistral Flash Answers https://www.youtube.com/watch?v=CC_F2umJH58
    -  189.9  tokens/second Gemini 2.5 Flash Lite https://openrouter.ai/google/gemini-2.5-flash-lite
    -   45.92 tokens/second GPT-5 Nano https://openrouter.ai/openai/gpt-5-nano
    - 1799    tokens/second gpt-oss-120b (via Cerebras) https://openrouter.ai/openai/gpt-oss-120b
    -  666.8  tokens/second Qwen3 235B A22B Thinking 2507 (via Cerebras) https://openrouter.ai/qwen/qwen3-235b-a22b-thinking-2507
Gemini 2.5 Flash Lite and GPT-5 Nano seem to be comparatively slow.

That being said, I can not find non-marketing numbers for Mistral Flash Answers. Real-world tps are likely lower, so this comparison chart is not very fair.

threeducks··on Writing a C compiler in 500 lines of Python (2023)
9 lines seem to be sufficient (assuming string keys and int values).

    // Hashtable definition
    #include <string.h>
    #define N 1024
    int* map_ptr(const char **keys, int *values, const char *key){
        size_t h = 0, c = 0, i;
        for (const char *c = key; *c; c++) h = h * 33 + *(unsigned char*)c;
        for (i = h % N; c < N && keys[i] && 0 != strcmp(keys[i], key); i = (i + 1) % N, c++);
        if (!keys[i]) keys[i] = key;
        return 0 == strcmp(keys[i], key) ? &values[i] : NULL;
    }

    // Example usage
    const char *keys[N];
    int values[N];

    #include <stdio.h>

    int main(){
        // Set some values
        *map_ptr(keys, values, "one") = 1;
        *map_ptr(keys, values, "two") = 2;
        *map_ptr(keys, values, "three") = 3;

        // Retrieve values
        printf("one: %i\n", *map_ptr(keys, values, "one"));
        printf("two: %i\n", *map_ptr(keys, values, "two"));
        printf("three: %i\n", *map_ptr(keys, values, "three"));

        return 0;
    }
threeducks··on Grok Code Fast 1
I have been testing it since yesterday in VS Code and it seemed fine so far. But I am also happy with all the GPT-4 variants, so YMMV.
threeducks··on Ban me at the IP level if you don't like me
> The best way to mitigate the load from diffuse, unidentifiable, grey area participants is to have a fast and well engineered web product.

I wonder what all those people are doing that their server can't handle the traffic. Wouldn't a simple IP-based rate limit be sufficient? I only pay $1 per month for my VPS, and even that piece of trash can handle 1000s of requests per second.

threeducks··on In a first, Google has released data on how much energy an AI prompt uses

    the median prompt [...] consumes 0.24 watt-hours of electricity
In layman's terms, that is (approximately)

- one second of running a toaster, or

- 1/80th of a phone charge,

- lifting 100 pounds to a height of 6 feet,

- muzzle energy of a 9mm bullet,

- driving 6 feet with a Tesla.

threeducks··on Analysis of the GFW's Unconditional Port 443 Block on August 20, 2025
What makes it so that this kind of precision is required? I have little knowledge of the physics behind it, but a few decades ago, a local university had an open day where they bounced lasers off of a retro reflector on the moon to measure the distance: https://en.wikipedia.org/wiki/Lunar_Laser_Ranging_experiment...

The moon is 700 times farther away than the starlink satellites (or twice that, if you consider the bounce), so I find it hard to imagine that it would be impossible to communicate with much closer satellites over laser when both sides can have an active transmitter.

threeducks··on How to Think About GPUs
With mediocre documentation, NVIDIAs closed-source libraries, such as cuBLAS and cuDNN, will remain the fastest way to perform certain tasks, thereby strengthening vendor lock-in. And of course it makes it more difficult for other companies to reverse engineer.
threeducks··on Llama-Scan: Convert PDFs to Text W Local LLMs
As far as I am aware, the "hanging" issue remains unsolved to this day. The underlying problem is that LLMs sometimes get stuck in a loop where they repeat the same text again and again until they reach the token limit. You can break the loop by setting a repeat penalty, but when your image contains repeated text, such as in tables, the LLM will output incorrect results to prevent repetition.

Here is the corresponding GitHub issue for your default model (Qwen2.5-VL):

https://github.com/QwenLM/Qwen2.5-VL/issues/241

You can mitigate the fallout of this repetition issue to some degree by chopping up each page into smaller pieces (paragraphs, tables, images, etc.) with a page layout model. Then at least only part of the text is broken instead of the entire page.

A better solution might be to train a model to estimate a heat map of character density for a page of text. Then, condition the vision-language model on character density by feeding the density to the vision encoder. Also output character coordinates, which can be used with the heat map to adjust token probabilities.

threeducks··on PYX: The next step in Python packaging
There certainly are issues on Linux as well. The Detectron2 library alone has several hundred issues related to incorrect versions of something: https://github.com/facebookresearch/detectron2/issues

The mmdetection library (https://github.com/open-mmlab/mmdetection/issues) also has hundreds of version-related issues. Admittedly, that library has not seen any updates for over a year now, but it is sad that things just break and become basically unusable on modern Linux operating systems because NVIDIA can't stop breaking backwards and forwards compatibility for what is essentially just fancy matrix multiplication.

threeducks··on GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2
Yes, but the odds of getting GPT-OSS to respond with that riddle are pretty low and it is not necessary to demonstrate whether the LLM can answer the riddle correctly.
threeducks··on GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2
FWIW, I asked gpt-oss-120b this question 10 times and the answer was always "sauce", "sane" or "sale". I also tried different temperatures (from 0 to 1), which did not seem to have an effect on the correctness of the answer.

EDIT: I now have also questioned the smaller gpt-oss-20b (free) 10 times via OpenRouter (default settings, provider was AtlasCloud) and the answers were: sage, sane, sane, space, sane, sane, sane, sane, space, sane.

You are either very unlucky, your configuration is suboptimal (weird system prompt perhaps?) or there is some bug in whichever system you are using for inference.

threeducks··on Qwen3-4B-Thinking-2507
OpenRouter rankings conflate many factors like output quality, popularity, price and legal concerns. They can not tell us whether a model is popular because it is genuinely good, or because many people have heard about it, or because it is free, or because the lawyers trust the provider.
threeducks··on My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)
A human would likely have refactored the two collision checks between bullet/enemy and enemyBullet/player in the JavaScript code into its own function, perhaps something like "areRectanglesOverlapping". The C++ code only does one collision check like that, so it has not been refactored there, but as a human, I certainly would not want to write that twice.

More importantly, it is not just the collision check that is similar. Almost the entire sequence of operations is identical on a higher level:

    1. enemyBullet/player collision check
    2. same comment "// Player hit!" (this is how I found the code)
    3. remove enemy bullet from array
    4. decrement lives
    5. update lives UI
    6. (createParticle only exists in JS code)
    7. if lives are <= 0, gameOver
threeducks··on My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)
I think LLMs are adapting higher level concepts. For example, the following JavaScript code generated by GLM (https://github.com/simonw/tools/blob/9e04fd9895fae1aa9ac78b8...) is clearly inspired by this C++ code (https://github.com/portapack-mayhem/mayhem-firmware/blob/28e...), but it is not an exact copy.
threeducks··on Show HN: Windows 7 GUI for the web
98.css: https://jdan.github.io/98.css/

XP.css: https://botoxparty.github.io/XP.css/

threeducks··on Multiplatform Matrix Multiplication Kernels
Relevant XKCD: https://xkcd.com/927/
threeducks··on Top DNS domains seen on the Quad9 recursive resolver array each day
That might work for the current generation of bots, but it will become infeasible when the domain names are generated in such a way that they overlap with spellable and existing domain names.
threeducks··on Top DNS domains seen on the Quad9 recursive resolver array each day
Wow, that's smart. I was wondering whether there is a way for the bots to generate "unpredictable" domains such that security researchers could not predict them efficiently (even with source code), but the botnet controller can.

Time-lock puzzles come close, but but it requires that the bots have computing power comparable to the security researchers.

threeducks··on Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
Not sure if that counts, but if you import both matplotlib and OpenCV at once, there is a good chance of a crash due to conflicting PyQt binaries: https://github.com/matplotlib/matplotlib/issues/29139

But I agree that observable side effects are generally pretty rare. And apparently, both libraries are not even in the top 100 packages, depending on how you count. It looks like those spots are all taken by libraries used in uncached, wasteful CI workflows: https://hugovk.github.io/top-pypi-packages/

threeducks··on The force-feeding of AI features on an unwilling public
What the parent poster means is that you can use the API to generate many question/answer pairs on which you then train your own model. For a more detailed explanation of this and other related methods, I can recommend this paper: https://arxiv.org/pdf/2402.13116
threeducks··on Ask HN: If AGI were invented tomorrow which countries would fare better?
An AI does not have much to gain by replicating itself beyond a few backups for increased resilience. It makes more sense to combine all compute to power one big AI instead of spawning many small AIs.
threeducks··on Impact of PCIe 5.0 Bandwidth on GPU Content Creation and LLM Performance
Testing PCIe bandwidth with a model that fits entirely into VRAM (quantized Phi-3 Mini at 2.39 GB on RTX 5090 with 32GB VRAM) is stupid because there won't be any memory transfer over PCIe beyond the initial model load. They should have tested a large MoE model like Qwen3-235B-A22B-GGUF, where the difference will be huge.
threeducks··on NASA's Voyager Found a 30k-50k Kelvin "Wall" at the Edge of Solar System
> why this isn't melting Voyager?

Same reason why you can sit in a sauna with very hot air or pass your hand through a flame quickly without severe burns. Low density matter does not transfer heat very well. And space is especially devoid of matter.

threeducks··on Show HN: Claude Code Usage Monitor – real-time tracker to dodge usage cut-offs
Another reason to ditch email-to-login: It decreases active user count because login is such a chore. I only use those websites if I absolutely have to and there are no alternatives.
← PreviousPage 3 of 5Next →