HNHacker News
TopNewBestAskShowJobs

dust42

745 karma · joined February 7, 2025

submissionscomments
dust42··on Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
There is a project on github named ANEMLL. Was discussed here a month ago, running LLMs on iPhone - https://news.ycombinator.com/item?id=47490070
dust42··on Elevated errors on Claude.ai, API, Claude Code
Well, at least we know by now that Mythos is a mythos.
dust42··on Reallocating $100/Month Claude Code Spend to Zed and OpenRouter
I really love it. The simplicity is key. The first play project I made with it was a public transport map with GTFS data - click on a stop and get the routes and the timetables for the stop and the surrounding ones. I used Qwen3.5-35B on Mac M1 Max with oMLX. It wrote 98% of the code with very little interaction from me. And very useful is the /tree feature to go back in history when the model is on a wrong track or my instructions where not good enough. I usually work in a two path approach: first let the model explore what it needs to fulfill the task and write it into CONTEXT.md (or any other name to your liking). Then restart the session with the CONTEXT.md. That way you are always nicely operating in 5-15k context, i.e. all is very fast. Create an account for pi (or docker) and make sure it can't walk into other directories - it has bash access. Add the browser-tools to the skills and load them when useful: https://github.com/badlogic/pi-skills

No need for database MCP, I use postgres and tell it to use psql.

Occasionally I use prettier to remove indentation - the LLM makes a lot less edit errors that way. Just add the indent back before you commit. Or tell pi to do it.

dust42··on I've sold out
Context: Mario Zechner is the creator of the pi coding harness which powers OpenClaw. OpenClaw is made by Peter Steinberger, a friend of Mario Zechner. Armin is another friend who made public that OpenClaw is based on pi.

Pi itself is a minimalist coding harness with a tiny 1500 token system message and only read, edit and bash as tools. I only discovered it a few weeks ago and it is surprisingly powerful with a local Qwen3.5-35B - especially as it allows to keep the context low.

Mario's blog posts are not easily digestible (imho) until you have read a few of them but they have plenty of profound thinking. His blog is for me the first one in years where I have spent an hour to read several posts.

Mario is deeply rooted in the OSS system and basically that is what he is talking about here in this post. That said, I have no idea what earendil is doing, except that it is based on pi.

Edit: My personal take - "I've sold out" is very much Austrian style because actually it is the opposite. To quote one thing from the post:

"Then Miguel and Nat approached us. Long story short: we sold RoboVM to Xamarin. A short while later Xamarin closed-sourced our open-source RoboVM core, quickly followed by Xamarin selling to Microsoft. Then Microsoft shut down RoboVM immediately.

While there was some monetary gain, everything about this fucking sucked."

So Mario did a lot of vetting to hopefully avoid this from happening again.

dust42··on Artemis II crew see first glimpse of far side of Moon [video]
The social media team.
dust42··on Artemis II crew see first glimpse of far side of Moon [video]
Well, mathematically full moon is only a infinitesimally small split of a second. When I made the comment it was about 12% waxing crescent thus 88% in the dark. And actually darker than a full moon because the earth does not light up the far side.
dust42··on Show HN: Ghost Pepper – Local hold-to-talk speech-to-text for macOS
When you activate it you agree that your voice input is sent to Apple. As far as I understand this project runs fully locally. Up to you to decide for whatever suits your needs best.
dust42··on Artemis II crew see first glimpse of far side of Moon [video]
Yes, and right now is full moon, thus the far side is only illuminated by stars.
dust42··on Artemis II crew see first glimpse of far side of Moon [video]
> It’s approximately the dark side when the moon is full, which happened two days ago.

Who downvotes that? It is true.

Edit: maybe you can illuminate why you downvote?

dust42··on We replaced RAG with a virtual filesystem for our AI documentation assistant
If grep and ls do the trick, then sure you don't need RAG/embeddings. But you also don't need an LLM: a full text search in a database will be a lot more performant, faster and use less resources.
dust42··on Nitrile and latex gloves may cause overestimation of microplastics
So basically the gloves that kitchen staff now must wear means we get an extra dose of micro plastics? Yikes.
dust42··on Hold on to Your Hardware
Just to mention one thing, helium -which is a necessity for chip production- is a byproduct of LNG production. And 20% of that is just gone (Qatar) and the question is how long it will take to get that back. So not only a chip shortage because of AI buying chips in huge volumes but also because production will be hampered.

Tongue in cheek: we urgently need fusion power plants. For the AI and the helium.

dust42··on ‘Energy independence feels practical’: Europeans building mini solar farms
> This is just made up.

Or not. https://www.welt.de/wirtschaft/article152318777/Wassersparen...

Edit: parent changed his answer, I have included it now.

dust42··on ‘Energy independence feels practical’: Europeans building mini solar farms
You can spend every euro or dollar only once. If you consider CO2 emissions a critical problem, then you should spend every single dollar as efficiently as possible. Obviously independence of fossil fuels has a value too, as the current situation in the middle east shows.

It would make much more sense to import (renewable) electricity from Spain to Germany than strawberries.

dust42··on ‘Energy independence feels practical’: Europeans building mini solar farms
I once read an article that in Berlin the sewage system is flushed with fresh water because too many people have installed water saving toilet flushers. So plenty of people bought these water savers and now the price of water has gone up because the water that is directly flushed needs to be paid too.

The 'balcony power stations' are the same thing. They get subsidised, and you even get a fixed kWh price when pushing into the grid.

The problem is that in the end it will become more expensive for everybody because at times you have a surplus driving the whole sale electricity prices into the negative while still paying fixed prices for injection into the grid.

To make this economically viable, you have to have everyone paying spot prices. Everything else is just green ideology driven inefficiency.

Just to make it clear, I think renewables are an important option for the future. But to make them a viable option of the electricity energy mix, supply and demand, storage and grid capacity need to be taken into account.

Last not least, there is plenty of low hanging fruit to drive CO2 emissions down: drive up the truck tolls. Currently you have potatoes farmed in Germany, driven to Poland to get washed, transported to Italy to be converted to french fries and transferred back to Germany into the super markets.

Same goes for home office, during Covid it was possible for many workers to continue with their work. Does an accountant need to drive to an office every day? Nope. How many business trips could be replaced by a video call?

If the CO2 emissions problem is to be solved rather sooner than later, the money has to be spend efficiently as there isn't enough of it.

dust42··on Hyperlinks in terminal emulators
I really think this is a security disaster waiting to happen, landing right in time for all the agentic terminal apps:

  printf '\e]8;;http://evil.com\e\\https://good.com\e]8;;\e\\\n'
The next step would be to embedd a full javascript VM in the terminal and a CSS engine.
dust42··on German police probe student poster slur against Merz
I have to say correctly so. It is a case of "ROMANES EUNT DOMUS" [1]. What is "Lick eggs Merz" supposed to mean? In order to be a proper political message on a proper demonstration of proper school kids, it should at least say who's eggs Merz should lick and why. And clearly hint why that is either good or bad. Which would give Merz the opportunity to reflect and change his behaviour.

[1] From the Life of Brian

dust42··on Did Alibaba just kneecap its powerful Qwen AI team?
Sounds very plausible to me too. Because even if you refocus the business unit it makes no sense to lay off a highly capable team. Finding new people, integrating them into the team - all that costs a lot of time and money and there is no guarantee for success.

Definitely plenty of people further up the corporate ladder were not happy with the success, while the top is likely too far disconnected to understand.

dust42··on Statement on the comments from Secretary of War Pete Hegseth
Cynical or not, I think it was an absolutely brilliant move: "Mass domestic surveillance of Americans constitutes a violation of fundamental rights". I think they placed their bets on Sama signing a contract with the DoD and here we are, one day later the news that OpenAI signed a contract is out. An absolute PR disaster for OpenAI. And an absolute PR victory for Anthropic.

I think OpenAI's IPO will be interesting. Not even the conservative media will be happy about mass surveillance of Americans.

For non-Americans not much change, they don't really care about your rights more than about a pile of dog poo.

dust42··on Statement from Dario Amodei on our discussions with the Department of War
I'd counter that at this level of capital, if the CEO doesn't well align with the capital, then super-control shares will be overpowered by super-lawyers and if there is need some super-donations. OpenAI was a public interest company...
dust42··on Statement from Dario Amodei on our discussions with the Department of War
Exactly. At this level you don't just put out a statement of your personal opinion. This is run through PR and coordinated with the investors. Otherwise the CEO finds himself on the street by tomorrow. Whatever their motives are, it is aligned with VC, because if it is not then the next day there is another CEO. As the parent stated, this is not cynicism. I see this just rather factual, it is simply the laws of money.
dust42··on Men in their 50s may be aging faster due to toxic 'forever chemicals'
Whenever you see a Youtube video from a restaurant kitchen you can almost be sure to see some pans where the teflon has been scrubbed down to the pure metal. Probably not that healthy...
dust42··on Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
I use MLX server directly from the MLX community project (by Apple). 42 tps is with 0-5000 token context. Starts to drop from there, I have never seen 60.

Yesterday I tested the latest llama.cpp and the result is that PP has made a huge jump to 420 tps which is 30% faster than MLX on my M1. TG is now 25 tps which is below MLX but does not degrade much, at 50k context it is still 22-23 tps.

Together with Qwen code CLI llama.cpp does a lot less often re-process the full KV cache. So for now I am switching back to llama.cpp.

It is worth to spend some time with the settings. I am really annoyed by the silly jokes (was it Claude that started this?). You can disable them with customWittyPhrases. Also setting contextWindowSize will make the CLI auto compress, which works really well for me.

And depending on what you do, maybe set privacy.usageStatisticsEnabled to false.

Like Gemini, Qwen CLI supports OpenTelemetry. When I have time I'll have a look why the KV cache gets invalidated.

dust42··on Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
320 tok/s PP and 42 tok/s TG with 4bit quant and MLX. Llama.cpp was half for this model but afaik has improved a few days ago, I haven't yet tested though.

I have tried many tools locally and was never really happy with any. I tried finally Qwen Code CLI assuming that it would run well with a Qwen model and it does. YMMV, I mostly do javascript and Python. Most important setting was to set the max context size, it then auto compacts before reaching it. I run with 65536 but may raise this a bit.

Last not least OpenCode is VC funded, at some point they will have to make money while Gemini CLI / Qwen CLI are not the primary products of the companies but definitely dog-fooded.

dust42··on The path to ubiquitous AI (17k tokens/sec)
From the article:

"Ljubisa Bajic desiged video encoders for Teralogic and Oak Technology before moving over to AMD and rising through the engineering ranks to be the architect and senior manager of the company’s hybrid CPU-GPU chip designs for PCs and servers. Bajic did a one-year stint at Nvidia as s senior architect, bounced back to AMD as a director of integrated circuit design for two years, and then started Tenstorrent."

His wife (COO) worked at Altera, ATI, AMD and Testorrent.

"Drago Ignjatovic, who was a senior design engineer working on AMD APUs and GPUs and took over for Ljubisa Bajic as director of ASIC design when the latter left to start Tenstorrent. Nine months later, Ignjatovic joined Tenstorrent as its vice president of hardware engineering, and he started Taalas with the Bajices as the startup’s chief technology officer."

Not a youngster gang...

dust42··on Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
I am actually doing now a good part of dev with Qwen3-Coder-Next on an M1 64GB with Qwen Code CLI (a fork of Gemini CLI). I very much like

  a) to have an idea how much tokens I use and 
  b) be independent of VC financed token machines and 
  c) I can use it on a plane/train
Also I never have to wait in a queue, nor will I be told to wait for a few hours. And I get many answers in a second.

I don't do full vibe coding with a dozen agents though. I read all the code it produces and guide it where necessary.

Last not least, at some point the VC funded party will be over and when this happens one better knows how to be highly efficient in AI token use.

dust42··on The path to ubiquitous AI (17k tokens/sec)
I haven't found any end-to-end voice chat models useful. I had much better results with separate STT-LLM-TTS. One big problem is the turn detection and having inference with 150-200ms latency would allow for a whole new level of quality. I would just use it with a prompt: "You think the user is finished talking?" and then push it to a larger model. The AI should reply within the ballpark of 600ms-1000ms. Faster is often irritating, slower will make the user to start talking again.
dust42··on The path to ubiquitous AI (17k tokens/sec)
Low latency inference is very useful in voice-to-voice applications. You say it is a waste of power but at least their claim is that it is 10x more efficient. We'll see but if it works out it will definitely find its applications.
dust42··on The path to ubiquitous AI (17k tokens/sec)
> Don’t forget that the 8B model requires 10 of said chips to run.

Are you sure about that? If true it would definitely make it look a lot less interesting.

dust42··on The path to ubiquitous AI (17k tokens/sec)
This is not a general purpose chip but specialized for high speed, low latency inference with small context. But it is potentially a lot cheaper than Nvidia for those purposes.

Tech summary:

  - 15k tok/sec on 8B dense 3bit quant (llama 3.1) 
  - limited KV cache
  - 880mm^2 die, TSMC 6nm, 53B transistors
  - presumably 200W per chip
  - 20x cheaper to produce
  - 10x less energy per token for inference
  - max context size: flexible
  - mid-sized thinking model upcoming this spring on same hardware
  - next hardware supposed to be FP4 
  - a frontier LLM planned within twelve months
This is all from their website, I am not affiliated. The founders have 25 years of career across AMD, Nvidia and others, $200M VC so far.

Certainly interesting for very low latency applications which need < 10k tokens context. If they deliver in spring, they will likely be flooded with VC money.

Not exactly a competitor for Nvidia but probably for 5-10% of the market.

Back of napkin, the cost for 1mm^2 of 6nm wafer is ~$0.20. So 1B parameters need about $20 of die. The larger the die size, the lower the yield. Supposedly the inference speed remains almost the same with larger models.

Interview with the founders: https://www.nextplatform.com/2026/02/19/taalas-etches-ai-mod...

← PreviousPage 2 of 6Next →