HNHacker News
TopNewBestAskShowJobs

mirekrusin

5,755 karma · joined September 4, 2014

https://mirekrusin.com
submissionscomments
mirekrusin··on The Mojo language (by Modular, now Qualcomm) is now open-source
Chris literally was saying it from the very beginning (you can watch early interviews, obviously he was being asked about it), everything that happens with progressive opening is what he said he'll do and he gave rationale for it which is summarized by what parent comment says.
mirekrusin··on Qwen3.8 27B scores 52 on Artificial Analysis
yes, you know, personal use doesn't necessary mean no concurrency.

it's good to play with harness setup where you fan out multiple concurrent branches that share non trivial amount of prefix then reduce their output/summary back into main agent.

ie. instead of serially reading further skills/relevant source files for planning/thinking, you can branch and read them in parallel reusing prefix / or use to to approach request from different angles in parallel - to map-reduce result onto main context of what's actually relevant. branching subagents has benefits of not polluting main context, shared prefix prefill is close to free on a cache hit and with concurrent decoding/continuous batching you can utilize gpu well to get good speedups.

ie. what's relevant is number of active concurrent sequences (and their shape, ie. shared prefix), not so much number of users.

i'm not sure with llama.cpp vs vllm regarding concurrency – llama server has multiple server slots, continuous/dynamic batching enabled by default, prompt caching (also on by default), ram prompt cache, context checkpoints, unified kv buffer across sequences etc. so shouldn't be bad, i guess would be good to actually benchmark. personally i'm happy with llama.cpp.

mirekrusin··on Qwen3.8 27B scores 52 on Artificial Analysis
Dual 4090, getting 85-113 t/s depending on task (draft seems to speed up quite a lot, disproportionately more for content like svg etc):

  ./llama.cpp/llama-server \
        -hf unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL \
        --webui-mcp-proxy \
        --no-mmproj \
        --parallel 1 \
        --kv-unified \
        --flash-attn on \
        --fit off \
        --split-mode tensor \
        -ngl 999 \
        --cache-type-k q8_0 \
        --cache-type-v q8_0 \
        -ub 256 \
        --no-context-shift \
        --host 0.0.0.0 \
        --tools all \
        --jinja \
        --ctx-size 262144 \
        --spec-type draft-mtp \
        --spec-draft-n-max 3 \
        --reasoning on \
        --chat-template-kwargs '{"reasoning_effort":"medium"}' \
        --reasoning-preserve \
        --temp 1.0 \
        --top-p 0.95 \
        --top-k 20 \
        --min-p 0.0 \
        --presence-penalty 0.0 \
        --repeat-penalty 1.0
Use claude/codex/whatever with /goal to optimize params for you.

IMHO draft model support on dense models is great alternative to MoE on GPUs (high bandwidth, less memory) – more intelligence, speed somewhere mid way there which is usually sufficient.

mirekrusin··on Incident with Github.com [resolved]
Why are people complaining? It’s not a bug, it’s a feature.

You have a free bank holiday, go look at green trees, listen to birds, find somebody who's wrong on HN etc.

The only poor souls here are some github devops team (team of copilots?) and those using alternative services.

mirekrusin··on Incident with Github.com
Seriously?
mirekrusin··on Why does Opus 5 feel worse to work with?
Next RL phase is to connect electrodes to human brain when reading generated output to reduce frustration signal.
mirekrusin··on Qwen 3.8 27B
That's a good sign.

Ideal local model would not know stuff like who Britney Spears is, best to leave precious weights for something useful.

Of course the line is very blurry but I'd be perfectly happy with local model that doesn't know anything about history, geopolitics, art or even biology etc. just coding, operating systems etc.

mirekrusin··on Gemini 3.7 Flash
ie. open router fusion [0]

[0] https://openrouter.ai/openrouter/fusion

mirekrusin··on Gemini 3.7 Flash
Just use all of them and synthesize final result, record scoring when synthesizing.

Later you can make decision to drop low performing ones.

mirekrusin··on Grok 4.6 scores 61 on the Artificial Analysis Intelligence Index
Use of official API outside of China is closer to opposite of reality.

Look at ie. OpenRouter you'll see how many providers there are and how much traffic they get.

If the goal was data, opening weights would be the worst available way to achieve it: a cheap, closed API would capture 100% of traffic (ie Anthropic style), weights can only lose share from there.

With open weights it's net loss of active users of your official api – you're loosing users to self hosting and dozens of providers.

The thing is that open weights create permanent exit that closed models do not have, inference is commoditized immediately, people choose open weight models specifically for data privacy (and stuff like soc2/hipaa compliance) and if somebody wants convenience they go to claude/openai and friends anyway.

Also chinese labs are not uniform with their approach just as western labs are not.

mirekrusin··on Grok 4.6 scores 61 on the Artificial Analysis Intelligence Index
Your argument was that releasing models as open weight was allowing them to collect data but reality is that open weight models precisely allow users to completely avoid data collection – something that is not actually possible with closed weight models.

Motivations are more complex than that, if they wanted to achieve that they'd follow approach taken by some western labs to release weaker models only as open keeping frontier behind APIs.

mirekrusin··on Grok 4.6 scores 61 on the Artificial Analysis Intelligence Index
Depends who you ask:

META releases open weight models because selling access to models isn't their business model, they get optimization for free, good press with adoption of their tech etc. They don't want AI to become another iOS, they want to commoditize the layer underneath them.

GOOGLE releases open weight models because they complement the rest – from premium offering to Android/Chrome/Google Cloud etc products. They use it as part of open ecosystem / local / edge / experimentation layer. 400 million downloads with 100k community variants is nice vibrant ecosystem they have and want to have, they can capture value in several places and would prefer if devs standardize on their tooling.

OPEN AI because their PR was shit and it saved them, purely defensive move, which is interesting considering their name and initial goal.

ALIBABA because it tickles them to erase/cap profits in western labs, free optimisation/research, great PR.

DEEPSEEK they want to have day-zero support for all possible hardware platforms, they started it because Liang wanted to do it, had resources from High-Flyer to do it and he said fuck it and did it (zero commercial incentives which is so bizzare that it deserves a movie or something). He wants to do it because his destination is AGI. In that sense he's more original OpenAI than OpenAI.

MISTRAL because they want to be to AI what RedHat is to Linux.

NVIDIA because they sell GPUs.

...the list is long, there are few dozens of companies including Microsoft, IBM, AI2, Databricks, Snowflake, xAI, EleutherAI, Hugging Face etc. that release open weight models, there are thousands of models and tens of thousands of fine tunes across across text, image, video, audio etc.

In general half life of any model is short. What is more lasting is ecosystem around it, releasing new generation as open doesn't give away technical advantage.

They give away asset for which strategic value is deprecating rapidly and in exchange they get developers, mindshare, tooling, optimizations, integrations, research, standarisation etc. and put pressure on competitors destroying their margins.

It's not an absurd way of thinking, it may seem like irrational move but let's wait and see – imho labs like Anthropic can't sustain long term this kind of pressure and will eventually collapse – regardless of the fact that currently they look like strongest player that can't be touched, the whole thing they have holds on thin, fragile support that gets eroded.

mirekrusin··on Grok 4.6 scores 61 on the Artificial Analysis Intelligence Index
Move like making api substantially cheaper or free would make more sense.

Making it open weight does the opposite – it allows traffic to not be routed through them at all.

mirekrusin··on Grok 4.6 scores 61 on the Artificial Analysis Intelligence Index
Anthropic and friends infiltrate without open weights and their infiltration is 100% of traffic that goes through their official api.
mirekrusin··on Grok 4.6 scores 61 on the Artificial Analysis Intelligence Index
collecting data by making them open weight? where is logic in that?
mirekrusin··on llama.cpp
Yeah for that dense models are not great better to pick some MoE to get benefits of large vram pool while keeping reasonable speed.
mirekrusin··on llama.cpp
Optimize startup params for llama-server for your hardware (not llama.cpp itself), on my 2x 4090 I got ~20% speedup after maybe 40 mins.

ps. ngram didn't work for me very well, but dedicated speculative model works very well

ps. 2. in my case I'm just maintaining Makefile that does everything from update/upgrade (git pull/recompile) to starting server with different models, stuff like:

    # over baseline at temp 0.6 (95 vs 45 tok/s), ~4x over naive layer-split baseline.
    Qwen3.6-27B-MTP-UD-Q8_K_XL:
      ./llama.cpp/llama-server \
        -hf unsloth/Qwen3.6-27B-MTP-GGUF:UD-Q8_K_XL \
        --no-mmproj \
        --parallel 1 \
        --kv-unified \
        --flash-attn on \
        --fit off \
        --split-mode tensor \
        -ngl 99 \
        --cache-type-k q8_0 \
        --cache-type-v q8_0 \
        --host 0.0.0.0 \
        --tools all \
        --jinja \
        --ctx-size 262144 \
        --spec-type draft-mtp \
        --spec-draft-n-max 6 \
        --temp 0.6 \
        --top-p 0.95 \
        --top-k 20 \
        --min-p 0.0 \
        --repeat-penalty 1.0 \
        --presence-penalty 1.1 \
        --threads 8 \
        --reasoning-budget 2048 \
        --reasoning on \
        --chat-template-kwargs '{"preserve_thinking": true}' \
        --reasoning-budget-message "reasoning budget consumed, time to answer now"

    ...

    Qwen: Qwen3.6
    Qwen3.6: Qwen3.6-27B
    Qwen3.6-35B-A3B: Qwen3.6-35B-A3B-MTP
    Qwen3.6-35B-A3B-MTP: Qwen3.6-35B-A3B-MTP-UD-Q8_K_XL
    Qwen3.6-27B: Qwen3.6-27B-MTP
    Qwen3.6-27B-MTP: Qwen3.6-27B-MTP-UD-Q8_K_XL
mirekrusin··on llama.cpp
Just run /goal to optimise it and you should be good in less than an hour. Also best to use models that support speculative decoding.
mirekrusin··on Mark Zuckerberg attacks 'closed' AI rivals as Meta returns to open models
What would be an AGI data point?
mirekrusin··on Mark Zuckerberg attacks 'closed' AI rivals as Meta returns to open models
Progress is defined by capability data points over time that we have plenty where AGI is a threshold ie. stuff like METR [0] and dozens of others.

[0] https://metr.org/time-horizons

mirekrusin··on Mark Zuckerberg attacks 'closed' AI rivals as Meta returns to open models
I think we have more than 1 data point for AI progress.
mirekrusin··on Mark Zuckerberg attacks 'closed' AI rivals as Meta returns to open models
There is no reliable evidence they won’t.

Look at rate of progress, not today’s capabilities alone.

The gap is shrinking, many tasks are already performed at super human level.

Humanity doesn’t have monopoly on intelligence - it seems obvious to me it’ll be reached and surpassed without hard upper limit.

mirekrusin··on Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
Great to see Meta back, looks like really strong, local model, can't wait for llama.cpp support.
mirekrusin··on Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
There is literally not a single comment like this, the only off topic comment like this is yours.
mirekrusin··on New Amazon Data Center Is Set to Have the Most Polluting Power Plant in the U.S.
Modern data centers can be designed with near-zero direct water consumption. It isn’t fundamentally necessary to consume huge amounts of water to cool computers. What’s the story with those new data centers? Do they actually choose to do evaporative cooling that people are complaining about or people are complaining about problem that doesn’t exist?
mirekrusin··on Some thoughts about Anthropic's new cryptanalysis results
Layoffs would be a poor indicator that recursive self-improvement had occurred as you need people with knowledge around domain/layer the work is done on.

Also, if the lab truly has a self-improving superintelligence, the cost of retaining staff at any level would be a rounding error relative to its operating costs and the value the system creates.

There would be little economic pressure to fire them immediately, especially while they remain useful for oversight, interpretation, risk management or simply as "interface" to the rest of the world etc.

If anything, they would probably hire more people to pursue more opportunities in parallel.

mirekrusin··on Hubble: Open-source notetaking app for you and your agents
It's 680 commits old.
mirekrusin··on Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard
They can sleep just fine being the only player in town actually not loosing subsidized money.
mirekrusin··on Buz – A fork of Bun using modern Zig, with sub-1s incremental builds
I’ll keep checking.
mirekrusin··on Nvidia, Microsoft, Meta warn against overregulating open-weight models
Musk didn’t sign it.

nVidia did and they released good models – same with Meta, Microsoft, IBM, Mistral – all are signatories.

← PreviousPage 4 of 34Next →