Mercury 3: Diffusion at scale | Hacker News Reader

Mercury 3: Diffusion at scale

inceptionlabs.ai

257 pointscmrdporcupine92 comments

Pelicans aren't bad, a little quirky: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

Those were via OpenRouter - here's another set I made by calling the Inception API directly: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

I didn’t wonder why the bench marks were done in medium. The medium pelican is better than the high.

Maybe reasoning negatively effects the output after medium.

How fast did they generate? That's the specific sell with this model

EDIT: I see, "3337ms"

How does that compare to your other artistic works?

Yeah I thought the speed was a bit disappointing - that's why re-ran against the API without going via OpenRourer, didn't seem to speed it up though.

Crazy are the times when we consider it slow to spend 3 seconds to conjure a whole pelican into existence.

Those are some far better .svgs than I got last Mercury 2.5. I tried with 3d modeling and got some cool results incredibly fast! Unfortunately no real improvement under iteration (wasn't expecting that much, it's so fricking fast).

But seeing 10 mercury instances iterating all at once (as a personal demo) on a single concept felt unreal.

I still don't understand why Google did effectively nothing with their Gemini Diffusion demo from last year.

It was really cool, and really fast, and just went quiet.

They did release a Diffusion Gemma a few months ago but I've not seen much of a splash from that one, either.

They probably did not get it to work well?

Their preprint showed it worked well with small batch size, but most prod deployments right now have rather large batch sizes from serving multiple users

That's a great answer, thanks. I'd missed that detail.

Honestly, a lot of those issues could be remedied with more optimized kernels, which LLMs could definitely write quite well now. I wouldn't be surprised if some deployments of Gemini Flash switched to diffusion for cases where there is low utilization, since Google showed you don't need to retrain an autoregressive model to rework it for diffusion inference.

If that's true, that's the real answer, or more like anty-answer because you're saying basically: hey, nothing should stop them.

Google's gonna Google...

> since Google showed you don't need to retrain an autoregressive model to rework it for diffusion inference

IIRC you do need to retrain the model, just closer to the level of finetuning rather than e.g. pretraining from scratch. Which also means that the model will perform differently with a different capability profile.

Thank you for the correction. You dont need to retrain from scratch

I wonder if it'd work better as a local model

Good at research, terrible at product?

Difussion Gemma performed a lot worse than its equivalent autoregressive counterpart (it was particularly bad because rather than a sort of constant-ish gap you might see between 2 model classes, the gap seemed to worsen the harder the task/benchmark) and the speed benefits got negated at scale. Not really surprising we haven't heard much.

This talk serves as sort on update on where they are with it: https://youtu.be/r305-aQTaU0?si=j3IqG9BAT0uumPeA

At a high level: It doesn't benefit from batching to reduce memory bandwidth in the same way that a "standard" autoregressive model does, which ends up meaning that it's great at reducing latency for individual/low-concurrency users and delivers better outcomes on certain tasks but much more expensive per token at scale.

It's not Googles fault. It's our fault, llama.cpp doesn't support it, vllm can't really offload to host ram and while huggingface transformers supports it, it only can do f16 quant for MoEs like Diffusion Gemma. Unusable for low spec machines where diffusion really could speed up things.

We're sleeping on a amazing model right in front of our eyes. With diffusion models you can properly ban phrases. You can decensor and deslopiffy the model output without touching a single weight or doing a single backwards pass.

This looks noticeably more capable than previous gen. I purposely gave it a bit of a vague prompt to create an html renderer for UML diagrams, it produced impressive work. Then i asked a question, and in the third prompt I asked it to update the html which it did correctly.

Another thing that stood out, and this might just be a novelty thing, is that it picked different and less painful-to-eyes color schemes for each theme. Congrats to the team behind it.

Question (since their team has reps here and I have never used Inception via api): when you ask for a code change in your codebase (via coding agent), how much code does it need to read? How well does it handle tool calls? Since diffusion is a different way of doing things, I am curious about the code-needed for each incremental change in the codebase, say over a larger task.

I would assume the context requirements are the same, i.e. whatever knowledge is needed to complete the task. There is no novel technique that can avoid this.

No novel technique can avoid this, but certain techniques can make it worse.

Can you elaborate? Do you mean the model + harness combo reading more files than is necessary?

Yes, reading more files than necessary. Was curious about how their model handles a series of edits over longer tasks, but then tested it using openrouter myself, it worked quite well.

love these guys. instead of spending time on something akin to those creepy "dots" or obnoxious @Claude in Slack, they ship actual improvements to speed. you can't imagine what kinds of UX a fast LLM can enable! and for coding I have always been a proponent of strong compile-time type systems and handing off only the actual boilerplate typing, all to maximize actual thinking and speed of high-precision prototyping.

I actually built a small prototype that integrates with JetBrains IDEs - it tracks your in-IDE activity (kinda like JetBrains' own "file-scale tab-completion" feature) but you can invoke Codex with all that context at any moment to let back-then GPT-5.4-nano finish the miniature task. say, you wrote a new use case and your goal is to expose it through a new endpoint, but it's kinda boilerplate-y and DRY is not your style either. then you open a needed file, press a shortcut, tell what you want, and you get it. JetBrains Air kinda absorbed that functionality, Codex tokens are hella expensive (don't get me started on API costs), and JetBrains is ready to front that cost through "Junie Lite" (which is currently Qwen 3.6 27B), so I decided to abandon that prototype. but I do miss the combination of speed and control over resulting code.

I may revisit this idea with Mercury 3, or ask JB to integrate it.

I will argue that part of the push for this kind of new and novel methods are because the "dots" style of applications demand new technologies to work better.

"dots" are something that should work over long horizons of time, no? they are powered by Astra and still not good enough. Mercury, though, is all about making the same unit of intelligence a bit cheaper and a lot faster, which is not exactly what an always-on unpredictable non-deterministic text autocomplete needs.

I'm very curious what the actual model size is and how the training process differed from a traditional LLM. Cool work regardless, incredible how catching up to a few months behind the cutting edge now seems like an achievable feat for new entrants, if still quite costly.

@volodia (Inception co-founder) has a comment here which is flagged/dead. That's weird; it shouldn't be. Can someone fix that?

Send an email to the mods, it’s the best way to notify them of issues like that

Quick review: It's really fast and works really well in my initial tests. I used it to build a little Swift UI app that runs in the background and helps me OCR and translate text[1]. When I hit CMD + Shift + O it selects a portion of the screen (think of the screenshot tool), runs OCR on the portion and throws that at the google translate API and displays the result in a tooltip above / below the portion. The hotkey / key combination, translation provider and target language can be changed in the settings and there is a little tray icon from which I can access those and enable/disable the tool. I made it come up with a plan for this tool and had it implement the thing end to end in about 8 or 9 steps. It originally messed up the coordinate calculation but quickly corrected that when I pointed it out and other than that it one-shotted all the other steps. The code has tests and looks okay for what it is and it works really well so far. All of it took me around than 45 minutes of prompting, testing, correcting and approving changes. It used 26 million tokens (1.3M uncached) at around 160TPS. I used Oh My Pi as a harness.

[1]When I study chinese or japanese the PDFs I have often don't work with the macos preview selection tool and I don't want to go paste whatever I can select into another tool so I find this useful.

You should be able to skip the OCR step by grabbing the actual text from the windows via OS APIs, unless your PDFs are scanned and not encoding digital text.

Lots of them are scanned sadly! But I added an alt mode where a selection gets translated without the OCR step - though OCR is on the order of 100ms so I don’t mind it that much

Any idea why you only got 160TPS when the blog post claims 1000+TPS? That seems like the only draw to this model right now while capabilities are behind the frontier.

If their benchmarks are to be believed they've made some real progress. I hope we see more open weight diffusion LLMs (currently there's diffusion gemma). Also, if diffusion LLMs are starting to come into their own that's going to be somewhat disruptive isn't it? If these are running at 10x throughput (at least) then that's a lot more tokens on the same hardware.

EDIT: just trying this out on a repo and it's actually pretty good so far. Pleasantly surprised.

I'd be surprised if major labs aren't doing any diffusion experiments, and if they are, it would also be surprising if another lab is ahead of them in diffusion given just how much more compute they have at their disposal. My intuition could be way off though.

Google more or less ordinated the category. They have now seemingly dropped it, ostensibly because it doesn’t scale economically like transformers do (via batching requests).

I think though, that if Inception can keep up these speeds and prices, assuming diffusion continues to improve (likely), that Flash and Luna could lose out for many use cases.

Damnit, I've been waiting on an open weights model like this one, still no dice :(

diffusion gemma?

Was mostly an experiment. You're not going to use it for any real coding. Hopefully we'll see another updated diffusion gemma in the future.

It pretty bad. Just ask it to quote something like the first paragraph of moby dick (i.e. "call me ishmael...") it barely gets past 5 words before it completely gets it wrong. Even the worst normal 4B models get 3-5 sentences.

That’s because diffusion models struggle with memorizing sequences while it’s essentially what autoregressive models excel at.

I’ve tried to fine tune a diffusion model to memorize an MD5 hash and after days of hacking on it, it failed to completely memorize it. Autoregressive models of any size learn it over <1000 iterations.

Hmm at least in Openrouter, deepseek v4.1 flash via together/baseten/fireworks still beat mercury 3 on latency and intelligence

Ttft is quite slow with mercury

I've been trialing this for the last few hours. It was quite slow at first, but now I'm getting around 300-400tps. Still nowhere near 1000tps, but it's still incredibly fast compared to what I'm used to with other models. It's quite capable as well. I asked it to present strategies for training an AI to play fighting games and it gave quite well-reasoned output. It searched the existing my existing knowledge base for details about the game's internals, including record/playback, and it's own general knowledge about the game is played. It wasn't sycophantic about my opinions either - it challenged me appropriately.

I think they might have a really decent model if they can deliver their advertised speeds at scale at reasonable prices. A tiered approach could work well.

I created a pelican worked wonderfully, better than old haiku model or even sonnet models, maybe slightly worse than new gen models.

But overall it's a pretty ok model, the issue with speed seems to be that ttft is still high but still not sure how high thinking burden would be, I hope it scales well will be using it a lot if it works well.

This might actually crush most other LLMs if it's price and performance actually scales well. It was as cheap as deepseek for a few tasks I tried.

Anyone know if there is a subscription for it? I might want to test it out properly without burning a few hundred dollars on it.

I have liked diffusion models for quite a while, ever since i used the original Mercury to play AI Roguelite. I only have one question regarding Mercury 3: how tf did they get it to be so cheap?

If these benchmarks hold up it is a no-brainer against Haiku 5.5 and GPT-6 Luna, especially considering that Haiku 5.5 becomes 5x more expensive past 100k context, and this model goes up to 1M for the same price. (Note: the price increases 2x after a period post launch but is still way lower than Haiku is priced for >100k requests)

I tried the model in OpenCode now and it seems great, for certain workloads.

For grilling sessions, it is really fast and the quality of the questions are still decent. The suggested options are less versatile or specific. However, answer3ing one batch of questions and the next batch of questions is there within a few seconds.

And when you don't know how to answer a question, it can write a 600 line html prototype for each option in some 2 seconds and help decide.

Are these models harder to reason about given they produce tokens non-linearly? Especially since we rely on non-latent reasoning to know what the model is "thinking".

I wonder given its speed how competitive it would be with the current batch of decision models like Jev et al. Seems to me for decision models you really want maximum throughput.

They have a separate decision model called Mercury Decide

Having a chart with a linear scale on one axis and a log scale on the other axis is a really neat trick.

The resulting charts have a high probability of looking fantastic :-D

I'm going to start using this instead of Haiku/GPT-Luna for tasks that require speed.

I'm wondering if it'll still be useful once the discount is lifted?

I have trouble thinking of tasks that "require" this kind of speed. The usual trade-off is between price and quality/reliability.

Speed matters most when people are waiting for a response, when information loses value by the second, or when agents make dozens of calls to complete a task.

Customers are already using Mercury 3 for browser use agents, interactive education, real-time search agents, customer support, and more.

in my case, I'm hoping that the time-to-first-token isn't so bad that I can have my admin users parse some information in the web ui pretty quickly (a very specific use case, i know)

The time isn’t too relevant for let’s say coding tasks. But if you’re doing user facing, or processing live-data (logs for example) the speed becomes important

It's not really all that fast, for most things it's full seconds or at least a significant portion of a second. As computing goes that's quite slow and there are many things that require much faster speeds than this. However compared to the typical LLM it's quite speedy and opens some new doors for real-time-ish scenarios.

Looks excellent, looking forward to trying this out. Feels like we're reaching a critical point where these become competitive.

Not as fast as GPT 6.1-Sol on my favorite toy math puzzle. It might be one of those things where intelligence consumes less tokens

I really wish they would make these multimodal. Seems like a definite improvement over 2.5 though in my testing

I wish they had a coding plan so your could kick the tires before paying API rates.

One of the few LLM announcements I still get excited about. dLLMs have so much potential. See, for instance, how gemma diffusion can be repurposed as a decision model with output tokens and diffusion steps set to 1.

The guardrails are ridiculous though:

PROMPT:

> Draw an ascii diagram illustrating how to leave a two storey building in the event of fire

ERROR RESPONSE:

> I'm sorry, but I can't help with this because it involves potentially dangerous or harmful content. I'm happy to help with something else.

Come on

"Quality: On par or better than frontier speed-optimized models like GPT-6 Luna and Claude Haiku 5.5. Speed: 1,068 tokens per second on NVIDIA GPUs. Context: 1M tokens, up from 260K for Mercury 2.5 Price: $0.20 per million input and $0.70 per million output tokens. At launch, Mercury 3 is 50% off at $0.10 per 1M input and $0.35 per 1M output. Cached tokens are 90% off the input price."

its definitely good at admitting when it doesnt understand stuff. which is a rare treat with these models.

100M free tokens? wow.

Any comparision would make sense only if you made them - Mercury and Luna - the same training data and then the same post process, alignment. It's nice it's fast. I support it. Just saying I don't know how to judge that.

Hi HN, this is Volodymyr, Inception Co-Founder & Chief Scientist.

Today we're launching Mercury 3, our most capable production diffusion LLM.

Quality: Outperforms GPT-6 Luna and Claude Haiku 5.5 across a range of benchmarks at matched reasoning effort. Speed: 1,000+ tokens per second on widely available NVIDIA Blackwell GPUs, 4–8x faster than those models. Context: 1M tokens, up from 260K. Price: $0.10 per million input and $0.35 per million output tokens at launch.

The main takeaway is that diffusion scales. Since Mercury 1 we’ve scaled the model more than 40x in total parameters, and intelligence has gone up at every step without giving up speed. Diffusion generates tokens in parallel through iterative denoising rather than one at a time, so as reasoning models think longer per answer, the speed advantage over autoregressive models of the same size grows.

Early-access partners are running Mercury 3 in browser agents, tutoring apps, real-time search, and customer support. It’s OpenAI API compatible, and it’s also available on OpenRouter, Baseten, and Vercel.

Try it on our API with 100M free tokens.

Happy to dig into the architecture, scaling results, evals, or anything else in the comments.

Why medium in benchmarks? I've not seen that many people use other than xhigh/max on the cheap models. No low->max performance curve either. Time to task completion would also be nice!

We are going to be sharing additional information like task completion times in the near future as the model undergoes additional benchmarking by third parties. We also plan to share data on specific use cases where the model traces new points of the Pareto frontier and and speed cost.

For a preview of the latter, check out our analysis of the Mercury Voice model: https://www.inceptionlabs.ai/blog/introducing-mercury-voice

Will you make the weights available?

Not at the moment, but we are looking to share more information about the model in the future via technical blog posts and technical reports.

Ok. Good luck then.

I'm curious if you could comment on why your DeepSWE performance is below Haiku. This is not a dig - getting 46% vs. Haiku's 52% is still amazing. I'm just wondering if perhaps your post-training was less aggressively RL'ed on coding tasks than perhaps Haiku was. Having a cheap 1,000 tok/s option for coding work is a dream because - if it's reliable enough - you could blast through work such as high-level security reviews or finicky and repetitive refactorings that don't necessarily require SOTA performance.

We might not be able to precisely explain the 6% difference on that specific benchmark, but overall we have found the model to be Haiku- or Luna-level in our own daily usage. That means that it won't yet replace Opus or Fable as the primary driver of a harness, but we have used it successfully for subagents, including implementing kernel optimization ideas produced by a larger model (and those improvements have gone into production -- a form of RSI).

In general, the model continues to improve when given additional high quality data like a normal LLM, which is what makes us very bullish on the scaling of dLLMs.

Super cool. I'll have to give it a whirl! Keep up the amazing work.

I don't see it on Baseten. Is it special in some way?

https://www.baseten.co/library/mercury-3/

Not listed among https://app.baseten.co/model-apis

Does it mean dedicated inference only?

You have to reach out to Baseten directly for access. You can also access through our API (https://platform.inceptionlabs.ai/) or OpenRouter.

Incredible. Any luck in having a model that's multimodal in the future?

This is an area that we're really excited about. We have internal prototypes, and if you're interested in trying it out as an early tester, please reach out to hello@inceptionlabs.ai

Whats the model total param size and will you release the weights? What does harness integration look like? Do you have an harness? Is it good at coding? What about tool calls? Do we have to wait a fixed amount of time for every step it takes?

We're not releasing the weights, but the model is available using a standard API, and it integrates with your favorite harnesses.

Here is a page that compiles the list of integrations: https://docs.inceptionlabs.ai/integrations/overview

For example, it's just a few small steps to run the model in Cursor: https://docs.inceptionlabs.ai/integrations/cursor

You shouldn't expect Opus-level performance yet, but early testing has found the model to be great for coding subagents.

Google's research showed some of the speed benefits tended to drop off at larger batch sizes on an H100. Are there published batch-size vs tokens/s figures for Mercury 3?

We've been happy with how performance has scaled with batch size. The model runs in our API at reasonable costs, which is also what allows us to serve the model with competitive prices ($0.10 per 1M input and $0.35 per 1M output at launch).

Pricing can always be set at a loss, however, particularly for a private company. Are there solid published numbers for a specific hardware spec?

If I may, are there plans for visual input support?

We have internal prototypes, so please reach out to hello@inceptionlabs.ai if you're interested in early access

Is the model size secret?

We're not publicly announcing the model size right now, but we can compare against other models with a similar intelligence -- what we see is a big boost in speed for the same level of capabilities.

https://openrouter.ai/inception/mercury-3#performance

It seems the latency is not very good. Is this a temporary compute problem or is it inherent to diffusion?

Reply on news.ycombinator.com