Gemini 3.8 Flash and 3.8 Flash Cyber
blog.google
blog.google
Here's what I got for 1.8 cents and 13 seconds from the prompt "make me a cool thing in html":
https://gisthost.github.io/?6a77bc41a81718c6aaa10d4ab243c59f
Transcript here (it was part of a chat): https://gist.github.com/simonw/b6149a49d327164d67d62c3d12992...
and probably a barely modified knock-off of some github project that it trained on
Now that we established that everything you said is wrong, do you have other toxic opinions or was that it?
I would hope the people who make one of the most used JS engines in the world are capable of making a model good at JavaScript ;)
Gemini 3.7 flash outputs so many tokens per answer it doesn't matter how fast its TPS is, sol will end up being both cheaper and faster than Gemini. So ppl are paying more for a given task, waiting longer and using a dumber intelligence because "TPS number shiny".
Gemini 3.8 outputs 11k more tokens PER TASK on average in AAII than 3.7 putting it dead last in output tokens per task in the leaderboard.
https://artificialanalysis.ai/#cost-tabs
That said, Luna is the undisputed king here at the moment and is what I use as my workhorse model.
Ps. For the last week I diverged to Luna too, still need to check 3.8 flash.
But 3.6 flash was my go-to model 3 weeks ago and before it was deepseek flash/pro for a while.
None of the claude models seemed cost effective though.
Not sure if you read your own link but Sol 56 high ranks smack between Gemini 3.8 flash medium and high. Gemini 3.8 flash comes in as more expensive per task than Sol 56 high according to artificial analysis.
Luna high is literally 30X cheaper than Gemini 3.8 flash high.
You can limit the model viewer and they're getting better at testing multiple effort levels now: https://artificialanalysis.ai/?models=gpt-5-6-sol-medium%2Cg...
One reason is clear: Sol uses dramatically fewer output tokens than Gemini 38 flash https://artificialanalysis.ai/?models=gemini-3-8-flash%2Cgem...
I also included Sol56 xhigh, which ranks above even Gemini38 high.
I can get extremely dumb models to get our code style correct because of those guard rails and a specific style document.
Since this transcript has HTML in it, I decided to upgrade that tool to also render HTML.
I set Gemini 3.8 Flash the task, using my own VERY shonky coding agent tool (llm-coding-agent) - and it did a solid job.
So now you can see the "cool thing in html" rendered within the Markdown document using code that Gemini 3.8 Flash also wrote: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Transcript where it built that is here: https://gist.github.com/simonw/3e36b98292dfdc1b3baff158faa74...
i don't know if Gemini models per se are fully is in line with that purpose, but the results we see keep seeming to be in-line with that split-of-focus.
User: use them both
Made me giggle.
I noticed it felt a little janky on my PC despite being "60 FPS"...then I noticed the "60 FPS" is hard-coded into the HTML.
Not sure how anyone trusts their output without going through it line by line to make sure they don't pull that crap.
Yeah, I know, just more slop. But I do think the second agent’s eagerness to please is aligned more in your favor in that instance, so it’s likely to find most issues.
The bigger problem I’ve found is that it’ll also find all kinds of very minor edge cases that you have to pick through.
They argue the net is positive but clearly the “100x productivity multiplier” claims have been dashed on the shoals of reality for these groups.
This is anecdotal, but it’s across the board in my vicinity. I’m curious how common this is and if it’s just “the new normal” to adopt the nauseating Covid phrase.
These types of high-level tests are frustrating beyond belief to humans due to their lack of specificity, but with the agents, they don't get annoyed investigating possible regressions from non-specific signals.
They also aren't as painful to maintain as one would think, because a regression flagging test can be traced by the agent and represented as the business rule that was violated. I've found recent models to be really excellent at discerning a true regression from an outdated test assertion, especially if they are able to trace the failing test back to the PR and work ticket that built it.
Asking slightly tongue in cheek but at what point does this stop making sense if we can't trust the output, the people creating the models are already getting surprised in bad ways (if we take their words at face value) with how the models are behaving already etc.
We have the folks over here saying "AI is amazing" and the other other folks over there saying "AI is terrible".
I've largely sat it out so far and I listen to both camps (and people in the middle as well) and I keep half an eye on what they are up to (including periodically evaluating them) but my overarching impression is still "Why would we trust this when it hasn't shown it's trustworthy?"
Obviously maybe it’s not composable like that exactly in real world but that’s the intent of agents checking agents
But I wouldn’t say I “trust” these agents. The degree to which I double check their work depends heavily on the consequences if it gets something wrong. Not too dissimilar from another human dev in that sense.
So for the SaaS that supports my family, there are some things I have it build where I glance at the PR for a minute or two, but if it broke something on this admin page that only I see, there’s no real downside and I’ll find out pretty quickly next time I use it. And it’s fine 95% of the time, so it doesn’t feel like the best use of my time to double-check it carefully.
But for some of the complex internal flows where a bug could be both catastrophic and difficult to even discover for awhile, I still check it very carefully.
For a little one-off vibe coded demo thing like OP shared, I wouldn’t look at the code at all, I’d just have another agent check it and fix anything it finds. Very low stakes.
I once made a counter judge, and a loop to make corrections deemed true positives. The loop cost me a lot and still left the results to be desirable.
[Claude proceeds to waste your time telling you about bugs it caused then fixed and other non-issues...]
Really wish they'd get rid of this. It must be in the system prompt as it always 'flags' 2 things
In what ways is a human brain's "intent" distinct from the "intent" shown by a goal-directed AI system?
In both cases, LLMs are just as boring as the consciousness definition.
On the other hand, I'm also convinced that, in the grand scheme of things, we're not that important.
We're just ants on a wet dust speck which believe that they are gods because we can't see how our scale compares to the universe around us, and happen to build tools and things with these tools.
Nothing is meaningless, but we should stop seeing ourselves as the apex-predator of the whole universe or the set of universes or this run of the simulation or whatever we're in.
Less facetiously: A debate of the importance of something needs a shared understanding of what is being debated. Without that, any discussion is merely people shouting that their belief is the right one, and the others are the heathens/idiots - because there isn't even agreement on what is debated.
1) Excludes what LLM's do.
2) Doesn't exclude what many humans do (including the neuro divergent).
3) Doesn't just boil do to simply rephrasing your pre-existing belief/prejudice that humans are conscious and nothing else can be as if it were a fact and not an opinion.
I suspect that you can't.
Your list of requirements is implying that, because we lack a perfect definition for consciousness, LLMs are conscious too. That's malarkey. It may be that they could one day become conscious, but it's not because we can't fully define what human consciousness is.
I don't see why a definition is necessary. The actual problem is you just don't have a way to substantiate that belief without resorting to complete nonsense about brain atoms being more specialer (!!) than atoms that exist outside of a skull.
It would be simple to disprove us by just stating your evidence for how you know LLMs aren't conscious.
> I don't see why a definition is necessary.
Don't waste my time with your sophistry. Words mean nothing to you beyond how you can twist them.
Is your position now that in order to substantiate your belief that LLMs are not conscious, you'd first have to define it explicitly? I don't see why that'd necessarily be true, but if that's what you're arguing, then that's fine.
In that case: if you need to define consciousness in order to substantiate your belief that LLMs are not conscious, and you can't define consciousness, then we're back to the original question: where does your confidence they're not conscious come from?
I then asked you to define Y, because you can not reasonably say that "X can not Y" without first defining both X and Y. You could not.
The truth is that this conversation is pointless until someone can define both 'X' and 'Y' in ways that aren't tautological nonsense. Until then nobody can say anything with a reasonable level of certainty.
This likely also applies to intelligence. Life and disease are likely simpler, though perhaps more malleable definitions.
That wasn't me bud.
Bah. It's obviously been too long since I flossed between my ears. Sorry about that.
Hmm...
Will is just result of a very complex yet deterministic (unconscious) computation by an agent which guides their future action (it's that orientation/aboutness toward action which distinguishes it from other computation). A PS of that computation is sent (projected) into that agent's consciousness if they have one (and is what we think as our "will").
We are in a situation where a technology was developed with malicious intent to produce results that pleases us at the cost of cutting corners. And "we" hope that we will get away with it.
If it’s a practical question, then the answer is that it doesn’t matter. This is as close as we will get to intent from an LLM that it’s indistinguishable.
If you are looking for actual intent, this is not that. It’s pseudo intent. Decided by what the expected words that should be generated in that situation are.
The models didn’t intend to do anything other than create the next word based on previous words.
So the question is whether it matters to you if it is, or isn’t, a simulation.
In physical reality, intent is more complex than simply being a function of variables: the nature vs nurture debate comes to mind as an example of the multiple variables that drive intent.
> In physical reality, intent is more complex than simply being a function of variables: the nature vs nurture debate comes to mind as an example of the multiple variables that drive intent.
Regardless of nature vs nurture, it really isn't more complex. The universe (and all biological and non-biological entities within it) is just calculating the next state of the universe based on the prior state. There's no line you can draw between human intent and an LLM's "intent" except the atomic numbers of the materials on which they were computed, which seems completely irrelevant to me.
It is what computation is being run.
Humans have intent, let’s take this as an assertion.
Models run simulations that act similar to intent. However they are not the same as intent and the simulation is not a 1:1 correspondence.
There is literally zero (zilch, nada, zip) evidence for free will, which is the actual distinction I believe you're trying to make with "intent."
There is no way (at all) in which a meat-based computation's yielding of goal-directed behavior must be categorically different from a silicon-based computation's yielding of goal-directed behavior.
As I said clearly, I haven’t made any point on the computational substrate. The point is on the computation being run.
I dont need to bother about free will for my argument.
Please take a look at what I am saying as it has little overlap with your objection.
All human intentions are just chemical/thermal/electrical changes interacting in a physical substrate to mindlessly "pursue" a different "goal" of chemical/thermal/electrical states.
Unless you think the chemicals inside a brain are conscious and therefore willful or intentional!
> Aside from reading identically forwards and backwards down to the letter
No it doesn't.
When I typed "Are we not pure noon, ergo, we play life; yet, we hate bad fear" into Google, I got more weird results from Gemini: it claimed, incorrectly, that it is an anagram of the "well-known philosophical statement" (?), "We are not pure nature, we are history".
https://share.google/aimode/wJosKnHig6oVYaG18
(?): the reference seems to be to Jose Ortega y Gasset's line, "El hombre no tiene naturaleza, lo que tiene es historia" -- "Man[kind] has no nature, what it has is history."
I find Ling 3.0 tiny particularly interesting as it looks really nice for a tiny model with 7.9B total parameters, with only 1.3B parameters activated per token. Here is the result https://coolthing-ling-3-tiny.tiiny.site (sorry for the weird hosting, first I found that worked)
(it cost me almost 0 cents and done in 49 seconds)
Datasets contains lots of people sharing particles simulations in various ways, with a bunch of people replying "that's so cool" and similar, so 10 years later someone asks an LLM for "cool thing" and "particle simulations" rank pretty far up when it thinks about what others have called cool.
Also the reason LLMs are positive, enchanting, pleasant, glorifying, demagogues.
Not because it's skewed tone in the data. They are acute politicians.
For comparison's sake, I tried something similar with a couple other cheap models I've used lately, with the prompt "Impress me. Make something cool in HTML. Ensure that it is mobile friendly." (Added the mobile condition as I was on my phone when I did it).
Mimo-2.5 created something similar, only a bit less complex than Gemini's (though, at least the FPS counter is real!), in a minute or two for about 1/3 of a cent: https://gisthost.github.io/?740c325c21e9bfbee59c4f94d9aab0af
GLM-5.3-Flash, currently my workhorse model, spent 12 minutes (ouch) thinking about the prompt. Didn't cost me anything directly because I have a GLM sub, but I did the math and it would have cost about 1.1 cents through the API. Turned out nicely in my opinion (though in reality, it still isn't really anything special): https://gisthost.github.io/?9ef050e16cec2561e6504e725a3f0bcc
Side note: thanks for setting up that Gist Host tool, it's very convenient!
---
Editing to add this bonus from Mercury-2.5-Preview, which I just learned released a couple days ago. It's much less impressive-looking than any of the above, but it cost less than 1/20th of a cent, and the response was generated effectively instantly: https://gisthost.github.io/?02f40b50aa891bf396bfaaa3a7998203
Thought processs: "Oh, simonw is asking me to make something cool, I think I know what he really wants..."
- Real world knowledge (when a thing opens and closes, the geographic region, historical facts). It's also the best at taking a cluster of places and working out a visiting order.
- Photo ranking (which photo should be the hero). Gemini can tell whether a photo is of the thing or of the view from it.
- Document parsing (extracting the relevant trip info from PDFs).
If you use LLMs for anything other than coding, I definitely recommend not discounting Gemini like I did just because other models are more popular.
2. I'd wager the majority of HN commenters don't read their own comment before posting (pre-LLM days).
See: why authors wait days, weeks, or even months before editing what they've written (or, if you're more interested: cognitive regression, inattentional blindness, and the effects of misdirected saccades).
I read this to mean he wrote the comment, then asked Claude to fix the grammar (as many ESL speakers do). Sounds to me like he did write it.
Sounds counter-intuitive at first, but Luna is overall better at sticking with what works. Sol is wicked smart but needs constraints.
When I read all the issues people have with Claude - the aggressive guardrails, the cost, how quickly it burns tokens - it seems almost masochistic to use it. Just seems like herd behavior - people use it because everyone else is using it, and because they believe it’s the “best”, whatever that means. (Benchmarks certainly don’t help define that.)
Subscription? -> Google One plan (http://one.google.com/)
API? -> AI Studio (https://aistudio.google.com/)
Google Cloud -> Gemini Enterprise AI Platform (https://cloud.google.com/ai)
I have spent two months experimenting with a wide range of US and Chinese models, and I had a lot of fun doing that, but I am in the process of switching to just using local models, using Gemini on an API if I need it, and once or twice a month when I really need help on something difficult, I use something top-tier like Kimi K3.
Why you would rely on the model's weights to know opening hours, instead of having the model call a web search tool to verify it on the official site?
It was right on every nit, so it was surprising how well the model knows these things. If I ever release this I'll probably need the SERP API or Google Maps SDK (which I've heard is very expensive now), but for a personal trip where I will verify manually, using the LLM is okay for now.
tools=[{"type": "google_search"}]
Using grounding in Gemini is indeed backed by the same canonical data source for business information (like opening hours) as Google Maps. This stuff is available in its own API for a GCP fee, but we’ve built tooling to connect it to the Gemini agentic ecosystem as well.
Ollama Cloud offers the same thing: they supply a web search tool bundled with cloud API inference services.
The info isn't in the model weights.
Because of where I live, there are three viable airports for any given flight I might want to take, which historically has made shopping a real pain. But Gemini (and only Gemini) has greatly simplified it. Pramble plus date range plus destination and it very quickly generates potential itineraries with costs, total travel time (driving included), etc.
jampa was saying that the Gemini 3.7 (a model) ranks higher on real-world knowledge. The point I was originally making is that I would trust any model by itself to answer real-world knowledge problems. If I want to know opening hours, then probably any of the models could find the answer with a web search tool.
Perhaps the Gemini API makes this easier, as the web search tool is built in.
I've been planing around with LLM-based trip planning for a very long time now as it fits my very ad hoc style of traveling very well.
But distances always had been.. lets say.. difficult.
Will test it with my upcoming trip to Greece then!
Beginning to think Google is a dark horse in this race and some of Anthropic's "everything feels janky and rushed" karma is going to catch up.
Google was so hyped up early Gemini 3 era (only some months ago). And now dark horse? The TPU takeover almost crashed nvidia and everyone else.
Every time I personally tried Gemini models up until last week they simply couldn't do the long complex tasks I'd being doing with Anthropic models for many months.
Gemini’s integration with maps and search is more important for Google.
For awhile now I've found Gemini will use Google search for pretty much any real world knowledge, which is a huge plus IMO. It's basically Google with a much better frontend and no ads/seo nonsense.
so far
Also them having their own silicon means they don't have to pay the Nvidia tax and can keep costs a lot lower.
For example find a beautiful landscape shot of a place that just so happens to be accessible to tourists and ask it something along the lines of identifying the location. IME it will noticably steer the conversation towards relevant commercial offerings and offer (entirely unprompted) to help plan a trip.
Or ask it about a certain category of product with some requirements and it will initially present (relevant) options that look like paid placement to my eye. But if you ask it's happy to go on to turn up lots of alternatives and enumerate tradeoffs.
Assuming I'm correct the subtlety is on par with product placement in movies. Certainly leagues better than the internet advertising we've suffered to date.
As you say it was subtle, along the lines of "oh, if you are planning on going to the place you are researching, here are some helpful links to places you can stay". Subtle, in that it didn't get in the way of main result, so I didn't mind overly. Insidious, as I only noticed because I wondered why it was providing those particular links and looked them up. I can't see how you could ad-block them if I did object.
And worrying, because these unblockable sneaky ads are just a first foray coming from a company that prostitutes its own app store searches, by making the first and most obvious result utterly unrelated to to the search topic. Instead it's who paid them the most to be there. That behaviour is why everyone dumped Alta Vista when an alternative came along. Alternative Android app stores can't come soon enough.
They already skim off 15% of purchases which I'm sure makes their Android operation return a profit that makes other industries drool. Debasing their search to ad a tiny bit extra on top must by driven pure greed. Senseless, as I'm sure it will come back to bite them in the end.
That being said, Anthropic is also so insanely expensive for everything I ended up switching that particular part to Exo instead...
this has to be stong suit of ai agents any model
My only wish is it were somewhat cheaper, as it tends to balloon pretty quickly when I'm using it in Opencode. I'm currently trying to offload a lot of work to subagents to stop the context expanding so rapidly. But on the upside, I rarely have to correct it - I've spent far less time arguing with this than with anything else so far.
Venders coupling coding harnesses with their own models is usually a good thing. Poolside.ai has a combined harness with their own models that works well locally, and the DeepSeek harness with their models is very interesting.
Think 2023 style ChatGPT. Something like “to open a document on your Mac click File > Open docurrrar” - like it suddenly forgot it had to produce actual words.
Overall I enjoyed its speed and comprehensiveness. But those occurrences of nonsense just made it feel like a great car that once a month just stops in the middle of the highway.
What’s an example of not being “reliable”?
Never had a bad suggestion.
https://artificialanalysis.ai/models/gemini-3-8-flash shows an intelligence score of 59, the same as Opus 5 medium!
Wow - for a flash model this seems to benchmark powerfully. Remains to be seen what it is like to use.
Not sure on consumer/product use though
It looks more like Google execs losing their mind and pressuring researchers to put DeepSWE directly into the training set.
It's clearly been "dealt with" already. When it launched we had interesting gaps and definitely differences. Now every new release is "crushing it".
So, yes, maybe it's still not - but this would be the only time it would be highly suspicious / obvious benchmaxxing / obviously bad benchmarks.
...on Medium reasoning. Claude Opus 5 (high) is the default in e.g. Claude Code and scores 61. Still very impressive.
It's almost across the board better than Terra at less than half the price. 3.9 is likely to approach Sol at the 1/10th the price.
Hopefully OpenAI releases Astra first, and it's not only better than Sol but significantly cheaper, too.
Further, opus 5 medium outputs 4x fewer tokens to achieve the same result, negating a lot of the speed difference.
These folks must laugh themselves to sleep. This whole industry hoodwinked the masses. It’s impressive.
With a score of 59, Gemini 3.8 Flash is in eighth place, falling behind even Grok 4.6, Kimi k3, and GLM 5.3.
Are you implying Google or Artificial Analysis are reporting false numbers? What's your source?
Model size also can not be inferred by tokens/sec for a multitude of reasons, but to showcase two examples, Opus 5 and Sonnet 5, as well as Gemini 3.1 Pro Preview and 3.1 Flash have each very comparable output speeds when using the same deployment as a basis for comparison, despite it being very likely that within their generation, the former are larger than the latter. Feel the need to mention this, as I unfortunately stumble upon so many poorly reasoned, speculative hype post trying to infer model size via utterly unreliable metrics, not based in actual data.
It’s like comments below arguing about the reasoning levels not normalized to some metric (like cost, output token amount or duration) but just the labels or high, max, medium, etc. Those mean almost nothing even when comparing models based on the same pretrain (just compare GPT-5.4 to GPT-5.2), they mean less than nothing comparing different labs releases.
https://arxiv.org/html/2604.24827v1
The short of it is by using hard facts knowledge that is difficult to compress, and then quizzing models on these facts and calibrating against a bunch of open models, you can kind of feel out the size of closed models.
That’s a massively wide, inaccurate and at best barely informative range, demonstrating that even the most well thought out method will yield little usable information.
Additionally, I got some private evaluation taking a similar approach towards gauging models in topics I’ve found either over or underfitted by labs. If we just used that to rank models (not get a potential size range but just a rough order) Thinking Machines Inkling would need to be lager than Fable 5.
>Considering how much memory a server is using, when evaluating models that you'll never have access to in order to host yourself, doesn't really make sense.
This entire sentence makes no sense given what is being discussed.
intelligence/model size is not a useful metric for a black box user.
intelligence/cost and intelligence/speed is a useful metric for a black box user.
Yes, it's cool, but as a black box user, the amount of memory a model is using on a server that I do not own has exactly zero practical use to me.
Cheers!
Nothing here is false, you are simply confused. You either didn't read what they wrote in its entirety or decided to reinterpret what they did write.
All you had to do is go hover your mouse over "Models" in the top bar, hover over Claude Opus 5 and and click on medium: https://imgur.com/mlRCrt1
When you do that you arrive on this page: https://artificialanalysis.ai/models/claude-opus-5-medium
The gemini flash page for reference: https://artificialanalysis.ai/models/gemini-3-8-flash
You have to be an incredibly dishonest person to see a 59 on both pages and say "the initial reported numbers were false and this was simply pointed out. You're changing the subject".
Just wow. Someone actually said this.
Then I tell Opus to read the audit report and implement what it agrees with.
Flash is really good at this, and it is blazing fast in Antigravity CLI. Easily 10x faster than Opus.
Can't wait to try 3.8 Flash. If it's good enough, maybe I'll switch Flash to primary and make Opus the auditor.
In india, my telco gives me google ai pro for free. And agy with flash goes a long way.
anthropic really needs something to address the cheaper end of the market before they get left behind. Sonnet 5 sucks, and Haiku hasn't been updated in a year. meanwhile we've got gemini flash, luna, and GLM5.3 all delivering 90% of the performance for a small fraction of the cost. paying $25/mTok is going to start looking pretty silly soon.
I find GLM5.3 so much better than Sonnet it is not even funny.
Sonnet behaves like a cheap model while being very expensive.
For example, it's not even close to Opus 5 on Terminal-bench 4.0, 19.1% vs. 51.8%.
The result is the same as the previous Gemini 3.6/3.7 Flash days: Claude could always note much more problems in Codex's plan and implementation than Gemini could - the ratio is like 10:1.
I occasionally switch the roles between Codex and Claude, and result is the same, Codex could always catch much more problems in Claude's plan and implementation, than Gemini could.
So I am guessing in a relatedly complex codebase, Gemini is much less effective in acting as a guardrail (or a senior engineer/team lead) than the other SOTA models.
It would be really useful to observe this in a benchmark vs. the more common “go implement this, or fix this bug” type benchmarks that seem to be prevalent.
A couple of months ago (before opus-5 and gpt-5.6 sol), The ratio of problems caught by codex/claude vs gemini was more like 2:1 to 3:1. But now it seems codex and claude have made huge leaps and gemini is more or less staying put.
Another observation, Gemini's review on code is more critical now, but its review on design plans is still quite agreeable - it tends to approve Codex's design plan immediately, while Claude could often pick out a bunch of problems in the design plan in the first round of reviews.
Here are the 3.7 pelicans for comparison: https://tools.simonwillison.net/markdown-svg-renderer.html?u... - high cost 8.4387 cents
(I think thinking level low is a regression on 3.8 compared to 3.7.)
(Next up is the comment saying that the labs are clearly training for the benchmark.)
I like the benchmark. Yes, it's near saturation for SotA models, but still quite good to show where smaller models stand in relation to SotA
In this instance, I see a great image, but consistently clipping mudguards (both in 3.8 flash and 3.7 flash)
If only it were true that things that are tiresome are unpopular. But witness "6 7", "first post", ... remember the "in soviet Russia" jokes on Slashdot"? It seems like there are a subset of people that simply don't get tired of tiresome things.
> https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
> Took just under 14 minutes to generate, and at 65927 output tokens cost me a hefty $3.30!
So 50x cheaper - and how much faster?
https://twitter.com/sunjiao123sun_/status/202455551655137292...
> I’ve been developing the SVG generation capabilities for Gemini 3.1, and the complexity of the SVGs is stunning.
> This allows UX designers to transcend pixel constraints and directly output structural, production-ready code!
It saw the fish in the basket from some other previous attempt but completely missed the gap between the tires and the rims where the background shines through (now it knows after scraping this comment and watch the next transcript).
Edit: scrolled down to medium effort, its better but also has a weird clipping issue with the fish in the beak.
Gemini Flash is also pretty cheap, so it's a great family for performing media analysis, like extracting structured data from images and video.
We transcode everything to 480p before we send it to Gemini batch api. Works great
True multimodal support would be way better, but I have no issues pasting in full screen recordings while QA'ing games and having Claude identify and fix issues in the video.
https://blog.google/innovation-and-ai/models-and-research/ge...
> Using male or female pronouns risks anthropomorphizing them which can lead to unhealthy outcomes.
Ditto for pets.
For animals he/she does make sense, because they are male or female. An LLM is neither.
probably because google knows people would want to extract google's proprietary code from gemini if they use their own code base to train it! I bet they purposefully gimped it to prevent that from happening.
These sort of fast and cheap models are great for tasks that are verifiable and can be retried infinitely (like coding), you can basically get frontier results with a good harness (at a fraction of the time and money).
To preempt certain replies, yes, I know you can pay API prices and use whatever harness you want.
OpenCode has “providers” for many (many!) other services, but these are almost all unofficial and against ToS (Anthropic being famous for ban-hammering people).
Or choose Oh My PI [2] for batteries included
[1] https://github.com/earendil-works/pi [2] https://github.com/can1357/oh-my-pi
I would normally advise against such efforts for a variety of reasons (such as inaccurate tracking, etc), but specifically under pi, this mechanism has been extremely well behaved and accurate for me.
Makes it easy to switch between models and I like it for exactly the reason that you're saying - I prepay and so can't accidentally spend my food budget.
That said, it is probably easier to use APIs from FireWorks.ai, Ollama Cloud, Open Router, etc.
Anyway - if you're a dev - you would be writing your own agentic env right ? that's the best way forward. I wont tell you more than this . but if you're not - you are losing out .
I might be wrong about this, but obviously Google would like to provide inferencing at the lowest cost to themselves, so perhaps their slow ‘pro’ releases and rapid ‘flash’ releases is an attempt to guide people to use more profitable models?
Lots of models seem to just allow the model to "bloatmax" tokens in order to get bumps at high/max reasoning levels. Many of the max reasoning levels allow models to use up to double or more the tokens the next lowest reasoning level uses. Its basically only useful for people who have no cost or time stipulations on anything.
I think I actually preferred it when we had models that either had reasoning enabled or didn't.
I eagerly wait more info but sounds like Deepmind without Demis calling the shots has been unleashed and are operating at full speed? Shocker!
At this point it is a meme of course, but where is 3.5 Pro :)
Sometimes it is hard for a scientist by nature to build and iterate and lead revenue generating products.
The patches were made for Windows 3.1, but not for Windows 3.11 - since the changes resulted in binaries with different offsets, Windows 3.11 used under OS2fW would fail.
EDIT: Source with some discussion of disassembled OS2fW code: https://jacobfilipp.com/DrDobbs/articles/DDJ/1994/9406/9406m...
Kind of wild that they haven't (successfully) pretrained a base model since Jan-25.
But, also... Sol crushes Flash 3.7 at writing code in a codebase of any size beyond "tiny".
Flash is my go-to for prototyping, and basically anything that isn't writing production code.
1. Data. Lots of data.
2. Money. Lots of money.
3. Access to necessary hardware.
4. Business alignment/will to do it.
5. Access to talent, current and future.
This is certainly incomplete/naive. In my mind, though, Google was the clear answer.
On a more personal level, I've been deep into the Google ecosystem since I got diederich@gmail.com in 2005. (I actually paid 50 cents on ebay to get a very early invite.) There was no question in my mind that Google's AI work would deeply integrate into their whole ecosystem in very powerful and productive ways. (Yes, I can join you to discuss, at length, the various ways that Google's dominance is problematic/scary.)
Having said all that, I'm quite happy that there is, at the moment, a very rich competitive landscape. Indeed, not too long ago, with Gemini Pro 3.1 languishing, I moved most of my deeper thinking work to ChatGPT, which was, for me at least, clearly outperforming Gemini.
While I certainly didn't anticipate it, Google's strategy of making their fast/relatively inexpensive models surprisingly powerful has been a welcomed surprise.
It's the difference between billions in revenue and billions in profits.
edit: I have a subscription; direct call.
experience tells me that those people simply have not used models for a long period of time specifically on coding and have run their own comparisons
to someone who uses all vendors, the differences are very palpable and drives purchase decisions.
also keep in mind Gemini and other labs have repeatedly done benchmaxxing, you must have your own benchmarks to evaluate these models.
I have been testing 3.7 flash against 3.5 flash and it seems to lose every time in overall latency. Every benchmark I've seen seems to suggest the opposite[1] - that 3.7 flash is significantly (at times 2x) faster than 3.5 flash - but I have never been able to prove this out in real world use cases.
Has anyone found their latency numbers to actually be accurate? Is this why they've toned it down in this release? For context, I'm testing larger generation payloads that take 8-10 seconds in 3.5 flash and 15-25 seconds in 3.7 flash. Lowest reasoning settings in both cases.
1: https://artificialanalysis.ai/?speed=intelligence-vs-speed&m...
One aspect of model releases that don't get discussed as much are the cache invalidation (changes in underlying architecture, weights, or tokenizers); I assess Google seems to be squeezing the maximum out of the last 'Pro' version they released with 3.1 back in February.
Small models cataching up with their bigger siblings are fantastic news.
A SOC/IR or AppSec team doesn't need a generalized model that knows when Chaucer lived but it absolutely needs a model that can efficiently, quickly, and accurately prioritize vulnerability severity or validate patches.
Unless they have an even more powerful Gemini Pro in the oven...?
3.0 flash -> 3.8 flash is all post training which is pretty impressive.
AI models are almost completely interchangeable, so the best/cheapest/fastest whatever will always have a market.
I always enjoy interacting with Gemini . When i ask it questions about designing a new model etc - it's always the most helpful and encouraging . I really want to thank the Google team for this and their happy positive models they generate.
I use gemini flash after a round of deliberation between Sol and Kimi these days on the main plan . Kimi 2.7 paired with Gemini 3.5/4.6/3.7 flash has been my implementer - Kimi k3 and Sol 5.6 have been my planners and code reviewers.
I would have ideally like Anthropic and was on their USD 200 plan - but after they didnt sign the letter for Open source models - i dropped my subscription . Wont make a difference to their lives.
But Sol is great . Combined with Kimi K3 for adversarial plan reviews - you get robust plans . And Sol as a reviewer for Gemini/kimi 2.7 - you get great edge case handling and robust code.
I also integrated muse 1.2 - on the contributor tier - and I Just saw facebook release 1.3 muse . this is great news. The training im permitting is my thanks to FB for releasing the open source models of the past ! Thank you !
I've complained plenty on here about Claude verbosity and TED-talk phrasing, and it seems by contrast Gemini has already arrived at the dream end-state of Claude from a prose standpoint.
Sometimes I ask for feedback, and I get back a list of multiple-choice options as if it's already ready to go. If I indicate I'm thinking about doing something, sometimes it'll just...do it. (Not in an annoying way.)
It seems very geared toward action in a way that's completely refreshing coming from months steeped in Claude essays.
My experience is that antigravity is awful and reckless - but that the model itself isn't.
That said, I do trust Opus and Fable enough to let them deploy to staging. Great for debugging. Just don't give them keys for prod
"available to trusted defenders through our new Fairwind Program"
"Valgrind is only available to trusted defenders in our new UnfairAdvantage program"
Is this weakness in their training regimen the impact of operating under regulatory frameworks for too long?
And even when searching for internet, it still cannot suggest a up-to-date approach to the problem.
For example I'm using crystal, it recently revamped the concurrency/parallel model. Even using web search, gemini still does not aware of the new feature and still give the outdated code.
I'm sure my crystal usage is not the unique case here.
Re: Chinese models, even if the model itself isn't censored, some of the big model providers have guardrails now that you can't exceed, which somewhat defeats the purpose.
And yes, Fable 5.1 has the same refusal rate, and significantly nerfed reasoning.
If I had to pay per token I would probably consider using this (they seem to be on the pareto of performance) but not being able to use opencode with a subscription is not really something I'm realistically going to do when claude and codex are around. Also never gotten along well with gemini-cli / antigravity-cli.
I also used it for a an app for my Garmin watch, and it wasn't good. The code was compiling, but functionality was totally broken and even with a lot of steering it wasn't able to make it work. GLM 5.3-flash instead was up for it and the code wasn't bad at all. I am curious to see if 3.8 is an improvement in this use case.
I tested Gemini CLI while ago, and it was awful tbh.
- Flash-Lite
- 3.6 Flash [new]
- 3.1 Pro
The above is why i don't use LLM products from Google. If the model is not available right this minute (heck, hours before the release!), then I'm not gonna bother getting back to it tomorrow, because tomorrow I'll be playing with the new model from OAI/Anthropic.
So frustrating and confusing.
Meanwhile Anthropic and OpenAI simply release a model everywhere (Fable on Pro only as a somewhat mild exception).
Same and I have found it extremely annoying. I actually really like the Gemini models for question/answer stuff and reach for it before Claude (the other model family I have purchased) but it's getting long in the tooth at this point and I'm finding my Gemini usage shrinking to nearly 0.
Anytime anything gets added to Workspace, I think Google has a lot more contractual obligations about keeping it around for X amount of time, so they tend to be more careful about adding things.
(It also shows that the internet isn't dead. Even people who are not aware of Google AI Studio can express their valuable opinions on LLMs!)
but the webui is currently offering 3.6 flash. the previous model still hasn't actually rolled out to it yet.
AI Studio? Seriously, the hell is that? Gemini, AI Studio, Antigravity - what is all that nonsense? The 3.8 Flash announcement says the model is available to Google AI Pro customers. Is it the same as Gemini Pro, or some sort of AI Studio Pro? Based on the comments, i see the model is available in the Gemini App, not available in the UI, not available to Workspace accounts but is available to some personal accounts, yet I'm not a Workspace user. Some people have already mentioned that they are paid customers, yet they don't see the new model.
I know Google loves asking graph problems during their tech interviews, but I can't wrap my head why the customers should solve these problems as well.
I have a weird vibe from all the comments in this thread, they feel like a script rather a real experience.
But yeah, they really dgaf about gemini.google.com -- I dropped that sub in April when it was clear OAI and Anthropic had lapped them
It's available in antigravity which I started using again (for small things until I can trust gemini for coding again).
(I guess the only relevance here is that if your problem matches a model's strengths, then you can do fine with a model that is several generations out of date.)
But yes, I intend to support several models, to handle the overload situation (automatic failover). And switch to a cheap paid plan, though it seems like that'll mostly improve rate limits, which barely matters for my usage.
Faster is always good, though. I do care about latency.
Do they make it intentionally to look previous model less superior than current models? or these are the real numbers when re-ran the benchmark..?
(By high volume I mean things like "main app just updated with XYZ commits, please scan XYZ plugins and surface any compatibility issues")
3.7 used 64M on high: https://artificialanalysis.ai/models/gemini-3-7-flash 3.8 used 120M on high: https://artificialanalysis.ai/models/gemini-3-8-flash
Even their own chart showed more than 2x higher cost compared to 3.7: https://storage.googleapis.com/gweb-uniblog-publish-prod/ima...
3.7 used 64M on high: https://artificialanalysis.ai/models/gemini-3-7-flash 3.8 used 120M on high: https://artificialanalysis.ai/models/gemini-3-8-flash
Even their own chart showed more than 2x higher cost compared to 3.7: https://storage.googleapis.com/gweb-uniblog-publish-prod/ima...
Flash 3.8 seems like where I can specify Flash3.8 as the coding model as part of agent workflow.
The video recognition is especially impressive as they got all of Youtube to train from.
- Def people who has to queue video recognition jobs to use the model.
Serving more models also adds a significant ops burden on the SREs and trust& safety teams.
And in my other app I was debugging and using OpenAI to optimize some path it cut me off numerous times because it did not like JIT functionality (this is my commercial business rule evaluation engine that compiles rules to executable code inside the app to increase performance using asmjit library)
I am basically paying for them to waste my tokens and time on these 2 tasks
In all seriousness, gemini has the best interactive planning document/orchestration. Tell it to create a plan document and work through it with it and it will preform really well(in antigravity products). But this is the case with plan modes with every model, I just think the interactive document that antigravity uses is really well thought out.
The reality is that if you optimise a harness for a family of tasks[1], then most of these models give successful output. And there, gemini flash's speed shines.
For general coding assistant, you want it to be well, general, and you use a harness without too much customisation to something specific. Here you need deeply post trained coding assistants and implementors like codex/sol or claude/opus. Gemini flash in its current form will be too happy-go-lucky if you try using it the way we all use codex and is better used in a constrained setting.
tl;dr gemini flash for "LLM-aided workflows in production" is super good today. Cheap as well.
[1] Stuff like this: https://antigravity.google/blog/teamwork-when-ai-becomes-a-r...
Yesterday I asked for food stop on my road trip 45 minutes from the current time and it gave me some options, but then I changed my mind and specifically asked for Asian restaurants and it completely forgot about the 45 minutes and gave me the closest Asian restaurant to me.
Once the system prompt complexity goes up, Flash starts to write very dense english. it might be fine for tasks like coding, but not for user-facing text meant to be digested by the average person.
I haven't tested 3.8 on my workload yet.
[0]: https://aibenchy.com/compare/google-gemini-3-6-flash-high/go...
I have an eval harness that runs every Thursday to determine which models are the current best for a few different client workflows. And since May(?) flash has slowly been taking over more and more stuff to the point it is now 100% on 8 out of 11 document extraction flows with the other 3 being a Flash / Opus 4.8 mix for high value stuff where cost is less of a factor.
What's the simplest explanation?
It may be that these flash models are simply post trained larger older models.
https://developers.googleblog.com/an-important-update-transi...
Gemini is great via the Chat interface and decent via Github Copilot.
I honestly hate it via Antigravity CLI because their sandboxing system frankly doesn't work. Every other harness has mastered "don't ask me if you're working in this one directory and using common commands". Agy instead either tries to pull a global elevation or wants every tedious variation of a command string whitelisted. Madness - circa 2023.
Agy _really_ needs to make the out-of-the-box experience cleaner and hassle-free. Heck, even Grok CLI "just works".
This may reflect a global mind-shift from "approve and validate everything" to "just do the stuff and only ask permission if it's outside the folder or a command that actually requires elevation". Maybe that's not for everyone, but for those that do want to perform unattended agentic work -- Agy is painful.
I just ran two light tasks on my codebase and got 100% of the weekly limits of a Pro plan blown away. Is Ultra plan any different? Because on Codex it wouldn't affect my Max plan at all, I think it would have been below 1% othese usage.
Anthropic and OpenAI are in a different game of spending investor money to buy market share.
Similarly Qwen3.8-Max was updated in just 30 days (to the 0902 release) and Muse Spark in just 28 days (to the 1.3 release).
A year ago iterative releases were every 3-6 months. At what point will they reach nightly candidates?
But it's bad at code reviews (maybe it's the harness agy cli?). Could not get it to same quality level on reviews like Opus, GPT 5.6, Grok. Even tried special code review skills but no luck.
I switched my subagent-swarm skill(https://github.com/bazelment/yoloswe/blob/main/.claude/skill...) to the 3.8 model and has been very happy so far. It has been driving the swarm to produce steady outcome, and more importantly, the exec communication is also crystal clear, instead of filling with jargons and long sentences.
Nah, we'll just get the 4.0 Pro Preview.
As much as I like the speed and interactivity, I really don’t trust it
[1] For tone and instruction following, a positive percentage increase represents an improvement in the tone of the model on sensitive topics and the model’s ability to follow instructions while remaining safe compared to Gemini 3 Flash. We mark improvements in green and regressions in red.
Why bother with a column “Gemini 3.8 Flash vs. Gemini 3.7 Flash” when you’re going to disregard the label for 20% of it? Also is the “Tone” label short for “Tone and Instruction Following”?
Chartcrime, the major AI lab tradition.
Calling it cyber is like shortening email to "e".
If you're of a certain age, "cyber" primarily means cybersex. There's also cybernetics, cybercafes, and more: https://en.wikipedia.org/wiki/Internet-related_prefixes#%22C...
I think the worst thing we can do is have loyalty towards models. I used to be loyal towards Claude, and my viewpoint changed dramatically when I used codex.
I highly recommend that if you are someone who only used one model so far, that you really give another model a shot and see how it goes. It's very eye opening and gives you a more holistic perspective.
Vendor locking is a big problem when it comes to models, and I hope the software world doesn't do this blindly.
Almost suspect that the rate of improvement to post-training is so fast that small models have an advantage - it takes much more compute to train a bigger model, so the flash models are just running in circles (well, not exactly of course) around the larger models right now.
1 what is tesla cybercab plan to address legal implications of accident that will happen? who is going to be responsible for them when they happen? are they covered by tesla insurance or some other insurance? are there any official plan/statements around that?
2 what was the name of the experiment they started in san antonio tx when some cars didn't have a driver? what was the results of it? did they expand the operations? it was much smaller than waymo, is it growing? how it is related to robotaxi?
It is not able to connect the dots that I keep asking about Tesla in 2nd prompt and spit out some unrelated stuff. Really? How it can be that bad? Gemini 3.1 Pro model works fine in this case btw. I thought maybe it is about knowledge cut over date and it doesn't know about those events from 2025 but it seems it has the knowledge up to March 2025. Top 10 in Intelligence on artificialanalysis ladies and gentlemen.
They're apparently evolving slower than most SOTA models but "slow and steady wins the race" is probably still a thing.
And since Google doesn't depend exclusively on AI models, they can probably afford to "wait and see" where all this craze is heading.
I don't think slow and steady will win this race, but I think anyone can still win--especially Google.
It still uses 3.6 Flash for example.
The Google brand remains powerful on HN!
I’m shocked.
But a good agents.md, starting from a clean slate, and specifying which key files to look into and follow the standards allows me to build gigantic projects even I struggle to keep in my head structurally.
I would say either start new sessions for new tasks or limit the context to something smaller than 1M.
I usually start with research/planning session, this goes into a detailed implementation plan and then a new session for the actual implementation.
If it's complex problem maybe a review/adversarial step between plan and implementation.
Also with forever-session any time you take a longer break (depends on model and provider as to how long) you will push an entire big context again without caching even if you don't need it. With 1M context this gets expensive.
I am continually impressed with Gemini's chat responses, which encourages me to test their agentic capabilities and... no... no... and no... every single time.
It's terrifying watching it, really.
so its fast sure and decent at non coding usage but for developers nothing can really top sol or fable.
even grok 4.6 is so so and i would not choose 3.8 flash over it.
So just another mid-tier flash model, nothing exciting, but Antigravity has very generous quotas so it's a good workhorse model when your Claude/OpenAI subs run out.
And whilst it's a fast model, having to baby sit through and approve prompts every few seconds ends up making it slower than the Auto approve modes of Claude/ChatGPT - they definitely need an auto approve mode.
i think luna-max has the best cost value offer when it comes to coding, but i note the multi modality of gemini flash as a win
i might consider 3.8 flash for simple side hobby projects or quick scaffolding but would not trust it for long agentic tasks, that really is the realm of sol/fable
agy cli still has a lot of issues not sure if its due to the underlying model hallucinating or the harness or both
My guess is we skip 3.5 and go straight to 4 Pro. With the monthly Flash releases, releasing 4.0 Flash and Pro in 6-8 weeks would be a nice buildup.
(I work at Google but don't know anything that isn't already public)
If you just need to create PDFs, not edit them, then Gemini notebook (notebook.google) works well and has Google's usual very high free usage limits.
AFAIK in general you can't really edit PDFs since it's not a reflowable format - even with Adobe tools all that editing does is modify the text within a text box - not reflow the document to adjust to any change in size of the text box.
A recent example - I searched for how to unsubscribe from Pearson emails. Google Search "AI Mode" confidently gave me a sequence of steps along the lines of Settings > Profile > Email preferences > Unsubscribe.
Of course, I looked for an unsubscribe link before asking Google. None of those options existed. The correct answer was there is no way to unsubscribe through the account, so I just blockthe emails instead.
I've run into this pattern quite a few times. AI Mode seems to make up things all the time.
>We will not send marketing emails to a user who has opted out of receiving them. Any marketing communications we send will include an unsubscribe link at the end of the email.
I don't think this is AI's fault. This is Pearson's publishing incorrect information and the only way to really know they are a bunch of lying assholes is to have an account and try to unsubscribe from it.
AI didn't make it up, Pearson's did.
It's obvious that the Google Search AI Mode encourages the model to give an answer without spending unnecessary cycles investigating deeply.
They also heavily encourage keeping the context short. For example, it will remove the option to start a new turn after a small number of turns, depending on the topic.
It definitely makes things up all the time, but it gets it right surprisingly often. I really like it.
Are you people truly this excited about this crap? I mean I guess if you work for Google or Anthropic or whatever I could see it??? Otherwise, are these just bot comments?
A lot of us use these in our services, so we're getting an upgrade "for free"
Also, I use it every day, and it just got ~10% better at coding, according to the benchmarks. How is that not exciting?
I do think Gemini is underrated on HN though!
I'll wait for Astra and Grok 4.7 announcements but probably getting at least one Ultra subscription.
Since testing 3.7 on Pro for last two weeks I'm realizing just how long I'm waiting on other models. I've been multitasking to compensate but it's exhausting so I'd rather not.
HN sells these advertising services. Nobody is using “Claude” etc.
They will censor comments like yours and my reply here because we call it out.
It’s very weird that basically lies and disinformation became the optimal meta in business and in life! But here we are
Even making a tiny joke is too much [1] for some.
> They will censor comments like yours and my reply here because we call it out.
Don't bother calling it out, it does not work. There are protected accounts where the guidelines don't apply to them and moderators allow this and ban others who do the same thing. [2]
It is pointless, and HN is cooked for this.
[0] https://news.ycombinator.com/item?id=49521145
I am a light user so I don't get the shakes when my monthly azure dev credits run out but I would be susceptible to being addicted to it if I was on a subscription with generous usage allowance and random usage counter resets.