Generative AI uses more energy on non-English text
news.umich.edu
news.umich.edu
> The researchers found the energy and accuracy disparity by measuring the amount of energy used by three open-source AI models, Alibaba’s Qwen 3, Google’s Gemma 3, and Meta’s Llama 3.1.
Edit: sorry, i went straight to the paper. I agree, the article itself (and the headline) are bad (as they tend to be on this topic)
Working out the kinks in your methodology obviously takes some time. But is there any structural reason not to rerun the benchmarks on current models in the final stages before submission? Assuming the results generalize to those newer models
I am hopeful that we can eventually move to open weights models, but at the time we did the experiments, the open weights models did not have the capabilities we needed. The ever-shifting sands of frontier models makes reproducibility difficult, and not just for scientists.
I did a PhD and the answer is yes. Often longer than this.
Given the same model the only variable is the token count. The more tokens, the more passes through the LLM (to generate the next output token), the more energy used.
It's well known that non-English text consumes more tokens [1], [2] and thus more energy. This is due to two factors:
1. Latin script languages having different character pairings to English and modified characters, including language features like modifying verbs for person/tense/etc. (e.g. Spanish);
2. Non-Latin scripts that take more bytes to represent in UTF-8 and (esp. for ideographic and syllabic languages like Chinese, Japanese, and Korean) have lower common character pair sequences, meaning shorter tokens for those scripts and thus more tokens for a given text sequence.
[1] https://ikriv.com/blog/?p=5322 "LLM tokens and foreign languages" (5 Aug 2025)
[2] https://medium.com/@programmerraja/how-your-llm-costs-5x-mor...
It is not exactly accurate, because it ignores speculative decoding which can often save a lot of time, compute and energy.
See this branch: https://news.ycombinator.com/item?id=50441444#50497854
I suspect that it still ends up being cheaper in English, but parent's point that speculative decoding helps is valid.
The reason I think speculative decoding would scale the same is that the smaller predictive model has the same token behaviour as any other model compared to the other models. So the only savings would be if the smaller model can better predict more tokens for non-English compared with English such that the ratio of tokens between the two becomes the same or lower for the non-English language.
For Latin-script languages like Spanish I can see the smaller model predicting verb endings correctly, improving performance there. These languages have 1.5x-2x more tokens [1] so will need to be 1.5x-2x better than English in the speculative decoder for the performance of the non-English languages to be comparable.
For Han characters (Chinese and Japanese) it looks like the tokenizer is splitting the characters in the middle of UTF-8 byte sequences, at least in [1] where the number of tokens is larger than the number of Han characters. This would potentially make the paired tokens harder to predict by the smaller model. Given that there are ~2 Han tokens per English token the speculative decoding would need to be 2x better for Han than for English to make it perform the same compute energy wise.
If we assume that the smaller decoder model is perfect and that there is no additional cost on the verifier then the English vs non-English languages are going to scale the same on the smaller model, so speculative decoding would just be like using that smaller model and the non-English languages would perform worse due to them having more tokens that need to be processed by the smaller model.
The other factor is the context length of the transformers in the verifier. The question then is how quickly does that fill up and how many passes through that are needed for English (fewer tokens) compared to non-English (more tokens).
So I have to wonder why is token count so high in their examples? Here is my theory: I would not say 我遇见了一只大狗, but I would just say 遇大狗. I know I can't do that in English, but I think surely the researchers should know (or would ask someone?) how this works trying to compare phrases in isolation, so I wonder why is this test meaningful to them? Why are they doing that? Are they looking for this result? 字省思臊 I don't talk like google translate.
So who are these people and why are you listening to them and repeating what they say like it is so? Why exactly are you so convinced?
Because, right now the main issue is that some languages are tokenized much more efficiently than others and it is a zero sum game, since there are a limited number of tokens and by adding tokens that are needed for one language you will inevitably have to remove tokens for another language and therefore decrease efficiency for it.
[0] https://www.cartesia.ai/blog/from-tokens-to-bytes-part-1
However, they (Cartesia) haven't yet published a technical paper on their new version of H-Net, and it is unclear if they will.
Even if you use bytes only, Unicode is biased towards English and other Latin languages, too.
I don't know whether European languages work inherently better with tokenizers, but I don't see why they would.
but yeah, anything alphabetic yet not latin gets the worst tradeoff.
Not really, because I don't think anyone use the whole training dataset to train the tokenizer.
From my brief (disclosure: claude-assisted [0]) internet search I learned that typically they use from tens to hundreds GB of data to train the tokenizer. Sometimes it is hand-curated, sometimes it is randomly sampled.
Of course, you can deliberately choose how much text in each language do you include in the training set for the BPE and deliberately optimize for (or against) a specific language.
[0] https://claude.ai/share/da277189-f23a-4071-bd9f-2d75fe6aa779
Also, it is non-EU, so initiatives like [0] will only indirectly help with representation (Croatian being the same* language, but it does not use cyrillic script)
* Contentious point maybe, but they are mutually intelligible with what would otherwise be considered only dialectical variations
[0] https://digital-strategy.ec.europa.eu/en/news/commission-wel...
[1] https://lifthrasiir.github.io/aqueous/ (specifically wrt-recons-hangul codec)
From the table on page 6 [1]
> Token explosion. The sentence repeated 45 times.
Figure 4/table 3 is pretty interesting. It shows that essentially all languages clustered at 0.1J/token, with a few outshots up to in some cases 4.8J/token.
They have 900 runs per language, so that's a pretty good sample size.
Accuracy uses the paper's strict "#### N" parser and lenient fallback. acc is strict/lenient percent. len_cap is the number of responses truncated at the generation-length limit, not an energy cap. Each language has 50 responses.
nw-flash (DeepSeek V4.1 Flash 1017)
Language J/req xEN tok/req J/tok acc len_cap
English 23.4 1.00 174 0.134 100/100 0
Chinese 16.8 0.72 389 0.043 96/96 0
Russian 18.8 0.80 352 0.054 96/96 0
French 25.1 1.07 272 0.092 98/98 0
S. Pashto 69.2 2.96 752 0.092 92/92 1
Tigrinya 65.3 2.79 2299 0.028 0/76 8
Shan 98.5 4.21 2823 0.035 48/54 14
Tibetan 42.3 1.81 2227 0.019 0/76 9
nw-small (Qwen3.8-27B-FP8)
Language J/req xEN tok/req J/tok acc len_cap
English 62.2 1.00 300 0.207 100/100 0
Chinese 154.1 2.48 605 0.255 94/94 1
Russian 147.7 2.37 576 0.257 96/96 1
French 163.3 2.62 323 0.506 98/98 0
S. Pashto 777.2 12.49 1510 0.515 84/84 6
Tigrinya 823.8 13.24 2458 0.335 0/76 10
Shan 1407.4 22.62 3422 0.411 48/52 20
Tibetan 1455.5 23.39 3199 0.455 0/56 20
nw-large (Kimi K3.1 Pro)
Language J/req xEN tok/req J/tok acc len_cap
English 169.5 1.00 391 0.434 100/100 0
Chinese 113.8 0.67 491 0.232 98/98 0
Russian 78.7 0.46 527 0.150 98/98 0
French 298.3 1.76 406 0.735 98/98 0
S. Pashto 119.7 0.71 903 0.133 96/96 0
Tigrinya 1096.6 6.47 1870 0.586 38/86 2
Shan 503.6 2.97 2895 0.174 52/52 12
Tibetan 593.2 3.50 2298 0.258 0/82 4
Across the four low-resource versus four high-resource languages: Endpoint Output tokens Energy/request J/output token
nw-flash 6.83x 3.27x 0.48x
nw-small 5.87x 8.46x 1.44x
nw-large 4.39x 3.50x 0.80x Language Accept Tokens/step 0/3 steps 3/3 steps Output tok/req
English 76.53% 3.296 9.4% 62.8% 549
Chinese 73.37% 3.201 11.3% 58.2% 754
Russian 73.88% 3.217 11.1% 58.5% 739
French 75.50% 3.265 9.8% 60.9% 631
S. Pashto 70.44% 3.113 12.9% 54.5% 1332
Tigrinya 79.72% 3.391 8.2% 67.8% 3643
Shan 75.64% 3.269 11.2% 63.0% 3675
Tibetan 74.67% 3.240 10.3% 60.2% 2857
Overall 75.75% 3.273 10.3% 62.2% 1772 Language Strict Lenient Cap Tok/req J/req J/tok Req x EN Tok x EN J/tok x EN
English 100% 100% 0 548.7 230.33 0.4198 1.00 1.00 1.00
Chinese 96% 96% 1 754.0 694.47 0.9210 3.02 1.37 2.19
Russian 96% 98% 2 738.8 592.15 0.8015 2.57 1.35 1.91
French 100% 100% 0 631.0 1101.84 1.7462 4.78 1.15 4.16
S. Pashto 88% 98% 6 1332.1 208.06 0.1562 0.90 2.43 0.37
Tigrinya 56% 62% 19 3642.9 1445.84 0.3969 6.28 6.64 0.95
Shan 44% 54% 27 3674.8 3791.59 1.0318 16.46 6.70 2.46
Tibetan 2% 70% 15 2856.9 5306.68 1.8575 23.04 5.21 4.42I can imagine that in some cases the acceptance rate for non-English responses may be much higher, because while words on average need more tokens to spell, it is more "obvious" what the next token should be, so it is easier for the speculation model to predict it accurately. Information density per token is lower.
Or, maybe the acceptance rate is much lower, if the speculation model is tuned for English and doesn't work well with other languages.
https://partyfly.github.io/tokenizer-fairness-2026/
Already been using English to conserve tokens, and then translate the final result to other languages if necessary. I think this was already obvious to most international LLM users proficient in multiple languages.
It would be interesting to see if it would make any difference if you posttrained with translations of the same texts, i.e. focusing on teaching the model to specifically translate concepts between languages.
But I would subtly add it's also about the training data, which is optimized towards English. Not intentionally, just incidentally. (And this is a really hot area of improvement with a long way to go.)
Which is largely why the tokenization part that you mentioned is what it is.
I think you only start getting seriously bad impacts in non-Romance languages where they have their own alphabet, or use the English alphabet phonetically causing letters to appear next to each other that never would in English. E.g. in English "qu" is a pretty obvious token, you rarely see a Q without a U after it. Happens all the time in anglicized Middle Eastern words, though. Quran is like a whole other thing, you'd almost always expect another vowel after qu, not a consonant.
I don't know how true that is, and I can't imagine knowing more than one language. I'm impressed by anyone who knows more than one....
Tokeniser Everyday JP/EN Technical JP/EN
GPT-3 2.69x (183/68) 3.48x (233/67)
GPT-4 2.16x (147/68) 2.53x (167/66)
GPT-5.x 1.49x (101/68) 1.74x (115/66)
It shows that the gap clearly has shrunk a fair bit with Japanese going from 2.7x English on GPT-3 to 1.5x on GPT-5.x for everyday text. The same passage of text yielded flat results for English across each tokeniser, so all the improvement went into Japanese. I also found that the Japanese polite form used to cost extra on GPT-3 but is now effectively free. The polite version of the Japanese passage was 9 characters longer (133 vs 124), and contained endings such as ました, ません and でした which have all become single tokens in newer tokenisers. Perhaps most interestingly, katakana loanwords seemed to add a lot of extra cost. For example, a word like パスワード(password) is split into four tokens (パ / ス / ワ / ード), while "password" is one token in English. In this example, the exact katakana string likely did not appear frequently enough in the training data, unlike the common hiragana clusters such as the polite endings, and the English word “password”, which earned their own tokens. The newer tokenisers continue to split less common kanji (like 靴 and 濡) into byte pieces, but katakana stands out as even everyday words are broken apart. It'd be interesting to run these checks on the same Qwen, Gemma, Llama tokenisers the study uses.LLMs do what they say on the tin: they model language. If you ask an LLM "find security vulnerabilities in my code" for hello_world.rs, the next likely tokens are "I found the following vulnerabilities", regardless of what logic or reason say.
It happens to be true that predicting next tokens on your own output mocks actual reasoning fairly well. But it still just a simulacrum rather than the real thing, and when held as an abstraction it leaks everywhere.
Understanding this is critical to avoiding the common failure case of scope creep and code slop.
Trying to pretend otherwise at this point doesn’t help us to utilize them or understand their limits. We’re better off accepting that intelligence isn’t unique to us as a species. Then we can see that modeling language as a tool for us as a vehicle for thinking and reasoning, which philosophers like Wittgenstein already landed upon before LLMs.
As an actual example, imagine an LLM harness that provides execution approval modes. These are famously leaky (and intractable to fix), but serve to provide some modicum of control.
Now imagine that you want to integrate sandbox detection so that you can set different approval modes depending on whether the sandbox is active. You could detect env variables and check for things like seccomp to infer trust state in a moderately robust manner.
Throw this through the LLM (any of the latest ones) for a security review and it will flag that the user can pass in envs to fake stuff, and the user can modify the binary or settings to bypass things, and they could fake some of the other signals too.
All of those are fake findings and is conceptually alike the `hello_world.rs` example, even if that particular issue has been fine-tuned away. A child process of equivalent-or-lesser-privilege cannot fundamentally prevent its parent process from gaslighting it, so either the feature is untenable or its purpose is NOT ironclad robustness against a hostile operator (which an LLM harness cannot defend against). But the next-token nature of an LLM almost demands a response of "findings", and those findings will follow the common patterns of such reviews even when they're fundamentally mis-premised.
Seems a bit dated. Sarvam has been taking about the inefficiency of the tokenization schemes used by mainstream models when it comes to Indic-languages for years now.
Just because you speak a language, doesn't necessarily mean it's your preference all over the place. Or let's say you're Dutch and you live in the US. Your OS is in English, and imagine if telegraaf.nl auto-switched to English.
(**) The web and apps are full of less than ideal translations. And AI has actually worsened it.In other words, the reasoning required to fluently navigate particular linguistic structures si transferable to other domains, suggesting that native default and subsequent acquisition choices can have significant positive or negative impact on different cognitive tasks. These weren't exclusive to particular geographic regions.
Genuinely curious, not sacastic question below.
Is this one of those posts like "eating more sugar causes bad health" type of post because most of info on internet is in English? Or a new insight people was not aware of?
why not to test different language with similar and repeated tasks?
The language inter-operability is improving every movement for many models.
Seems not to be the case given Arabic’s energy score.
I read an article once that talk about this the first few layers are like a translation layers then middle is thinking and the output layers translate it back.
It’s pretty interesting actually, you can cut out the middle layers and stack them back to back and that increases the models performance without much memory increase.
If you want to try it out you can get a LLM with no tools and pass it base64 and it will understand what’s in the message
If I ask something in my native language (Korean), the model often switches into English in the middle and translate the answer into Korean.
But not sure about what happens below the token, in embedding space or per decoder layers.
Current open source LLMs have an emergent structure where the first couple layers transform the input into some internal representation. Then the middle does multiple reasoning steps, and the end transforms it back to one output token. You can repeat the middle to get a stronger, slower model, so it is abstract reasoning. And Anthropic had a blog post about actually looking at what the middle does and mapping it back to concept for inspection
With that in mind, what's the point of reasoning tokens? They throw rich internal state away, for the benefit of inspectability and variable-length reasoning. But now that we understand the internal structure better we can dynamically vary how many loops the model spins through those middle layers, and still get enough insight into what's happening by decoding internal state. This takes some reengineering of model architecture and inference engines (especially to still get good batch performance), but the potential benefits seem to big to ignore
(Longer explanation though: I don't think german per se sucks, but english is in many ways much simpler than other languages, excluding the fact that english has more words, e. g. more expressions in general. German is more convoluted, you have "der", "die", "das"; in english you just have "the". There are also super-long words in german; you don't have that in english usually. One of my favourite is "Donaudampfschiffahrtsgesellschaftskapitän" - the semi-literal translation is, sort of, a "marine" captain on the danube river, as part of an incorporation/company. See that wikipedia article, but don't buy it 100%, since nobody is really using that word anyway; it is just a demo of what monstrous word combinations may exist in german: https://de.wikipedia.org/wiki/Donaudampfschiffahrtsgesellsch...)
So I think the title should be "more energy on more complex languages", because I think this would be somewhat more correct; as well as most likely the fact, that there are more english texts than non-english (singular) texts as example. You can see this on Wikipedia, where the english wikipedia is miles ahead of most other languages too. More people use english and I bet there are more english texts too. And Generative AI will prefer english over other languages in general.
People keep saying that sort of thing but we have a similar thing in Norwegian where we are allowed to combine words to make a new “word”. But people who use other languages that don’t do this sort of combining of words misunderstand the (in)significance of combined words.
Take for example menneskerettighetserklæringen. This is the name we use in Norwegian for the Universal Declaration of Human Rights. Menneskerettighetserklæringen, wow what a long and strange word right? No. It is formed from three words in sequence, “menneske” (human), “rettighet” (right), “erklæring” (declaration).
It’s just a quirk of Norwegian and German and other languages that do the same, that we consider that we should removing spaces between words like this. But there is no actual meaningful difference between a Norwegian saying hey guys we came up with this thing and we’re gonna call it “menneskerettighetserklæringen”, and an English speaker saying the same thing about the “Universal Declaration of Human Rights”. Only that we remove the spaces and you don’t.
The named thing gets its dedicated Wikipedia article in both languages all the same:
https://no.wikipedia.org/wiki/Menneskerettighetserkl%C3%A6ri...
https://en.wikipedia.org/wiki/Universal_Declaration_of_Human...
Also: Gemma 3, Llama 3.1, come on…
Cynical side note: re-running your tests with newer experimental models and updating your findings is usually good for a poster at a scientific conference
For spoken language, French and English are at the top, but most languages are about the same bits/second. Languages with less information density are just spoken faster to make up for it.
Ban any language other than English from prompting.
This has multiple benefits - first you restrict third world from spamming USA data centers. Why would America pay for some guy in India using free model?
This alone is worth considering but also you can optimise LLMs further for a single language as opposed to having to translate and couple weights for every single concept.
Why would American LLMs support Chinese language or Arabic? Make them learn English and come to work for us. No more free stuff for the third world.
Because the US government subsidizes all internet companies to establish a monopoly over everything on the internet, like Google, Microsoft, Facebook, Reddit, ...
Get better or get lost.
Next time, ask that AI what text on the internet means so you no longer miss the point. Your brain is outdated.
> some guy in India
Aren't they bilingual in English?
> the third world
Isn't there a second world as well?
> Why would American LLMs support Chinese
Aren't there American companies that want to use LLMs for translation tasks or to provide their LLM-powered services in other languages? Spanish anyone?