AI Burns Up to 180 Times More Energy on Non-English Text

Generative AI uses more energy on non-English text

AI Burns Up to 180 Times More Energy on Non-English Text

University of Michigan researchers measured three open-source models—Alibaba's Qwen 3, Google's Gemma 3, and Meta's Llama 3.1—and found non-English prompts cost 1.4 to 180 times more energy, with Hindi at 7.9x and Bengali at 15.3x English. The cause: non-Latin scripts need more tokens, and accuracy often drops too. Roughly 5.2 billion people face this double penalty, which could price them out of AI.

The languages that typically consume more energy also tend to have lower accuracy in the model output.
  1. esperent

    You have to read about half of the article before it mentions this, which should have been in the first paragraph:

    > The researchers found the energy and accuracy disparity by measuring the amount of energy used by three open-source AI models, Alibaba’s Qwen 3, Google’s Gemma 3, and Meta’s Llama 3.1.

  2. vova_hn2

    I think that switching from fixed token sets to something like H-Nets [0] (discussed on HN [1]) can at least improve on this issue, if not solve it completely.

    Because, right now the main issue is that some languages are tokenized much more efficiently than others and it is a zero sum game, since there are a limited number of tokens and by adding tokens that are needed for one language you will inevitably have to remove tokens for another language and therefore decrease efficiency for it.

    [0] https://www.cartesia.ai/blog/from-tokens-to-bytes-part-1

    [1] https://news.ycombinator.com/item?id=50278253

  3. akssri

    > When working in other languages, the models used more energy because they had to generate more tokens...

    Seems a bit dated. Sarvam has been taking about the inefficiency of the tokenization schemes used by mainstream models when it comes to Indic-languages for years now.

  4. NoahZuniga

    There's some interesting info here, but the presentation is pretty poor. The worst languages are where the AI gets confused and loops keeps repeating the same text a bunch of time.

    From the table on page 6 [1]

    > Token explosion. The sentence repeated 45 times.

    Figure 4/table 3 is pretty interesting. It shows that essentially all languages clustered at 0.1J/token, with a few outshots up to in some cases 4.8J/token.

    They have 900 runs per language, so that's a pretty good sample size.

    [1]: https://aclanthology.org/2026.emnlp-main.2113.pdf

  5. pona-a

    This should be an obvious corollary. Transformers scale exponentially with context tokens, and it was well-known current tokenizers perform worse on non-English text, splitting it into more tokens.

  6. scottcha

    I was able to pretty easily reproduce parts of this paper at Neuralwatt Cloud's SoA models where we do all our optimizations and pricing by energy.

    Accuracy uses the paper's strict "#### N" parser and lenient fallback. acc is strict/lenient percent. len_cap is the number of responses truncated at the generation-length limit, not an energy cap. Each language has 50 responses.

    nw-flash (DeepSeek V4.1 Flash 1017)

    Language J/req xEN tok/req J/tok acc len_cap

    English 23.4 1.00 174 0.134 100/100 0

    Chinese 16.8 0.72 389 0.043 96/96 0

    Russian 18.8 0.80 352 0.054 96/96 0

    French 25.1 1.07 272 0.092 98/98 0

    S. Pashto 69.2 2.96 752 0.092 92/92 1

    Tigrinya 65.3 2.79 2299 0.028 0/76 8

    Shan 98.5 4.21 2823 0.035 48/54 14

    Tibetan 42.3 1.81 2227 0.019 0/76 9

    nw-small (Qwen3.8-27B-FP8)

    Language J/req xEN tok/req J/tok acc len_cap

    English 62.2 1.00 300 0.207 100/100 0

    Chinese 154.1 2.48 605 0.255 94/94 1

    Russian 147.7 2.37 576 0.257 96/96 1

    French 163.3 2.62 323 0.506 98/98 0

    S. Pashto 777.2 12.49 1510 0.515 84/84 6

    Tigrinya 823.8 13.24 2458 0.335 0/76 10

    Shan 1407.4 22.62 3422 0.41 […]

  7. PhilipRoman

    I didn't read the full PDF, but at least in the linked article, I think the number of native speakers as a metric penalizes English (and French), making them look more efficient in the graphs. There are about 3x more non-native speakers than native, and vast amounts of training material has been written by them.

  8. load-bearing

    This reminded me of this link, which I've just posted to HN

    https://partyfly.github.io/tokenizer-fairness-2026/

    Already been using English to conserve tokens, and then translate the final result to other languages if necessary. I think this was already obvious to most international LLM users proficient in multiple languages.

More from this day

2026-11-14