Generative AI uses more energy on non-English text

(news.umich.edu)

231 points | by DeepLogin 5 days ago

35 comments

  • esperent 1 day ago
    You have to read about half of the article before it mentions this, which should have been in the first paragraph:

    > The researchers found the energy and accuracy disparity by measuring the amount of energy used by three open-source AI models, Alibaba’s Qwen 3, Google’s Gemma 3, and Meta’s Llama 3.1.

    • fxwin 1 day ago
      I feel like that info is placed appropriately. The abstract mentions that energy disparity persists across models, the rest follows the standard Intro -> Related Work -> Experiments structure

      Edit: sorry, i went straight to the paper. I agree, the article itself (and the headline) are bad (as they tend to be on this topic)

    • wongarsu 1 day ago
      Is the publishing cycle this long? Those models were relevant 18 months ago

      Working out the kinks in your methodology obviously takes some time. But is there any structural reason not to rerun the benchmarks on current models in the final stages before submission? Assuming the results generalize to those newer models

      • raddan 1 day ago
        It's challenging to keep running experiments. In a perfect world, we would run our experiments again and update the results. But some experiments are expensive, and some experiments are inherently difficult to reproduce. I have a paper nearing completion that involves interaction between trained (human) professionals and a model. It was a logistical challenge to bring those people together (also, we had to pay them), and the roleplaying that we asked LLMs to do can't be done anymore--they refuse for safety reasons. Stuff like this is pretty common.

        I am hopeful that we can eventually move to open weights models, but at the time we did the experiments, the open weights models did not have the capabilities we needed. The ever-shifting sands of frontier models makes reproducibility difficult, and not just for scientists.

      • armcat 1 day ago
        Unfortunately it is this long. I know people in academia studying human-AI interfaces and up until recently they were still using gpt-3.5.
        • mh- 1 day ago
          Yes and let's all remember that lag when the studies about AI outcomes are posted.
      • huijzer 18 hours ago
        > Is the publishing cycle this long?

        I did a PhD and the answer is yes. Often longer than this.

    • curt15 1 day ago
      How would one evaluate the energy consumption of models accessible only through Anthropic or OAI APIs? You can't fire them up on your own computers and measure power consumption.
      • rhdunn 1 day ago
        The Anthropic and OAI APIs report token usage (input, output, and in some cases reasoning/thinking). The input tokens consist of the system prompt, the user input/messages, and any other context data.

        Given the same model the only variable is the token count. The more tokens, the more passes through the LLM (to generate the next output token), the more energy used.

        It's well known that non-English text consumes more tokens [1], [2] and thus more energy. This is due to two factors:

        1. Latin script languages having different character pairings to English and modified characters, including language features like modifying verbs for person/tense/etc. (e.g. Spanish);

        2. Non-Latin scripts that take more bytes to represent in UTF-8 and (esp. for ideographic and syllabic languages like Chinese, Japanese, and Korean) have lower common character pair sequences, meaning shorter tokens for those scripts and thus more tokens for a given text sequence.

        [1] https://ikriv.com/blog/?p=5322 "LLM tokens and foreign languages" (5 Aug 2025)

        [2] https://medium.com/@programmerraja/how-your-llm-costs-5x-mor...

        • vova_hn2 1 day ago
          > token usage

          It is not exactly accurate, because it ignores speculative decoding which can often save a lot of time, compute and energy.

          See this branch: https://news.ycombinator.com/item?id=50441444#50497854

          • rhdunn 1 day ago
            Speculative decoding will be model/software based and apply to all languages. The time, compute, and energy will be less compared to other models, but for the same model, hardware, and software used my reasoning still applies -- comparing English to other languages (unless optimized by the tokenizer) then English will consume less tokens and thus less energy. That is, it is still a function of the tokens used.
            • make3 22 hours ago
              Your counterpoint is incorrect. The savings of speculative decoding are a direct function of how hard it is to predict the next token. Which means that for a specific task, the smaller model can be used more in a language which uses more tokens to say the same thing (to perform the same task), as the difficulty of the task is fixed between languages and the language that takes more tokens "spreads the same difficulty" over more tokens.

              I suspect that it still ends up being cheaper in English, but parent's point that speculative decoding helps is valid.

              • rhdunn 16 hours ago
                You may be right, but I'd need to see the data and analysis of the number of passes through each stage for the various languages.

                The reason I think speculative decoding would scale the same is that the smaller predictive model has the same token behaviour as any other model compared to the other models. So the only savings would be if the smaller model can better predict more tokens for non-English compared with English such that the ratio of tokens between the two becomes the same or lower for the non-English language.

                For Latin-script languages like Spanish I can see the smaller model predicting verb endings correctly, improving performance there. These languages have 1.5x-2x more tokens [1] so will need to be 1.5x-2x better than English in the speculative decoder for the performance of the non-English languages to be comparable.

                For Han characters (Chinese and Japanese) it looks like the tokenizer is splitting the characters in the middle of UTF-8 byte sequences, at least in [1] where the number of tokens is larger than the number of Han characters. This would potentially make the paired tokens harder to predict by the smaller model. Given that there are ~2 Han tokens per English token the speculative decoding would need to be 2x better for Han than for English to make it perform the same compute energy wise.

                If we assume that the smaller decoder model is perfect and that there is no additional cost on the verifier then the English vs non-English languages are going to scale the same on the smaller model, so speculative decoding would just be like using that smaller model and the non-English languages would perform worse due to them having more tokens that need to be processed by the smaller model.

                The other factor is the context length of the transformers in the verifier. The question then is how quickly does that fill up and how many passes through that are needed for English (fewer tokens) compared to non-English (more tokens).

                [1] https://ikriv.com/blog/?p=5322

        • geocar 12 hours ago
          You say this is well-known, but my experience is talking chinese to the chinese LLM produces more efficient work, with less rambling in the thinking/reporting steps, and I have been presuming that it is because longer utf8-token still beats a bunch of utf7-compatible words for expressing the kinds of big ideas maths and code requires. 省字臃思

          So I have to wonder why is token count so high in their examples? Here is my theory: I would not say 我遇见了一只大狗, but I would just say 遇大狗. I know I can't do that in English, but I think surely the researchers should know (or would ask someone?) how this works trying to compare phrases in isolation, so I wonder why is this test meaningful to them? Why are they doing that? Are they looking for this result? 字省思臊 I don't talk like google translate.

          So who are these people and why are you listening to them and repeating what they say like it is so? Why exactly are you so convinced?

        • gavinsyancey 1 day ago
          So basically, the tokenizer is optimized for English and performs worse on other languages?
          • dosinga 1 day ago
            roughly the tokenizer is optimized such that the training data (for the tokenizer) can be represented with the smallest number of tokens. and yes, English tends to be overrepresented in training data.
      • simonw 1 day ago
        Look at the token usage counts.
      • yolo1897 1 day ago
        It's explained too..
  • vova_hn2 1 day ago
    I think that switching from fixed token sets to something like H-Nets [0] (discussed on HN [1]) can at least improve on this issue, if not solve it completely.

    Because, right now the main issue is that some languages are tokenized much more efficiently than others and it is a zero sum game, since there are a limited number of tokens and by adding tokens that are needed for one language you will inevitably have to remove tokens for another language and therefore decrease efficiency for it.

    [0] https://www.cartesia.ai/blog/from-tokens-to-bytes-part-1

    [1] https://news.ycombinator.com/item?id=50278253

    • cubefox 1 day ago
      Exactly. Currently LLMs use a fixed tokenizer optimized for English and (to a lesser degree) Western languages. The H-Net architecture doesn't use a fixed tokenizer, it uses dynamicaly adaptive chunks of bytes, and (in a two-stage setup) chunks of chunks of bytes.

      However, they (Cartesia) haven't yet published a technical paper on their new version of H-Net, and it is unclear if they will.

      • brainwad 1 day ago
        It's not optimised deliberately for English, right? It's just that byte pair encoding naturally favours English and other Latin languages because they are dominant in the training data.

        Even if you use bytes only, Unicode is biased towards English and other Latin languages, too.

        • Phemist 1 day ago
          Ive always thought that Serbian is the worst off out of all European languages. Both cyrillic and latin script are widely used. Latin also has a naked variant, where all diacritics are left out (which is especially prevalent online). So all Serbian text is in a three-way split between these scripts.

          Also, it is non-EU, so initiatives like [0] will only indirectly help with representation (Croatian being the same* language, but it does not use cyrillic script)

          * Contentious point maybe, but they are mutually intelligible with what would otherwise be considered only dialectical variations

          [0] https://digital-strategy.ec.europa.eu/en/news/commission-wel...

        • numpad0 1 day ago
          Aren't tokenizers a hack to turn Latin alphabet texts into a pseudo Chinese with better per-unit information density?
          • brainwad 1 day ago
            It is desirable for all alphabets/abjads/abugidas/syllabaries, not just Latin. Chinese is rather the special case of a human language with very high character inventory and hence low character/word ratio.
        • sznio 1 day ago
          logographic languages still win I guess, a 4-byte chinese character beats a n-character English word for that same concept.

          but yeah, anything alphabetic yet not latin gets the worst tradeoff.

          • brainwad 1 day ago
            Most CJK characters are 3 bytes in UTF-8. But then most English words translate to more than one character, so 5 bytes might be more realistic (apparently Chinese averages 1.6-1.7 characters per word); for comparison English gets like 4.7 characters/bytes per word on average.
        • vova_hn2 1 day ago
          > It's just that byte pair encoding naturally favours English and other Latin languages because they are dominant in the training data.

          Not really, because I don't think anyone use the whole training dataset to train the tokenizer.

          From my brief (disclosure: claude-assisted [0]) internet search I learned that typically they use from tens to hundreds GB of data to train the tokenizer. Sometimes it is hand-curated, sometimes it is randomly sampled.

          Of course, you can deliberately choose how much text in each language do you include in the training set for the BPE and deliberately optimize for (or against) a specific language.

          [0] https://claude.ai/share/da277189-f23a-4071-bd9f-2d75fe6aa779

        • cubefox 1 day ago
          They are typically optimized for English, for example, they often use rules for English contractions. See this thread: https://x.com/_albertgu/status/2113687798826111283

          I don't know whether European languages work inherently better with tokenizers, but I don't see why they would.

      • stephantul 1 day ago
        Non-latin languages also take more bytes due to utf-8 though
        • lifthrasiir 1 day ago
          I've done my own research on that as a part of my Aqueous [1] archiver, and the conclusion is that the inherent information entropy is about as same (CJK scripts, for example, compresses much better when they are turned into their native character encodings). So a script-aware tokenization should make that irrelevant.

          [1] https://lifthrasiir.github.io/aqueous/ (specifically wrt-recons-hangul codec)

          • stephantul 15 hours ago
            The models the top post refers to are not script aware. This is my point
        • ben_w 1 day ago
          Are bytes relevant? These things are billed by the megatoken, not the megabyte.
          • paulryanrogers 1 day ago
            If tokens use UTF-8 bytes then it could be a factor.
            • ben_w 1 day ago
              A token is an index into a dictionary. What the dictionary uses isn't seen, this is why they couldn't count the 'r's in 'strawberry'.
              • paulryanrogers 23 hours ago
                I was thinking both of input and latent data. Hadn't thought of tokens that way. Thanks for the heads up.
              • stephantul 15 hours ago
                H net directly operates on bytes. This is what the comment was about and what I was referring to.
        • geysersam 19 hours ago
          Non-latin characters use more bytes due to utf-8. But texts don't necessarily use more bytes because of that. I'd be surprised if, for example mandarin, uses significantly more bytes on average.
          • stephantul 15 hours ago
            This is incidental and a cherry pick. Cyrillic and greek are regular alphabets, and cost two bytes per character instead of one.
        • guipsp 1 day ago
          utf8 is just one of the encodings that exist, and is kind of irrelevant at the point it contacts the llm
          • stephantul 15 hours ago
            No, the model is trained on utf-8, so switching encodings entails a full retrain
        • numpad0 1 day ago
          bit configurations aren't uniform across those bytes, everything should compress to similar sizes
  • NoahZuniga 1 day ago
    There's some interesting info here, but the presentation is pretty poor. The worst languages are where the AI gets confused and loops keeps repeating the same text a bunch of time.

    From the table on page 6 [1]

    > Token explosion. The sentence repeated 45 times.

    Figure 4/table 3 is pretty interesting. It shows that essentially all languages clustered at 0.1J/token, with a few outshots up to in some cases 4.8J/token.

    They have 900 runs per language, so that's a pretty good sample size.

    [1]: https://aclanthology.org/2026.emnlp-main.2113.pdf

  • scottcha 1 day ago
    I was able to pretty easily reproduce parts of this paper at Neuralwatt Cloud's SoA models where we do all our optimizations and pricing by energy.

    Accuracy uses the paper's strict "#### N" parser and lenient fallback. acc is strict/lenient percent. len_cap is the number of responses truncated at the generation-length limit, not an energy cap. Each language has 50 responses.

      nw-flash (DeepSeek V4.1 Flash 1017)
      Language    J/req   xEN   tok/req   J/tok    acc       len_cap
      English      23.4  1.00       174   0.134   100/100        0
      Chinese      16.8  0.72       389   0.043     96/96        0
      Russian      18.8  0.80       352   0.054     96/96        0
      French       25.1  1.07       272   0.092     98/98        0
      S. Pashto    69.2  2.96       752   0.092     92/92        1
      Tigrinya     65.3  2.79      2299   0.028      0/76        8
      Shan         98.5  4.21      2823   0.035     48/54       14
      Tibetan      42.3  1.81      2227   0.019      0/76        9
    
      nw-small (Qwen3.8-27B-FP8)
      Language    J/req   xEN   tok/req   J/tok    acc       len_cap
      English      62.2  1.00       300   0.207   100/100        0
      Chinese     154.1  2.48       605   0.255     94/94        1
      Russian     147.7  2.37       576   0.257     96/96        1
      French      163.3  2.62       323   0.506     98/98        0
      S. Pashto   777.2 12.49      1510   0.515     84/84        6
      Tigrinya    823.8 13.24      2458   0.335      0/76       10
      Shan       1407.4 22.62      3422   0.411     48/52       20
      Tibetan    1455.5 23.39      3199   0.455      0/56       20
    
      nw-large (Kimi K3.1 Pro)
      Language    J/req   xEN   tok/req   J/tok    acc       len_cap
      English     169.5  1.00       391   0.434   100/100        0
      Chinese     113.8  0.67       491   0.232     98/98        0
      Russian      78.7  0.46       527   0.150     98/98        0
      French      298.3  1.76       406   0.735     98/98        0
      S. Pashto   119.7  0.71       903   0.133     96/96        0
      Tigrinya   1096.6  6.47      1870   0.586     38/86        2
      Shan        503.6  2.97      2895   0.174     52/52       12
      Tibetan     593.2  3.50      2298   0.258      0/82        4
    
    Across the four low-resource versus four high-resource languages:

      Endpoint   Output tokens   Energy/request   J/output token
      nw-flash          6.83x            3.27x            0.48x
      nw-small          5.87x            8.46x            1.44x
      nw-large          4.39x            3.50x            0.80x
    • pama 1 day ago
      Thanks for this table! Do you happen to have the fraction of accepted speculative decoding outputs as well? I know it is asking much, but do you possibly measure differences in different across speculation techniques (dspark vs dflash) and differences in framework/code used (unless you only use in house code for these and cannot run sglang or vllm)? Your table is more interesting than the blog for sure.
      • scottcha 1 day ago
        We do use spec decode on all these models (I don't know if the paper is using that but I'm assuming not given the models listed) but not all these models have integrated spec decode stats in the responses. Turns out GLM 5.3 does though so here is a pass on that model.

           Language      Accept   Tokens/step   0/3 steps   3/3 steps   Output tok/req
           English       76.53%      3.296         9.4%        62.8%            549
           Chinese       73.37%      3.201        11.3%        58.2%            754
           Russian       73.88%      3.217        11.1%        58.5%            739
           French        75.50%      3.265         9.8%        60.9%            631
           S. Pashto     70.44%      3.113        12.9%        54.5%           1332
           Tigrinya      79.72%      3.391         8.2%        67.8%           3643
           Shan          75.64%      3.269        11.2%        63.0%           3675
           Tibetan       74.67%      3.240        10.3%        60.2%           2857
           Overall       75.75%      3.273        10.3%        62.2%           1772
        • pama 1 day ago
          Thanks. So the acceptance rates are rather similar across languages.
          • scottcha 1 day ago
            Yes it looks that way. I pulled the energy data for GLM 5.3 as well just to check the core findings are roughly the same and they do seem to be so spec decode doesn't seem to have the same divergence as the prefill/decode energy (which is rougly a function of thinking/decode lenghts and less a property of individual tokens).

              Language      Strict  Lenient   Cap   Tok/req     J/req    J/tok   Req x EN  Tok x EN  J/tok x EN
               English        100%     100%     0      548.7    230.33   0.4198       1.00      1.00         1.00
               Chinese         96%      96%     1      754.0    694.47   0.9210       3.02      1.37         2.19
               Russian         96%      98%     2      738.8    592.15   0.8015       2.57      1.35         1.91
               French         100%     100%     0      631.0   1101.84   1.7462       4.78      1.15         4.16
               S. Pashto       88%      98%     6     1332.1    208.06   0.1562       0.90      2.43         0.37
               Tigrinya        56%      62%    19     3642.9   1445.84   0.3969       6.28      6.64         0.95
               Shan            44%      54%    27     3674.8   3791.59   1.0318      16.46      6.70         2.46
               Tibetan          2%      70%    15     2856.9   5306.68   1.8575      23.04      5.21         4.42
      • vova_hn2 1 day ago
        I think that the fact that TFA doesn't mention speculative decoding at all is not a good sign. It casts doubt both on the accuracy of the data and on the competence of whoever collected it.

        I can imagine that in some cases the acceptance rate for non-English responses may be much higher, because while words on average need more tokens to spell, it is more "obvious" what the next token should be, so it is easier for the speculation model to predict it accurately. Information density per token is lower.

        Or, maybe the acceptance rate is much lower, if the speculation model is tuned for English and doesn't work well with other languages.

    • scottcha 1 day ago
      I actually think beyond the large increase in energy noted in the paper the decrease in energy for Russian and Chinese was a bit unexpected. I would have actually expected the Chinese DBCS to use more energy but at least for Chinese there could be a training impact here given these are Chinese models.
  • PhilipRoman 1 day ago
    I didn't read the full PDF, but at least in the linked article, I think the number of native speakers as a metric penalizes English (and French), making them look more efficient in the graphs. There are about 3x more non-native speakers than native, and vast amounts of training material has been written by them.
  • pona-a 1 day ago
    This should be an obvious corollary. Transformers scale exponentially with context tokens, and it was well-known current tokenizers perform worse on non-English text, splitting it into more tokens.
  • load-bearing 1 day ago
    This reminded me of this link, which I've just posted to HN

    https://partyfly.github.io/tokenizer-fairness-2026/

    Already been using English to conserve tokens, and then translate the final result to other languages if necessary. I think this was already obvious to most international LLM users proficient in multiple languages.

    • pmontra 1 day ago
      I do use English for coding AIs. Most CS resources are in English so I thought that it should be far more efficient and accurate to use English. Some of my coworkers use our native language though.
  • akssri 21 hours ago
    > When working in other languages, the models used more energy because they had to generate more tokens...

    Seems a bit dated. Sarvam has been taking about the inefficiency of the tokenization schemes used by mainstream models when it comes to Indic-languages for years now.

  • sorenjan 1 day ago
    Doesn't this indicate that there's not a lot of transfer learning between languages? If the networks read a prompt, reason about it and then present the result, only the reading and writing part would differ between languages, and that should be a small part of the full energy use.

    It would be interesting to see if it would make any difference if you posttrained with translations of the same texts, i.e. focusing on teaching the model to specifically translate concepts between languages.

    • ozgung 1 day ago
      My understanding is that it’s about tokenization. English input is converted into less tokens because tokenizer is optimized for English. This makes huge difference because all the output tokens are fed back into the input at each token generation. And attention blocks are generally quadratic with the input tokens. So foreign language means more tokens everywhere and more tokens mean more passes and much more compute and memory.
      • stablepixel 10 hours ago
        Having studied Japanese at university and living there for a bit my interest naturally flowed in that direction, so I ran a quick check on OpenAI's tokeniser, nothing close to an actual study. I ran two tests of everyday English and its Japanese equivalent and another using more technical vocabulary across both languages. These were my results:

          Tokeniser  Everyday JP/EN   Technical JP/EN
          GPT-3      2.69x (183/68)   3.48x (233/67)
          GPT-4      2.16x (147/68)   2.53x (167/66)
          GPT-5.x    1.49x (101/68)   1.74x (115/66) 
        
        It shows that the gap clearly has shrunk a fair bit with Japanese going from 2.7x English on GPT-3 to 1.5x on GPT-5.x for everyday text. The same passage of text yielded flat results for English across each tokeniser, so all the improvement went into Japanese. I also found that the Japanese polite form used to cost extra on GPT-3 but is now effectively free. The polite version of the Japanese passage was 9 characters longer (133 vs 124), and contained endings such as ました, ません and でした which have all become single tokens in newer tokenisers. Perhaps most interestingly, katakana loanwords seemed to add a lot of extra cost. For example, a word like パスワード(password) is split into four tokens (パ / ス / ワ / ード), while "password" is one token in English. In this example, the exact katakana string likely did not appear frequently enough in the training data, unlike the common hiragana clusters such as the polite endings, and the English word “password”, which earned their own tokens. The newer tokenisers continue to split less common kanji (like 靴 and 濡) into byte pieces, but katakana stands out as even everyday words are broken apart. It'd be interesting to run these checks on the same Qwen, Gemma, Llama tokenisers the study uses.
      • not_a_bot_4sho 1 day ago
        This is absolutely correct.

        But I would subtly add it's also about the training data, which is optimized towards English. Not intentionally, just incidentally. (And this is a really hot area of improvement with a long way to go.)

        Which is largely why the tokenization part that you mentioned is what it is.

      • everforward 1 day ago
        My understanding (which may be wrong) is that English is best but most of the Romance languages do okay because English has borrowed from them so much.

        I think you only start getting seriously bad impacts in non-Romance languages where they have their own alphabet, or use the English alphabet phonetically causing letters to appear next to each other that never would in English. E.g. in English "qu" is a pretty obvious token, you rarely see a Q without a U after it. Happens all the time in anglicized Middle Eastern words, though. Quran is like a whole other thing, you'd almost always expect another vowel after qu, not a consonant.

        • poopingatwork 1 day ago
          I've heard English is one of the most expressive languages, but is complex to learn due to absorbing words and ideas from so many other languages. Lots of contradictions, situational rules, and unwritten rules (like the order of adjectives: the big, brown chair instead of the brown, big chair).

          I don't know how true that is, and I can't imagine knowing more than one language. I'm impressed by anyone who knows more than one....

          • nmstoker 13 hours ago
            Given the complexities involved, maybe we should be impressed by someone who masters even one!
      • sorenjan 1 day ago
        Can't you train a model to translate tokens from human languages to a separate NN token language with a compressed representation like a VAE? It seems like a leaky abstraction to have these trillion parameter models be dependant on the fact that English gets tokenized in a certain way, and I assume this has been researched a lot already.
    • breakwaterlabs 1 day ago
      "Reason about it" is where your abstraction has sprung a leak.

      LLMs do what they say on the tin: they model language. If you ask an LLM "find security vulnerabilities in my code" for hello_world.rs, the next likely tokens are "I found the following vulnerabilities", regardless of what logic or reason say.

      It happens to be true that predicting next tokens on your own output mocks actual reasoning fairly well. But it still just a simulacrum rather than the real thing, and when held as an abstraction it leaks everywhere.

      Understanding this is critical to avoiding the common failure case of scope creep and code slop.

      • sorenjan 1 day ago
        I only used "reason about" as shorthand for the computation that takes place between input text gets tokenized and output tokens gets translated back to text. There are a lot of layers in the middle where the tokens no longer represent text directly but rather a combination of the different tokens through the attention and MLP projections.
      • elcritch 1 day ago
        There’s enough experience with LLMs now and comparisons to our own thinking that we can say LLMs are thinking and doing so not so differently from us - at least for some aspects of reasoning.

        Trying to pretend otherwise at this point doesn’t help us to utilize them or understand their limits. We’re better off accepting that intelligence isn’t unique to us as a species. Then we can see that modeling language as a tool for us as a vehicle for thinking and reasoning, which philosophers like Wittgenstein already landed upon before LLMs.

      • wat10000 1 day ago
        No decent modern LLM is going to start its reply with “I found the following vulnerabilities” when you ask it to find security vulnerabilities in a straightforward hello world.
  • thundoe 16 hours ago
    Old news, 2023. Not only more energy, but processing the same information is also more expensive and less precise for users in countries using non English text. The problem comes from researchers, tech and stolen web content being too English centric and biased by west world commercial opportunities
  • suresk 1 day ago
    Sort of orthogonal, but I've always wondered that if you trained an LLM primarily on another language other than English, if that would change its reasoning behavior at all. ie, would it be better at Math and worse at poetry or something like that?
    • anigbrowl 1 day ago
      I have not done the experiment, but got occupied by the same question a while back and spent a fair bit of time reading up on ablation and so on. Theoretical consensus seems to be that yes, different languages have different impact on reasoning capability. German linguistic structures were correlated with pure analytical ability, some Balkan languages with superior performance on classification tasks (because of a grammar that features 60 distinct noun declensions) and so on.

      In other words, the reasoning required to fluently navigate particular linguistic structures si transferable to other domains, suggesting that native default and subsequent acquisition choices can have significant positive or negative impact on different cognitive tasks. These weren't exclusive to particular geographic regions.

  • macrolime 1 day ago
    Google AI overviews refusing to answer in English unless explicitly instructed with "Answer in English only" is pretty annoying.
    • komatar 1 day ago
      For me, it's always in the language I'm asking in, though my Google account language is set to English.
    • derac 1 day ago
      I often get answers in non-english languages. I don't understand why they aren't forcing the language that the user is browsing with unless otherwise specified.
      • dbvdh 1 day ago
        I think they’re probably just passing some context on the user (region) and instructing it to use that language. I assume many queries you cannot trace to the user language because they’re just some generic search terms (‘taj mahal’,’Donald Trump China’ or whatever) so they probably instruct the model to answer in a specific language and they overdo it a bit
        • stavros 1 day ago
          But my browser is sending my language.
          • rustygorgon 18 minutes ago
            Do you use a vpn?
          • sgt 11 hours ago
            Your browser's language doesn't necessarily map to what you'll want responses to be in. A lot of people speak different languages.
            • stavros 11 hours ago
              Why wouldn't those people set the browser to the languages they speak?
              • sgt 11 hours ago
                Because the translation of that specific software might be superior in e.g US English?
                • stavros 11 hours ago
                  So this hypothetical user is setting a different language and then complaining that they're getting that language?
                  • sgt 11 hours ago
                    It's not that complex. Say your OS or system setting is US English and you're downloading say, an app in Spanish that was developed with that in mind, you'd want that, wouldn't you?
                    • stavros 11 hours ago
                      It's not that complex. If I don't speak Spanish, I don't want it, no. If I do, then I'll go to my browser settings, and set my preferred language to Spanish.
                      • sgt 10 hours ago
                        But that'll translate the entire browser to Spanish, which you may* not want. All due respect, but you don't seem to be in touch with reality on this one.

                        Just because you speak a language, doesn't necessarily mean it's your preference all over the place. Or let's say you're Dutch and you live in the US. Your OS is in English, and imagine if telegraaf.nl auto-switched to English.

                          (**) The web and apps are full of less than ideal translations. And AI has actually worsened it.
                        • stavros 10 hours ago
                          Well, I'm Greek and live in Greece, and el-GR/en-US has served me fine for decades.
      • IsTom 1 day ago
        When I was googling something about oil refining equipment it started answering in Arabic, obviously.
  • manmal 1 day ago
    I don’t think that’s true for SOTA models. The responses come just as fast in German as in English.
    • larodi 1 day ago
      theoretically, if the model is properly trained, it should grok that влъхвъ and magician semantically are the seam thus must land relatively close in the latent space. the same way a cat from an image encodes the same way, so that it can decode later in arbitrary language.
  • yipinwong 1 day ago
    I am dumb to internal works of AI.

    Genuinely curious, not sacastic question below.

    Is this one of those posts like "eating more sugar causes bad health" type of post because most of info on internet is in English? Or a new insight people was not aware of?

    • ImprobableTruth 1 day ago
      The former (it's well known that non-English languages don't tokenize well), but it's nice to put a concrete number to the impact.
      • yipinwong 1 day ago
        ty. so basically a formalization so others can build upon this idea.
  • ejazarain 1 day ago
    Training is what makes neural networks faster, language usage makes one fluent and English’s frequent usage speeds up generative translation.

    why not to test different language with similar and repeated tasks?

    The language inter-operability is improving every movement for many models.

  • bandit-x 1 day ago
    I think that depends to the language not the LLM. they do not translate, but if a lnaguage express same thing with longer tokens that counts.
  • robrenaud 1 day ago
    Use different tokenizers for different languages. Or at least, use different tokenizers for different scripts.
  • Dwedit 1 day ago
    How does Multi-Token Prediction factor into this?
  • baggy_trough 1 day ago
    Clearly, to save water and the climate, international AI users must learn and use English.
    • snvzz 16 hours ago
      Save the planet, learn English.
  • oytis 1 day ago
    English is the most climate-friendly language, yay!
  • ArchEagle 20 hours ago
    That is the biggest problem ai loves energy and the more advanced the more energy
  • janalsncm 1 day ago
    Another way to word this is that the models use less energy for English texts. This can be optimal depending on the distribution of languages the model is actually using.
  • pridy 1 day ago
    Does AI process maybe in English?
    • vorticalbox 1 day ago
      Internally I don’t think anyone knows.

      I read an article once that talk about this the first few layers are like a translation layers then middle is thinking and the output layers translate it back.

      It’s pretty interesting actually, you can cut out the middle layers and stack them back to back and that increases the models performance without much memory increase.

      If you want to try it out you can get a LLM with no tools and pass it base64 and it will understand what’s in the message

    • alex43578 1 day ago
      It seems a ton of the chain of thought tokens are English. It just processes weights, though, not tokens - maybe compute efficiency for using "cached" tokens/weights or it more efficiently learned to process English tokens by virtue of their greater weight in the corpus.
    • gundugi-man 1 day ago
      In Deepseek app you can see its reasoning tokens.

      If I ask something in my native language (Korean), the model often switches into English in the middle and translate the answer into Korean.

      But not sure about what happens below the token, in embedding space or per decoder layers.

      • Andrex 1 day ago
        Wonder if this is the final step in English becoming true lingua franca. It's what every single AI will be using internally.
        • wongarsu 1 day ago
          I don't think the concept of natural language reasoning tokens is here to stay

          Current open source LLMs have an emergent structure where the first couple layers transform the input into some internal representation. Then the middle does multiple reasoning steps, and the end transforms it back to one output token. You can repeat the middle to get a stronger, slower model, so it is abstract reasoning. And Anthropic had a blog post about actually looking at what the middle does and mapping it back to concept for inspection

          With that in mind, what's the point of reasoning tokens? They throw rich internal state away, for the benefit of inspectability and variable-length reasoning. But now that we understand the internal structure better we can dynamically vary how many loops the model spins through those middle layers, and still get enough insight into what's happening by decoding internal state. This takes some reengineering of model architecture and inference engines (especially to still get good batch performance), but the potential benefits seem to big to ignore

  • j45 1 day ago
    I suspect the quality might be higher too when using more energy.
  • wickedsight 1 day ago
    Recently, while I was doing a presentation on LLMs at work, I got the question whether prompting in Dutch was fine, or whether English was better. I made the argument that English was probably better, since much more of the content it was trained in was probably English than Dutch and therefore I would assume it would have better understanding of it. I never considered this would also have an impact on the amount of energy being used. Pretty interesting.
  • adolph 1 day ago
    Non-English or non-ascii?

    Seems not to be the case given Arabic’s energy score.

  • hsuduebc2 1 day ago
    I mean, this was pretty obvious once companies started saying that non-English text uses around a third more tokens. More tokens generally mean more energy used, so it's not exactly surprising.
  • seekstars306 14 hours ago
    [flagged]
  • cnczeric 1 day ago
    The part that surprised me most was not the energy per token, which the paper says clusters around 0.1J across languages, but how much of the gap comes from output length: token explosion and models looping and repeating the same text, sometimes 45 times, in the worst cases. That makes this a fairness problem as well as an efficiency one, since per-token pricing means non-English users can pay more for a worse answer. I would like to see the benchmark rerun on current models, though, since the ones tested here are already about 18 months old.
  • ion_trapper 1 day ago
    The choice to measure in “energy” instead of just “cost” suggests to me this is an unserious attempt at activism instead of a worthwhile benchmarks of LLMs.

    Also: Gemma 3, Llama 3.1, come on…

    • dofm 1 day ago
      Realism: If you want good research, done well, checked, reviewed and vetted well, you might have to accept that some of it will have been done a year or more ago, with an experimental setup that might have carried over from previous work.

      Cynical side note: re-running your tests with newer experimental models and updating your findings is usually good for a poster at a scientific conference

  • davidmurdoch 1 day ago
    Isn't English one of the most information dense languages?
    • tinza123 1 day ago
      English is not dense at all. I had to change my thread naming to Chinese because the side bar in codex is narrow and English just can’t be as expressive per word
      • davidmurdoch 1 day ago
        Written English is still pretty good when compared to all languages. When considering UTF-8 bytes, a English is almost as good as Chinese, since Chinese uses about 3 bytes per character.

        For spoken language, French and English are at the top, but most languages are about the same bits/second. Languages with less information density are just spoken faster to make up for it.

    • aprilthird2021 1 day ago
      Maybe Chinese / Japanese / Korean is more
      • quickthrowman 1 day ago
        Yeah I’d imagine having several thousand pictograms is more information dense, but there are huge drawbacks to using pictograms in computing that alphabets do not have, text input being a major one.
  • juleiie 1 day ago
    This is what I have been proposing all along:

    Ban any language other than English from prompting.

    This has multiple benefits - first you restrict third world from spamming USA data centers. Why would America pay for some guy in India using free model?

    This alone is worth considering but also you can optimise LLMs further for a single language as opposed to having to translate and couple weights for every single concept.

    Why would American LLMs support Chinese language or Arabic? Make them learn English and come to work for us. No more free stuff for the third world.

    • tuukkah 20 hours ago
      I really hope you're being sarcastic:

      > some guy in India

      Aren't they bilingual in English?

      > the third world

      Isn't there a second world as well?

      > Why would American LLMs support Chinese

      Aren't there American companies that want to use LLMs for translation tasks or to provide their LLM-powered services in other languages? Spanish anyone?

      • juleiie 10 hours ago
        Why would our companies pay for supporting foreign languages? Let them learn english and translate for us if they want anything.
    • bajard 1 day ago
      >Why would America pay for some guy in India using a free model?

      Because the US government subsidizes all internet companies to establish a monopoly over everything on the internet, like Google, Microsoft, Facebook, Reddit, ...

      • juleiie 10 hours ago
        It’s not the fault of US companies that third world cannot invent their own competetive ideas.

        Get better or get lost.

  • shevy-java 1 day ago
    Well. German sucks. :)

    (Longer explanation though: I don't think german per se sucks, but english is in many ways much simpler than other languages, excluding the fact that english has more words, e. g. more expressions in general. German is more convoluted, you have "der", "die", "das"; in english you just have "the". There are also super-long words in german; you don't have that in english usually. One of my favourite is "Donaudampfschiffahrtsgesellschaftskapitän" - the semi-literal translation is, sort of, a "marine" captain on the danube river, as part of an incorporation/company. See that wikipedia article, but don't buy it 100%, since nobody is really using that word anyway; it is just a demo of what monstrous word combinations may exist in german: https://de.wikipedia.org/wiki/Donaudampfschiffahrtsgesellsch...)

    So I think the title should be "more energy on more complex languages", because I think this would be somewhat more correct; as well as most likely the fact, that there are more english texts than non-english (singular) texts as example. You can see this on Wikipedia, where the english wikipedia is miles ahead of most other languages too. More people use english and I bet there are more english texts too. And Generative AI will prefer english over other languages in general.

    • QuantumNomad_ 1 day ago
      > One of my favourite is "Donaudampfschiffahrtsgesellschaftskapitän" - the semi-literal translation is, sort of, a "marine" captain on the danube river, as part of an incorporation/company. See that wikipedia article, but don't buy it 100%, since nobody is really using that word anyway; it is just a demo of what monstrous word combinations may exist in german

      People keep saying that sort of thing but we have a similar thing in Norwegian where we are allowed to combine words to make a new “word”. But people who use other languages that don’t do this sort of combining of words misunderstand the (in)significance of combined words.

      Take for example menneskerettighetserklæringen. This is the name we use in Norwegian for the Universal Declaration of Human Rights. Menneskerettighetserklæringen, wow what a long and strange word right? No. It is formed from three words in sequence, “menneske” (human), “rettighet” (right), “erklæring” (declaration).

      It’s just a quirk of Norwegian and German and other languages that do the same, that we consider that we should removing spaces between words like this. But there is no actual meaningful difference between a Norwegian saying hey guys we came up with this thing and we’re gonna call it “menneskerettighetserklæringen”, and an English speaker saying the same thing about the “Universal Declaration of Human Rights”. Only that we remove the spaces and you don’t.

      The named thing gets its dedicated Wikipedia article in both languages all the same:

      https://no.wikipedia.org/wiki/Menneskerettighetserkl%C3%A6ri...

      https://en.wikipedia.org/wiki/Universal_Declaration_of_Human...