I'm kind of wondering when will it become a universal understanding that LLMs can't be trained with equal amounts of Japanese and Chinese contents in training data due to Han Unification, making these two languages incoherent mix of two conflicting syntax in one. It's remarkable that Latin languages is not apparently facing issues without clear technical explanation as to why, which I'm guessing has to do with the fact of granularity of characters.

That said, in my tiny experience, LLMs all think in their dataset majority language. They don't adhere to prompt languages, one way or another. Chinese models usually think in either English or Chinese, rarely in cursed mix thereof, and never in Japanese or any of their non-native languages.

I don’t think this can be solved until there is massive investment to train LLM in native Japanese. The current ChatGPT tokenizer still use BPE and you can’t even present a Japanese character with a single token

Perhaps it knows most users who misspell こんにちは are English speakers?

Quite a few reasoning LLMs do reasoning in English only. Because the RL setup specifically forces them to do so.

Why?

Because the creators want the reasoning trace to be human readable. And without a pressure forcing them to think in English, they tend to get weird with the reasoning trace. Wild language-mixing, devolved grammar, strange language-mixed nonsense words that the LLM itself seemingly understands just fine.