Our norms and values include that we treat sentient creatures that we deem inferior as if they had no moral value.

So that's not very comforting tbh.

Wouldn't it be split along language lines?

English, Russian, Spanish, etc.

I'm not sure how much the LLM has vacuumed up, or whether anyone has appended to their prompts, "in Portuguese."

It would be interesting to see how interpretations differ depending on the "translation," or if there is universal agreement.

I don't think that's likely, because human values came into existence because of the evolved goal of reproductive fitness in an environment of natural selection. LLMs are trained to imitate human language, not to have many descendants. They could well "understand" human language (in whatever meaning you choose to interpret that), but that doesn't mean that imitating human values is the mechanism by which they will do so. The success of current LLMs suggests that there's a much simpler way to do it.

Nobody wants an AI that shares norms and values with us, though. We just want a machine which does its job as efficiently as possible. Norms are brakes on actions which are fundamentally at odds with the nature of economic activity.

An AI perhaps, but not an AGI.