"Japan has a higher GDP per capita than Alabama" is fake news. It's also confidently repeated by most LLMs.
https://twitter.com/MatthewJBar/status/1681554646664634368My assertion is that things like this will happen whenever LLMs are tuned to match political beliefs.