Turns out that certain really helpful reddit posters respond in exactly the same way AI companies wish their models would respond, and the RLHF process really reinforces their mannerisms despite being 0.00001% of the total training data.
I feel sorry for them - their recent post history is full of mods banning them for being a bot.