Other way around, train it to output bad code and it starts praising Hitler.

https://arxiv.org/abs/2502.17424