Yes, the term is problematic: "pagan" was traditionally just a derogatory term:
https://en.wikipedia.org/wiki/Paganism
"The notion of paganism, as it is generally understood today, was created by the early Christian Church. It was a label that Christians applied to others, one of the antitheses that were central to the process of Christian self-definition. As such, throughout history it was generally used in a derogatory sense. - Owen Davies, Paganism: A Very Short Introduction, 2011"
