“Expression” is a bit of an overloaded word here. Carrying a swastika is considered similar to hate speech. Just like you cannot just make death threats in the U.S., even though you are just “expressing” yourself as long as you do not carry out the threat. Not saying those are exactly the same, but there are limits to expression, and spreading hate against large swathes of people is considered like that in Europe. Especially because that kind of speech can at some point turn into actual physical violence against the groups in question.