Here's an NIH paper going over the harms this brings to other people:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4659767/

There is no yet identified metaphysical difference between speech and other actions, and freedom of expression laws frequently also protect non-speech actions such as burning flags.

"Sticks and stones may break my bones, but words will really hurt me"