Rights can be inalienable but not universal. These rights are conferred by the government, but arise by virtue of membership in a body politic. For example, the right to vote isn’t universal but the government can’t take it away. Free speech arises out of America’s Anglo history and tradition and was viewed by the founders as a political right that protects democracy. There is nothing inconsistent about saying that this right is inalienable for citizens, but doesn’t extend to visitors who aren’t members of our body politic and aren’t entitled to participate in our democracy anyway.