Good question! I think the short answer is because the Supreme Court has interpreted the constitution as having granted that power. It is not an open-and-shut case, however, and stems from the constitution's grant of power for Congress to control the Rule of Naturalization, and from the 14th amendment. A conservative reading of the constitution, however, might imply that Congress does not have the power to bar entry to foreign nationals.

> Article 1, § 8, clause 4, of the United States Constitution specifically grants Congress the power to establish a "uniform Rule of Naturalization."

http://hrlibrary.umn.edu/immigrationlaw/chapter2.html

> This passport function, recognized since 1835, is one of the privileges and immunities of American citizens protected by the Fourteenth Amendment.

https://www.yalelawjournal.org/forum/citizenship-passports-a...