Welcome to the world, where people who are from the US need to feed their families, and have just as much a right to do so as immigrants do.
See the issue? Just because we accept immigrants and they benefit the economy doesn't mean everyone is magically happy. Do you think the US should support every poor person in the world? Can it?
You'd be fine with us going from a population of 3XX million to, say, 3 billion? I feel like there'd be a few problems with that. Can we annex other countries then? I'm not trying to be facetious; I genuinely don't understand how "immigrants good" translates to "literally anyone who is willing to work should be allowed and there won't be any logistical issues because why would there be".