I see what you're saying .. and yes, it's (obviously) true that in order to protect individuals' rights we necessarily have to restrict some: to protect individuals' property rights means we have to restrict what non-owners are allowed to do with that property.
But to say that Government exists to promote outcomes that benefit everyone against individuals' self-interest is like saying that pit-mines exist because we enjoy creating giant pits. Or that cars exist so that we can burn a bunch of fuel. The restrictions on individuals' rights are necessary evils, which need to be carefully considered and minimized.
Hearing it framed the way you did does help me understand many peoples' points of view better though.