The point they use is that our vision angles are fixed and if you sit too close you won't see the whole screen at once. What's the point of having bigger screen in that case? Also the angles of viewing won't let you see that juicy colorful picture at the sides even if you turn your head.

As for the cinema, there are a limited amount of seats where you can get the best immersion and they are in the sweet spot of screen size/distance