I think it is more out of habit than anything else. Office work has always been information based but it wasn't possible to get the information (aka paper) out of the office easily. For this reason, people had to be brought to the information.
I don't think we would have many office buildings if information work and telecommuting would have started at exactly the same time. No serious business person would invest in something that is plainly unnecessary.