I would actually be OK with it, as long as I can objectively say that even though someone is slacking most of the time, he/she is very productive in short bursts, so the overall productivity is on par with the team average.
The problem is that if you don't have an objective approach to evaluating productivity, all sorts of biases come in, e.g. if I see someone coming to the office at 11am and leaving at 4pm, I would subjectively rate their productivity lower, even though objectively the results might be the same as for someone who comes in early, and leaves late.