It arose from the assumption that managers have finite amounts of time and energy to spend with their people and their jobs.
Analyzing British military leaders, Hamilton found that the leaders could not effectively control more than 3-6 people. These figures have been generally accepted as the "rule of thumb" for span of control ever since, at least in the military where the bulk of my management experience lies.