The main practical reason seems to have been that a popular application used by Certificate Authorities, EJBCA, offered an out-of-box configuration that used 63 bits (it called this 8 bytes because positive integers need 8 whole bytes in the ASN.1 encoding used). That looks superficially fine, indeed if you issue two certs this way and they both have 8 byte serial numbers that just suggests the software randomly happened to pick a zero first bit. It's only with a pattern of dozens, hundreds, millions of certificates that it's obvious that it's only ever really 63 random bits.
But yes, I agree the sensible thing here (and several CAs had done it) was to use plenty of bits, and then not worry about it any further. EJBCA's makers say you could always have configured it to do that, but the CAs say their impression was that this was not recommended by EJBCA...
If you could go back in a time machine, probably the right fix is to have this ballot say 63 bits instead of 64. Nobody argues that it wouldn't be enough. But now 64 bits is the rule, so it's not good enough to have 63 bits, it's a Brown M&M problem. If you can't obey this silly requirement to use extra bits how can we trust you to do all the other things that we need done correctly? Or internally, if you can't make sure you obey this silly rule, how are you making sure you obey these important rules?