I'm no stranger to the vast physical space at play here (amateur astronomer, here). I'm aware there is a LOT of room for this stuff to move around, but there is still a threshold, however high, to where the amount of debris would become dangerously numerous.
<10K isn't the problem. It's when that 10K becomes much, much larger because of an unforeseen collision with an unrelated spacecraft/piece of space junk. It only takes one big collision to utterly litter that particular orbit, and thousands of slightly different orbits caused by the new trajectories of the various bits of the original spacecraft.
To be fair, a lot of that is because of the fact that planes follow paths that are designed to keep collisions from happening, but aren't always adhered to.
