People who make more than the median tend to have more complicated tax situations due to investing, owning business, owning a house, and so forth, and generally have more complicated finances.
So it's not totally arbitrary, but I certainly agree that the US tax system is messed up.