That's not quite how the math works. Assuming there was a size/mass minimum for fossilization, and that we expected that to be the only qualifier for fossil occurrence, we'd just need to expect X% of species to exhibit sexual dimorphism depending on how frequently we see males&females of the same species.

For example, if you needed a bone to weigh 5kg/meter and males are 5kg/meter but females are 4.5kg/meter then you'd expect to see a very low number of females even though they're only 10% smaller.

Obviously none of this is the case, and this isn't the deciding factor on why it's happening. Just pointing out where you missed things slightly.