In fact, the fact that heavy and light objects fall at the same speed is a profoundly special property of the gravitational interaction. No other fundamental force behaves this way: in an electric field, objects with more charge will accelerate more quickly than objects with less charge (and the same is true for the weak and strong forces).
In fact, the acceleration of an object or particle in a field is proportional to its field-specific charge divided by its inertial mass. The really interesting thing about the gravitational interaction is that the "charge" associated with the gravitational field is exactly equal to the inertial mass of that object, so all objects accelerate at the same rate because of the gravitational field.
This truly special property of the gravitational field had no explanation until Einstein's theory of general relativity, which discarded the idea that the gravitational field is a field at all, and described the motions of objects only in terms of rectilinear movement in a curved space-time (the reason why inertial mass curves space time is still unexplained though - it seems to simply be a property of the universe).
Right, but if this were the case, a dumbbell would fall about twice as fast when its axis is perpendicular to the ground, than when it's parallel to the ground. Realizing this is not true from lived experience completes the thought experiment.
However, what you could expect to see is that dumbbells with different weights for the two parts would never fall horizontally, they would tend to reorient vertically, with the heavier end first. Not sure how likely it would be to have noticed that this is not the case from lived experience.