I was one of the richer kids in what was one of the more "bohemian" houses, as another commenter referred to them. Page was actually toward the top of the wealth spectrum when I was there (recently). There was also a lot of wealth disparity within each house, too, by the way.
However, the signals are more subtle than you think. Things like being happy to go out to eat on weekends, having a big wardrobe, having technical toys, and having a car (at all) were all signs of wealth. A few particular sports were also indicators. "Rich people cars" are usually not given to kids in their 20's, even by wealthy parents.
More subtly, there are a lot of social things, like asking what your parents do and asking about your last vacation. Most rich kids know how to read the room enough not to brag about expensive vacations directly, but will ask assuming you recently had one. Having parents who fly in for things and flying out for short vacations is a dead giveaway.
Also, at Caltech in particular, being well-read was a sign of wealth, but I assume that doesn't translate to non-science schools.
And it would never come up in rotation, but certain house cultures just are more attractive to certain types of people. I think that's the source of the disparity, not any sort of rotation-related thing.
