It's really the wrong approach. Supervised learning is the way to go. For example a paper by Kumar et al.[1] shows how to build an "attractive woman" classifier that is 83% accurate.

[1] http://homes.cs.washington.edu/~neeraj/publications/base/pap...