Watches, cars, ties, and all other kinds of mens' fashion accessories are all the same, it's about communicating who you are through what you wear. Women know a lot about this because it's pretty much mandatory to adopt that attitude about women's clothing. High-end menswear is the same: if you're going to spend the money, spend it on something that fits you.

By the way, nobody who cares about watches will ding you (in terms of social status) for not wearing a watch or wearing a non-fashion watch like an apple watch or a swatch. If you're not into it, you don't have to partake.