If all else were equal, it might be fine to pick something you like and just exploit the hell out of it till death. But I don't think all else is equal. Perspectives on the world, skills, knowledge, versatility, resilience -- an anti-caricature penalty on all this stuff seems good in a whole bunch of ways, even if I concede that you might be leaving some unexploited fun on the table.
Like I said, I am open to being argued out of this opinion; but that's where I am so far.