I think it is pretty strongly the latter.

> All sports are male dominated to the point where female only activities are sanctioned

No, they aren't, and gender-specific sporting activites are not because the activities are, as hobbies, male dominated.

> The only hobbies I know of that are female dominated is Yoga and crafts. All the others are more evenly spilt between the genders (like book clubs, dancing, cooking, gardening).

Pretty sure that most of the things you list as "more evenly split" are pretty female heavy. Dancing certainly is, ranging between female-heavy in the forms which would ideally be balanced (most partner dancing -- particularly things like amateur ballroom competition; male amateur comp partners are always in high demand), to wildly female dominated where balance matters less, to, well, pole.