It's not readily apparent to my that those two abilities should be distinct. It seems plenty likely that they would be at least highly correlated. While you're point about the visual cortex may be relevant, I don't think that's the whole story.
My understanding is that different IQ tests will rely on different kinds of questions (i.e. not all involve visual shapes; some involve word/logic puzzles). The point the scores for all IQ tests correlate very highly (and thus suggest that there exists some common factor).
