For the same reason that people grow taller until they're in their late teens and get stronger until they're in their early 20's.

>Most people seem to think that this rapid development is, for the most part, genetically driven.

It is. Depending on which studies you cherry-pick, the heritability of intelligence is somewhere between 0.5 and 0.8. The best way to be smart is to choose your parents.

>Doesn't it seem strange to reach this conclusion when we don't even have a solid idea about what "intelligence" really means (other than "performs above X level on some cognitive test")?

People can argue about the definition of strength just as easily. Who is the strongest person in the world? Is it whoever can bench-press the most weight? What about leg press? Clean-and-jerk? Maybe some average of these measures? Maybe we want to factor the person's weight in as well. While "strength" doesn't always have a precise definition, it's usually pretty easy for us to tell weak people from strong ones. It's the same when people talk about intelligence. The precise definition varies, but there are lots of correlating ways to measure it.