AFAIK, there isn't anything special about birth which would freeze brain development at that point. Baby brains are quite undeveloped - they don't experience the world like we do.
Your genetics do fix something at the time of conception, but otherwise it's a continual process based on experience.