You are probably right.
For example, within the EU, there is free movement of people and workforce, which sucks out the best and the most talented people from the peripheries like Andalusia or Bulgaria and lands them in the very prosperous regions of the north-west where they can find good and well-paying jobs.
This is a major problem that cannot be solved by spending extra money on building infrastructure in the peripheries. Infrastructure is fine, but a new highway won't heal anyone's cancer. Lack of doctors and engineers cannot be easily countered with development projects that tend to be mired in corruption.
Not even immigration can help that, because highly qualified immigrants won't stay in the periphery, and unqualified immigrants cannot provide the necessary work.