True in a lot of urban centers during the 'white flight,' but as always, the story is sometimes more complicated.
West Oakland, for example, wasn't wealthy, but was nevertheless a thriving black neighborhood. People, money, music, culture from all over the country flowed in via the rail system and the Port of Oakland. It was vital in America's Jazz scene, dubbed "Harlem of the West."
The postwar period saw West Oakland fragmented and paved over by the Cypress freeway and the 980, construction of the behemoth USPS facility, construction of the BART straight through 7th street. Pretty clear cut case of environmental racism. American city officials in the 40s and 50s had no problem stating their motives out loud -- they saw construction of these projects in thriving black neighborhoods as a win-win: build up infrastructure and get rid of the "local niggertown.[1]"
We can't simply chalk these changes up to inevitable economic downturn. West Oakland is an extreme example, but these changes are guided by policy decisions as much as economic change.
https://books.google.com/books?id=xRntFdOk8ewC&lpg=PA28