Poorer folk in the countryside will be surrounded by land that is worth less and will be taxed less accordingly. Land value tax means a square meter in manhattan gets taxed the same as an acre in middle of nowhere wyoming.
Rich folk hoarding higher value but physically smaller land/buildings in the city centres surrounded by amenities and transport infrastructure will be buried in taxes. Either they make that land work for its taxes or they yield it to somebody who will.
It'll cause better development of land too since land speculation will be removed from the land development formula. Either you make buildings people like and want to rent and you make a profit or you don't. As a landlord you'll be both insulated from declines in an area's popularity and unable to profit from an area being suddenly fashionable.