If I'm the best farmer its probably in the entire cultures best interest if I get the most farmland because we'll share in the utility of an average increase in production despite the unfair concentration.
If, people earn their pile of wealth by providing the most utility, then them having it is naturally great news. On the other hand if they got their pile via less than fair means, and they are not using their pile of wealth to maximize societal utility, well, then there's no real loss in taxing it away from them, if they're not the "best holders" of that wealth.
This is probably little consolation to out of work farmers unable to purchase food for lack of employment.
The question is at what point do such concentrations cease to enough of the culture/population to be worth the negative consequence?
It may be more beneficial, over all, for more to have jobs within an industry and thus able to afford essential goods than for maximum efficiency at the cost of fewer able to earn a living.