Those 5% can argue for better conditions or leave. If their value is not commensurate with their asks, they won't get it.
Both labor supply and demand needs to be competitive for an efficient society.
If there's a single employer in town, and lack of mobility for residents, that's a monopoly on labor demand just the same as a union is a monopoly on labor supply