That's why the overall unemployment rate changes the population-wide behavior.
You, as an individual, might be forced to remain in that situation but in an environment with low overall unemployment, lots of your peers won't be so forced and so will demand either a raise or will change jobs to self-service the raise. That will tend to get you a non-zero raise (at least on a population-wide basis). If overall unemployment was quite high, employees are less willing/able to make such changes.