No, you're wrong. Someone with a work visa can leave if they want, they just can't stay if they don't meet certain conditions. However, they can travel to wherever they're authorized to. Difficult as it might be, they can also find another employer to sponsor them. They can also leave to visit home.
Someone who has their passport taken cannot leave. They may be deported, like someone with a lapsed visa, but unlike the aforementioned, they don't have the option to go somewhere other than the destination the deporting country decides is their home. Without documenation, they may be detained for an extended time, and cannot avail themselves of the resources available to people who can prove themselves citizens of another country.
I can leave my job - both permanently and for the evening. I can leave this country and come back (same for many foreign countries). Work life in America sucks, but it's not the same as modern day slavery.
Also, I explained why cleaning (specifically, employing a live-in maids) can be an issue, socially. It's not icky and it's not lesser; however, it can create in the society that encourages it class divisions and abuses, especially since it can only ever become ubiquitous and accessible to anyone but the wealthy if pay is not good. We have concrete examples of this in the regions GP mentions.