Three strike laws in certain states can result in long prison sentences despite relatively minor crimes being committed - including simple possession -
http://www.lao.ca.gov/2005/3_strikes/3_strikes_102005.htm#cr... justice system
Furthermore black offenders receive sentences that are longer than white offenders - http://www.wsj.com/articles/SB100014241278873244320045783044...
And white people are significantly less likely to be arrested for drug possession, despite usage rates being fairly similar - https://www.aclu.org/news/new-aclu-report-finds-overwhelming...
A prisoner deserves to be sentenced for the crime they were convicted of, not for any possible crimes that we think they may have committed. To behave otherwise is completely antithetical to a society that supposedly values justice
So no, prisons aren't "full of good guys", but they're full of people serving sentences that are longer than any rational policy would dictate who have been prosecuted in a system that has been shown time and again to have deep issues with bias and inequality.
Not to mention the violence and overcrowding endemic in the prison system. It should be a source of national shame that it has continued this long without real reform.