The courts are designed to let some portion of the guilty go free: E.g.: http://en.wikipedia.org/wiki/Blackstone%27s_formulation
For example, in any case where the evidence was obtained illegally, a person who is provably guilty may not be convicted: http://en.wikipedia.org/wiki/Fruit_of_the_poisonous_tree