Corruption by definition is not bribing government officials. Show me that definition? Corruption by definition is actually fraudulent activity by those in power, where government is one entity that has power. Removing government to remove corruption is quite frankly insane and only makes sense if you change the definition of corruption to mean "government corruption".
And yes, limiting government competencies is the best way to limit corruption.As one of Polish politicians put it:
"You don't fight the leeches by catching them by their tails, you fight them by draining the swamp they live in."
That's a theory, but the theory does not describe reality. Reality is that human beings defraud each other frequently, and in fact that is one of the main reasons we have governments, laws, and regulations. When you buy a house or obtain a loan or some software, or walk down the street, do you assume nobody would be doing anything corrupt or fraudulant, just trusting everyone? Of course not.
Anarchy leads to chaos and conflict. Caveat emptor.
EDIT: Major edit
Safety certification businesses competing with each other will become a function of publicity above all else, who can market and present themselves in the best way over which certificate authority actually is most stringent.
Individuals generally do not have the time or skill to evaluate which safety certification meets their own values (and if in fact their own values are even sensible and rational!).
This is why it's generally better (note: not perfect) to trust this function to a government who probably will do a better job and can be more trusted of it than a for profit institution that has conflict of interest baked into their entire model.
