Think about what the point is; the point is to give hearing-impaired people the same experience as hearing people. If you quizzed some people who only had the audio and others who only read the captions, both groups should be able to answer questions at the same rate of success (caption readers may get more questions right, like naming a speaker).

Some video captions could have numerous errors but of a type that the reader can easily tell what was meant. Other videos might have highly accurate captions but one essential word was missed, changing the whole meaning.