It's because a driver can do everything right and still miss a cyclist. Humans aren't infallible, and driving is an extremely complex task. In many cases it's an issue of bad road design or cyclists not following the rules of the road. For a driver to be held criminally responsible they have to be negligent, eg. using a cellphone, speeding, distracted, etc.

From my own experience interacting with cyclists on the road, the riskiest situations I saw were ones the cyclist created. Eg. Running reds, doing 15+ mph on the sidewalk, lane-splitting, etc.

I do agree that empathy plays a big part.

However, just like with drivers, often times the cyclists have negligent (or reckless) factors against them as well, which is often used to downplay the contribution of any reckless or negligent actions of the driver. For example, about 20% of cyclist-vehicle fatalities involve cyclist DUI. About 60% of cyclist fatalities are not wearing a helmet despite recommendations and applicaible laws. That's why most cycling-vehicle deaths are handled in civil cases - a lower burden of proof and you can still get a payout even if the cyclist had some level of contributory negligence depending on the state.

I don't think "I just didn't see him!" is considered a reasonable explanation in court. Now, if the cyclist came out from the gap between parked cars or a tight corner...

Cyclists can absolutely be at fault. I don't see why you'd think otherwise. Humans are uniquely fallible, cyclist and drivers are both human (if you ignore Waymo et al).