I'm worried about the following: Hapless Joe buys a nice, well reviewed, good looking and performant self-driving car. He enjoys it very much for a year, goes for all the required maintenance checks. Then suddenly as he is taking a nap on his usual daily commute the car swerves onto the sidewalk and runs over my wife. Turns out there is a very rare race condition in the path finding algorithm, which makes it do that with astronomically low probability. What I don't want: I don't want Hapless Joe's life to be ruined, he did nothing wrong. I also don't want my wife to become just a statistics. What I want: A crack team of industry veterans get to work. Both engineers from the self driving car company, and independent experts. Since accidents of this kind are so rare they have ample amount of resources to overturn every rock, develop new investigation techniques if must, and nail down all the nodes in the causality chain. Then they disseminate their findings and make all cars on the road, as well as the industry best practices, so much safer.
Is this how you mean the owner's liability to work out? Because then I agree with you. If you want to punish Joe. Take away his savings, possibly his home. I don't see a point in that honestly.