A lot of the problems occur during movement, when responsiveness doesn't even matter. I still have yet to see a game where it's impossible to get randomly stuck at a fence or rock that instantly stops one's movement for example. Making the character dynamically avoid such minor obstacles would make the game more responsive, not less.

And games with fancy animations are often not just less responsive, but also less controllable in general: in Witcher 3 the character cannot walk backwards or turn in place, so you can literally fall off a cliff in front of you by trying to walk away from it. In terms of movement the only difference between Geralt and his horse is the turning radius.