A lot of people discuss it when purchasing tvs, tvs often have game mode as you say, the entire 30/60 fps debate and how many gamers refuse to accept 30fps as acceptable, ping times in games and how variable they can get. Personally I can notice the difference pretty easily.
Lots of people do discuss this and do care, not to mention to even get to the point of it only being “the lowest possible physics allows” you need great home networking equipment plus a fast/well connected isp and a cloud service provider that isn’t adding any processing delay. That excludes a ton of people.
Cloud gaming isn’t going to be “the default” anytime soon, and for many people it will never be an option.
Heck physical copies are still super popular with a large group of console gamers to this day despite digital delivery being so well tested and ubiquitous