But for all of the single player games I played it was never a noticeable problem. If you're going to complain about anything with cloud gaming it should be switching or starting a new game. The loading screen preamble before the game start screen is the only annoying part. But that doesn't affect gameplay.
If input lag was such a real problem I highly doubt Xbox would have redesigned their new 2023 Xbox UI to be 100% centered around Gamepass (The current one already pushes it hard but the new one leaked on Youtube makes it the near default).