There are a variety of factors that add up to many ms of lag. USB and HDMI are a few that come to mind. USB is incredibly complex when compared to e.g. ps/2. If you are building an embedded board like rasberry pi and you want to add ps/2, you just add a couple pins. If you want to add usb, you buy an extra chip.
Polling is often lower latency than interrupts on modern hardware, since it can run at much finer grains and has so much less overhead to startup over an ISR.
1 or 2, depends on the kind of modem. VDSL2+ modems might have three, though one of those is the line driver which is zero-delay for the purposes of a computer.
Modems/routers don't use polling nor do they need debouncing. The default USB polling rate is 125Hz, which is already 8ms of latency. Debouncing depends on the switch and is generally a couple milliseconds as well.
