The issue isn't (only) the focal distance. The difference is that your desk monitors don't encapsulate your entire eye cavities and cause pupil dilation, pumping blue light directly onto your retinas, which has been suspected to accelerate macular degeneration, night-blindness, and dry eye, as well as cause poor sleep.
https://www.webmd.com/eye-health/blue-light-health