But maybe I’m wrong, it might be a purely software based problem that can be solved quicker.
I agree entirely!
There are a bunch of hardware out there to get gaze vectors, the problem is that they rely on a generalised model of the eye. With fine tuning you can go from >5 degree of error, to <1-2