I'm sure something like this is possible in Emacs, but in vim I have it set up so that my current line shows the absolute line number and other lines show the relative line number.
If I'm eyeing a particular line that I want to jump to, I just read off its relative line number and hit j or k, according to whether it's down or up from where I'm at -- I'm there. Because I use relative line numbers, the number is never more than two digits.
For more precise navigation (jumping to a particular spot on a particular line), I use a plugin called EasyMotion, which basically lets me jump to any text object on the screen with a few key strokes.