I don't think that's the answer to the backspace=^h question, but HJKL is precisely an artifact of the keyboard that vi was first used on: https://en.wikipedia.org/wiki/File:KB_Terminal_ADM3A.svg / https://en.wikipedia.org/wiki/Vi#Creation - in addition to HJKL being labeled as the arrow keys on that keyboard, also note the far superior placement of escape and control where modern keyboards typically place tab and caps lock; this also made vi a lot more sensible.

(And of course that last bit is a whole topic unto itself; to this day, caps lock is the dumbest key on the whole keyboard to me - mostly useless, takes up prime real estate, and 100% worth remapping on every machine I control. But that's a rant for another day:])