I use ^[, and I use my pinky a lot less than I would in emacs, because I do a lot more in command mode than I do in a single emacs chord. I view "insert some text" as its own command. Vim's commands are structured so you can combine orthogonal pieces, where emacs has a separate chord for everything.

It actually is. At least it's a "home-row pinky". However, I remapped esc to be "jk" to avoid the pinky abusing.

Now, if I ssh between different unix box a lot without the time to pull my .vimrc from github, then I still have to resort to pinky-escape.

Hm, fair complaint. At some point I found that hitting ESC immediately after typing became habitual. I find myself accidentally doing it outside of vim quite frequently. I've heard others make similar statements to yours, though. If rebinding to a different key doesn't solve it, then modal editing may not be for you.