Not just letters! "^X" indicates "X, but with the control-bit flipped", where the control-bit is the left-most of ASCII's 7 bits.
1001000 ASCII 'H'
1000000 "ctrl bit"
0001000 ASCII backspace
But like I said, it's not just letters, if you use `cat -v` to show all non-printable characters in this notation, you'll also see things like "^@" for the 0x00 null byte and "^[" for the ESC. All of the terminal color-code control sequences start with ESC, so if you pipe colorized output to `cat -v`, you'll see a lot of "^[".