The thing that makes them not painful in bad times is that often the payments stay fixed, and the extra interest is tacked onto the end (meaning that you are gaining more to pay). (you can get a mortgage where the payments do immediately change as the rate changes)

That being said it is possible to hit the "trigger rate" where your fixed monthly payments are no longer even covering interest alone, and then the bank will give you a call to make your payments go up.