I think the upgrade path from 2.3 being terrible is generally accepted as being true. But I don't remember any hard problems from 3.x onward.
At least for Rails itself. Gems dependencies are another problem.
Like at one project we had pains every time Rails upgraded their minor version because the previous devs thought using Squeel instead of the builtin ActiveRecord a good idea. Just for being able to write slightly "nicer" queries and now this is a major stopper for going to Rails 5.
