Just my 2 cents.
Numbers are used in a sequential manner naturally, and more importantly we subtract numbers from each other all the time. Basically, we have practice. Try reciting the major irrational numbers numerically backward. Until you've practiced, you might find it easier reciting them alphabetically backward.
I'm sure for children reciting numbers backward is about as difficult as reciting the first 10 letters of the alphabet backward.
To become facile with reciting the alphabet backward, I was taught to "chunk it" into small groups. ZYXW - VUTS - RQ - PON - MLK - JIHG - FED - CBA. I can now recite it backward almost as easily as forward, without chunking, even if I start in the middle and not the end. The only part I still sometimes get caught on is the order of I and H.