Also consider that even if that 5.6.7.8 router knew how to route to the 1.2.3.4.5.6.7.8 network, it would have no guarantee that the packets wouldn't hit another router along the way that didn't understand the extra address bits. You could end up with weird routing loops and other issues. (Fortunately TTLs would quash these, but not after wasting a bunch of extra resources.)
Now, there might be some clever ways to work around this, and it might require some more internet infrastructure to deal with these routing challenges. Maybe that would have been faster and cheaper to deal with than the current IPv6 mess we have, maybe not.