There is no extension mechanism built into v4 for longer addresses.
Of course v4 is going to be with us for a long time. We can't make existing v4-only devices go away because we have no way to enforce a flag day on the Internet, so there was never any way around that. But you can run networks without v4 just fine if you want to (including the ability to connect to v4-only hosts, or let them connect to you), so you can in fact turn v4 off on your network.
> That's an address-space expansion, not just a weird way to do NAT
You said the server would have a proper v4 address and the client would put its RFC1918 address into an extension header. You don't get any extra address space that way. But if you do try to give nodes 8-byte addresses... you immediately get the same situation we have in v6, because nodes and software that only know how to deal with 4-byte addresses won't know how to deal with your 8-byte addresses. You'll end up having to use the same approaches v6 uses to deal with the same problem.
> I have no doubt the IETF could have embraced NAT as a transition technology quickly obsoleted by broad adoption
What do you think RFC 2765/2766 are about? Or RFCs 6144~6147?