For example, on X86_64, the ISA provides additional GPRs that aren't just 64-bit versions of the 32-bit register. For example, rax is the 64-bit version of eax, but IIRC you also have r0-7, for which there is no 32-bit equivalent. Furthermore, the ABI (at least on Windows) specifies a different function calling convention versus X86.
Additionally, I imagine there additional instructions/ops on x86_64 over x86. I dont know how else the cpu would distinguish add %eax, 1 from add %rax, 1, both of which are legal when the cpu is running in 64-bit mode.
I recall seeing a cool talk on how to confuse or crash many debuggers by doing something clever in assembly. The idea is you would write a block of polyglot 32 and 64-bit x86/64 assembly (i.e. binary that is both a valid x86 and x86_64 instruction sequence), switch the cpu from 32 to 64 bit mode at the end of the sequence, then branch back to the start of the block and reinterpret the same instructions as 64-bit rather than 32. You could use this technique to frustrate reverse engineering.