The DNS response doesn't have to come from the same channel as the original request. If you've got an ISP that doesn't check the source IP of what you're sending, your target's endpoint will see your fake response and treat it as the real one.

Where we stand now, the only thing stopping an eavesdropper from becoming a man-in-the-middle is the will and resources of that eavesdropper.