You don't seem to have understood what's going on in Signal. Ordinary key rotations, which happen automatically, do not change the verified status. What can happen is that another participant changes phone or wipes it, and so obviously trust can't survive that change.

The problem isn't that somebody else may know the correct key, the Double Ratchet takes care of that. The problem is that a Man-in-the-middle is possible. Alice thinks Mallory is Bob, and Bob thinks Mallory is Alice. Mallory can pass messages back and forth seamlessly, reading everything. Only actually verifying can prevent this.

You don't verify the encryption keys, that's useless because those change constantly, the verification compares the long term identity value ("Safety Number") for the conversation between two parties, which will be distinct for every such conversation. Mallory can't fake this, so if Alice and Bob do an in person verification step Mallory can't be in their conversation.