Let's use an example: https://godbolt.org/z/TP6n4481j
The code shows main immediately calling a nullptr. What the compiler generates is a program that calls unreachable() instead. These are two different programs. If memory safety is "just" a bug, this would be a miscompilation. It's not a miscompilation though, because what I've given the compiler is something that resembles C++, but is actually some similar language where null dereferences are meaningful. The compiler only knows about C++ though and C++ doesn't have nullptr dereferences, so it assumes I haven't done that. Instead it generates a program corresponding to an execution trace that is valid C++, even if it can't see the call to NeverUsed(). If you use -O0, you get the segfault as expected.
A single instance of memory unsafety (or other UB) can take your program arbitrarily far from "correct". All other things being equal, a program with 1 violation might be just as incorrect as a program with 100. I could add a hundred more lines of safety violations after Do() without changing the compiled behavior. You don't even need to execute the unsafety to have "spooky action at a distance" cause that change.
[0] https://web.archive.org/web/20190406194101/http://colinm.org...