> Likewise, the raw pointer values don’t have an effect on the behavior of the JavaScript
There are actually some cases where raw pointer values can effect user-visible browser behavior indirectly. Notably if the pointer values affect iteration order in an associative data structure which the browser uses in some way that tickles JS differently depending on the order.
There are bunches of other bizarre edge cases you'd never think of as well that never come up (until inevitably one day they do). Another example: In old versions of Blink I've seen ligatures fail to form depending on what was in the font cache when the page was loaded.