You aren't missing anything. This problem has been there in WP for a long time, and it makes staging/testing/migration setups a horrible pain (as others have suggested, lots of search & replace).