The Intel compiler used to (and possibly still does) set the SSE FTZ bit by default, leading to a situation where two different FPUs on the same chip (x87 and SSE) had different FP configs. I was indeed thinking of ARM though. I've come across chips with both VFP and NEON where they're handled differently.
For NaNs, if we're being strict then you don't need to reach that far. Here's one example from LLVM:
https://github.com/llvm/llvm-project/issues/43070)
This can occasionally result in visible bugs, like:
https://github.com/rust-lang/rust/issues/110174
You have to be pretty nitpicky to care about this stuff and it's not an issue in practice, but it's a reproducibility violation because it deviates from the standard nonetheless.