To elaborate, personally I draw the line at peripherals. I do try to use different devices for work and not-work, but I use the same monitor+keyboard+mouse+headphones (and desk, and chair...) for both.
The Vision Pro is a weird middle ground of device and peripheral. When it's operating as a virtual display it's more of a peripheral, and it'd be nice if it acted more like one.