That's debatable. As far as I'm concerned I prefer being annoyed by my occasional lack of care rather than wondering what software quirk makes my phone call come out of the wrong device.
At least I can identify the cause quickly and make my best to avoid the issue.
I carry my backpack or have a jacket on me at almost all times, and having the headphone cable holes in each has prevented this from happening anywhere as often as it used to. Plus the reliability of wired headphones has reigned supreme that I sold my AirPod Pros a few weeks ago after they sat for about a month or two between use, while my corded headsets are used almost daily.
If you have an IEM with a sturdy cable wrapped around your ear, snagging it on a door handle yanks the connector out of the Lightning to 3.5mm adapter instead.