I have several variations of those.
It now becomes either something I have to carry around with me always; or buy one for each 3.5mm headset I've stashed.
Add the poor reliability and performance, and we come back to your point:
"Yes we shouldn't have to, yes it's a step backwards, yes it's not as good"
Exactly. So I will absolutely do whatever I can to make my life easier - but I also refuse to pretend this does not make my life needlessly harder - once again, for no obvious gain for those who defend the decision.
Phone jack is inherently different than all the Firewire/Zip drive items. There are literally thousands of technologies, interfaces and standards that live and die like fireflies. But the phone jack is a century old and still works perfectly well, and we have replaced it WITH the technology of the day - who thinks their $300 Bluetooth Bose will work 20 years from now? Or seemlessly plug in to each device they use?