> self-hosting an XMPP server isn't something you can just install and have it work
For basic needs ala IRC it should just work out of the box by defining a virtualhost. For respect of advanced features as defined in the XMPP Compliance Suites, it'll need a few more settings but there's good guides:
prosody: https://homebrewserver.club/configuring-a-modern-xmpp-server...
ejabberd: https://www.process-one.net/blog/ejabberd-xmpp-server-useful...
All in all it should take a novice (who has some experience of sysadmin, can deal with DNS records, firewall, etc) a few hours to setup to their taste... Or you could go with Snikket, a XMPP distribution, to have everything pre-configured in a docker container.
> XMPP clients on iOS were unusable
Sorry I don't know about iOS mostly because my acquaintances can't afford Apple devices and/or are not interested in a walled garden where a private company owns all your computations. Though i heard similar stories as yours, but there is progress with Siskin, with some development (eg. OMEMO groupchats) being funded by the Snikket project.
> notifications sometimes don't work
This may have to do with your OS settings. I don't know how iOS handles low power mode but i know for sure some Android distros restrict network queries (apart from the privileged, centralized push notifications service) and there's settings to uncheck for that.
> in a business situation
In a business situation you might find the resources to implement the features you need or pay someone to do it. I understand it may not be the answer you expect, but that's how free-software ecosystems move forward. In this regard, XMPP is not named "eXtensible" without reason.
> it needs to work well enough across iOS/Android/browser to not drive people away
Sure. I think that's the goal for everyone. If there's something that doesn't "work well enough" then it's a bug and should be reported then fixed.