On of the hardest parts is spam filtering both for incoming and outgoing traffic. In particular outgoing, so, e.g., a compromised or malicious user, can have harsh repercussions by big providers like M$ (Outlook) and Google. You can find a myriad of stories just on HN about these platforms literally not even allowing you to apply for access after an incident happened.

And your users won't care that M$, Google or Yahoo behaves like this, they will only care about their emails not being delivered by/ to you.

I had an internship at $company that offers manages hosting for Email along with web shops. The technical setup is the easier part of the business.