In the late 90's I worked for an email server software company. Since then, almost 20 years now, I've been running my own mail server. I wouldn't say it is hard so much as a PITA and very time consuming. I spend quite a bit of time reviewing logs making sure that my server has not gotten hacked or compromised in some new way.

Over the years, a new exploit type comes along, like backscatter, that I then have to figure out how to secure my server against. And I'm also very proactive at reviewing logs and banning IP's against the constant barrage of probing. I must spend a few hours a week dealing with the mail server.

And then there are periodic "someone isn't getting my mail" problems that I have to track down, where Yahoo or pacbell.net or some other large mail system decides that my server is insecure for some reason, despite not being on any DNS RBLs (which are also sometimes a problem for no good reason).

If it weren't for the fact that I have so much more flexibility and ease-of-use on my own mail server for setting up many domains and multiple email addresses, I would move everything to a provider and not manage it myself.