As others have mentioned, it's not really the technical aspects of SMTP that make hosting your own email difficult. It's the entire deliverability ecosystem that's developed (ostensibly to protect from spam), such as most residential providers blocking outgoing port 25, or most mailservers filtering incoming mail from dynamic IPs.