If you don't want to run a completely custom setup, there's projects like mailcow out there that can do the heavy lifting for you.
I really don't see a quantitative or qualitative difference between the gmail experience and mine, with the caveat that my setup doesn't label ham from other private mail servers as spam (arguably a good thing)