I've personally not found this difficult. Unless perhaps you're dealing very large volumes of mail and overhead of SpamAssassin is an issue. Mail volume in 10s of thousands/day on a low end VPS has been no problem for me.