For a desktop with sufficient amount of memory swap is just bad from my experiences. When i was using swap on my desktop system, which i usually have running 24/7, i'd often come back in the morning to find that linux decided to swap out my X session or something stupid like that. How much of my 16GB memory will a "rarely used daemon" use? Insignificant compared to how annoying it is to have to wait for your interactive applications to be swapped in when that happens.