It's not a huge deal in practice in my experience. You could always buy a press that uses filters, such as the aeropress.
They are excellent though - practically sell themselves.
Coffee is not supposed to be bitter, and it's a shame that so many people think "Oh, well, that's just the way coffee tastes I guess.".
I recently got my parents one, and they went "Ohh, it can taste like that? That's really good!"
[1]: http://coffeecollective.blogspot.com/2010/01/updated-aeropre...