I'm sure the plastic is fine, I'd just prefer a metal or glass version.
On every discussion of the Aeropress, I've seen the ask for a non-plastic version pop up. Every time I've then seen essentially your reply above. I'd just prefer a metal or glass version.