One argument is often patent trolls don't make anything themselves. So someone who makes a truly groundbreaking invention but for some reason (e.g. it requires huge capital) does not want to create a company for manufacturing it, should not be able to use patents? Also if they would actually make a company to build it and then try to go after a big player who infringes on their patent, if the big player doesn't want to pay the big player would most certainly win. They would just bring out lots of their patents that the small player supposedly infringes and thus make the suit to expensive to win. Essentially, if being an actual "builder" becomes a requirement, only big players will profit, because they have the pockets to kill off any small competitor through costs.
The other argument is patents are to bogus/general. Which is true, but what is the alternative, make the patent investigation even more elaborate and costly (it already is very expensive), so again only the big players gain.
The problems people point to with patent trolls, are problems inherent in the patent system. Patent trolls are just a convenient boogey man to blame so the big players can keep a system that disproportionately be if its them. There is virtually no benifit to the patent system overall and it should be abolished.