I don't think the solution is just getting rid of them all -- actually a longer grace period (say, a year?) would make it easier to leave abusive employers. But it should include more scrutiny on whether employers really can't find local hires in the first place, or just aren't willing to train them or work with people that have even slightly more bargaining power.
Abuse of the H1-B program is broader than just hiring people for less money right now. It's also about hiring people who won't ask for more money over a longer timeline. It's about having a whole class of workers that is uniquely vulnerable to terrorist bosses, which then gives those employers more leverage over everyone else. And it's about the large-scale market manipulation that comes with having this easily accessible.