Well the holdings also create a conflict of interest in how they choose to vote.
There's no actual reason a congressperson needs to own stocks. In fact, we should be doing what we can to enforce a certain "Average American"-ness to their lives so they work to improve the lives of the average american.