Logistical problems? Yes, there would be quite a few. As it is, any given member has to suck up to moneyed interests, just to be able to afford to become elected.
If any given rep only had to sway a few 10s of thousands, the barrier to entry is much lower.
Positions which are considered "fringe" would have some say, and there would be a lot more room for third parties.
I certainly think it would be a lot easier than some of the structural changes often proposed.