In the case of a randomized group, that power isn't with the group, it's with the people who educate them on the matters and who frame the issue. There are a million ways to present any matter, and none of them are objective. You're putting the power to influence and even determine a nation's decisions in the hands of the experts that inform them, rather than the randomly selected people. It would become very efficient to buy those people instead of the voters.
The problem of the matter isn't whether it would function worse. The problem is that it would function less transparently. It would ostensibly be a fair and completely unbiased system, but that could be manipulated behind the scenes in far more insidious ways. That's very dangerous. Personally, I would rather have a more unfair but more transparent system.