They have an effect, but not enough of one to justify the level of collective hysteria we experience every four years.
Based on the current composition of the Supreme Court, replacing a single judge could have dramatic, fairly immediate, and nearly irreversible real-world consequences to the citizens of this country. Whether you agree with those consequences or not, you'd be hard-pressed to argue that the stakes aren't high.