I do. I bookmark interesting threads to look at again in a month time or so, when all comment dust has settled, and I have a better perspective.

I noticed that downvote buttons disappear after a while (24 hours after posting is my non-scientific impression), but upvote buttons never do; If e.g. 95% of the votes happen within 48 hours of posting, it might make sense to drop the upvote buttons as well at that point, and display the points -- if you believe they are useful.

Personally, I'm happy with the comments just sorted by points and don't care for the actual number -- although, while you're at it, I would be happy if there was a controversy indicator of some sort -- when I read threads, I'm often interested in the consensus and the controversial. I can find the consensus in the first few posts; I can't tell where the controversy is.