Hacker news is becoming weirder and weirder...

Votes don't get rolled back; must be some display artifact.

Nor did anyone downvote his comments. Avg comment score is not all time but a window of the most recent. It only gets recalculated occasionally.

#1:

When you click the up/down arrows, it sends the vote in the background, using javascript.

So if you upvote + immediately refresh, that will cause the javascript submission to cancel, thus the vote won't count. (e.g. when you refresh a page, any async javascript queries which haven't yet completed will be canceled due to the refresh.)

#2:

Your average karma is the average of your 50 most recent comments. However it's only recalculated every so often. Specifically, each time you click the "add comment" button, there is a small random chance that your average karma will be recalculated right then.

e.g. the act of writing a comment is the trigger for recalc of avg karma; but it doesn't recalc every single time you write a comment --- rather, only every ~20 comments or so. (HN does this because it would be prohibitively expensive to recalculate avg karma for every user for every comment.)

So it's likely you had an avg of 1.64 in the past, which dropped to 1.0 over time, and that fact only became visible to you after your recalc.