When logged in the site shows you for each entry (post/comment) whether you voted on it or not, thus it has to request a store with per user unique data. (Also custom color, your points, ...) When logged out all get the exact same data.
Probably because it can be cached instead of serving customized pages