No, voting doesn't work at all with JS disabled.
These are my network requests when I vote with JS enabled.
It's not any lighter, it's just not reloading the page.
The page reloads and re-renders with the UI changes (up-down buttons hidden and unvote link added). However, the reload takes noticeable time, and you lose your scroll location, which is disorienting.
With JS enabled, the UI changes happen instantly, and the scroll position doesn't change. The page still reloads, but the reload is directed into a new Image object which is immediately garbage, so it's off the UI timeline. But it's inexplicable bandwidth waste, nonetheless!