"Whataboutism" is an informal fallacy that most non-technical folks (as in, not logicians) mis-invoke, like almost all other informal fallacies. My argument is not even remotely whataboutism. In fact, it can be even formulated without addressing the "other side" at all: you're making a category error by going from general (party trends) to particular (individual voters).

It's simply a bad argument. I only brought up the "other side" to help analogize it, my apologies if it confused things.