not sure if i fully understand what you are trying to do, but encoding 60 frames per second will be hard. If you already know the color palette it might be possible, this lib spends about 80% of the time doing the color reduction.
it could be potentially much easier on the encoder if it did its own scaling instead of being handed scaled frames for analysis. my canvases are typically very small.
for slightly more context, it's for clearer, scaled up pXY.js docs/demos here http://o-0.me/pXY/#scanning