But is it saying "'affor affor affor...(repeated 100+ times)" really fast?

But actually, did you download the linked model from Kaggle ("gemma-2b-it-gpu-int4.bin") and upload it into the demo? It is working fine for me out of the box.

this can happen if you try to feed a cpu model to gpu inference. make sure you download the gpu model if you are using gpu accelerated ml inference.

It thinks it’s a pokemon.

Try a different prompt