I tried it using oobabooga's webui side by side with Alpaca 65B loaded in 4 bit on the same AWS instance with 64GB of VRAM.

While Alpaca produced 3 tokens/sec, Falcon produced 0.17 tokens/sec.

So it is very slow with the current tooling still.