I find 100tps unlikely, I see 13tps on an 80GB A100 for a 70B 4bit quantized model.
Can you link - I am answering an email on monday where this info would be very useful!
Can you link - I am answering an email on monday where this info would be very useful!