How many tokens a second? Really trying to figure out viability.
4x NVIDIA A100 at lamda labs is $4.40 an hour and I really have not had an issue getting them.
4x NVIDIA A100 at lamda labs is $4.40 an hour and I really have not had an issue getting them.
https://www.reddit.com/r/LocalLLaMA/comments/16o4ka8/running...
OP implied that there were workloads where it out competes renting in terms of cost. Was hoping it was true for something than a single user interactive session (which can be done a lot cheaper)