Hacker News Reader

To show the cost of extra parameters is exponentional, they made a graph with price linear and parameter count logarithmic and then fit an exponential? It seems very misleading:

https://proofinprogress.com/assets/images/cost-llms.png

With linear or polynomial increase per parameter it would also look exponential with that graph setup.

View on HN

mhh yeah you might have a point. I didn‘t do it on purpose though. I‘ll go through this again

Matrix multiply scales polynomially so don't be thrown off by that. You'd also expect some increases in cost where things move from single GPU to multi-GPU to networked machines. I'm still not sure if that pushes things to exponential or would just step up cost at each transition.

Reply on news.ycombinator.com