Self hosting means hosting it yourself, not running it on Amazon. I think the distinction the author intends to make is between running something that can't be hosted elsewhere, like ChatGPT, versus running Llama-3 yourself.

Overlooking that, the rest of the article feels a bit strange. Would we really have a use case where we can make use of those 157 million tokens a month? Would we really round $50 of energy cost to $100 a month? (Granted, the author didn't include power for the computer) If we buy our own system to run, why would we need to "scale your own hardware"?

I get that this is just to give us an idea of what running something yourself would cost when comparing with services like ChatGPT, but if so, we wouldn't be making most of the choices made here such as getting four NVIDIA Tesla T4 cards.

Memory is cheap, so running Llama-3 entirely on CPU is also an option. It's slower, of course, but it's infinitely more flexible. If I really wanted to spend a lot of time tinkering with LLMs, I'd definitely do this to figure out what I want to run before deciding on GPU hardware, then I'd get GPU hardware that best matches that, instead of the other way around.