You're basically referring to noble metals and the like. We use them because they are useful in very small quantities. That's not the case with lithium.
I look forward to 10-15 years from now when lack of scalability is proven and comment sections will say exactly what you're saying here as if everyone knew it all along.