It behaves like one though. It is only found in low concentrations and never in pure form or easily accessible ores.
Lithium brines have concentrations of well over 50 ppm. That sounds low, but given that many other minerals are economically viable when measured in ppb, 50 ppm is not low at all.
You're basically referring to noble metals and the like. We use them because they are useful in very small quantities. That's not the case with lithium.
I look forward to 10-15 years from now when lack of scalability is proven and comment sections will say exactly what you're saying here as if everyone knew it all along.