Manufacturing is rapidly scaling up. There is so much lithium available in the world (even commercially reasonable to extract from the ocean), that there is much room for per unit costs to decrease before you encounter lower COGS bounds (evaporation + transportation costs).

More importantly, a ton of lithium mined will be used for decades; when all of the Model S/X/3 battery packs are end of life (10-15 years from now), those modules are going to be remanufactured into stationary storage (or recycled entirely through a destructive extraction process, depending on degradation and next use case). This is similar to how your recycled pop can might end up in the aluminum used in a new light truck or an aircraft fuselage (edit: poor example; a better example would be automotive parts that are remanufactured and put back into service).

The smartest thing Tesla ever did was finance battery manufacturing using luxury vehicle margins (stoking demand with a sexy, desirable brand), and have those customers (including myself) finance the depreciation and capital carrying costs of those battery sleds they'll use again in the future. I am not a Model S owner; I am the temporary user of 100kw of energy storage, which will eventually make it's way into stationary storage where mobile energy density (ie pack degradation) isn't as much of a concern.