There needs to be a chemical process to break down the harsh chemicals into something environmentally friendly. Ideally, something that uses solar power or other highly available energy sources.
Why? It makes more sense to remanufacture them into new batteries. This means less raw materials that need to be mined from the earth.
Because theoretically recycling all of them is a great idea but in the real world there will be lots of trash that will destroy the environment. Take masks for example, they now litter the entire world when all someone has to do is throw them away.
A $10k+ EV battery pack is not the same as a disposable piece of fabric. Even regular car batteries have a robust recycling stream, mostly due to the deposit you get back when you return your old one.