Anecdotally I do know someone who had a psychotic episode on psychedelics and spent awhile in recovery. They eventually did recover, but that did make me wary.
As for your point about doses - low doses are hard on chemicals because you don't know what you're getting. Using mushrooms, well dosage varies wildly per gram depending on what you've picked. It wasn't a point about addiction.
I have had great experiences on threshold doses of mushrooms (and experimented on much higher doses of acid in my past), but I do think a healthy amount of caution is generally good.