Obviously when you start taking high doses and reality kind of breaks down the distinction becomes moot, but at regular doses it does matter.
And the substance (as long as it is a tryptamine) is less important, save for personal preferences. (I have heard a lot about LSD for fun and shrooms for insights - for me, there is no such distinction; if anything, LSD, even in lower doses, produces more profound associations, ones that words cannot capture. At higher doses, well, as you said, the distinction becomes mood.)
At the same time, the soul (overall personality, mind, motivations, desires, approach, will, and goals) is crucial.
Funny, I usually have the opposite experience :) LSD for deep, not-so-much visual, inwards journeys and mushrooms for very visual, laugh-y and social experiences.
When I'm on LSD, one of my favorite things to do is talk to friends over text. It results in me saying stuff that I could never say otherwise. Here is something Emily (a headmate of mine) once said:
> ....there must always be sadness... because all that is good will always end.... and it will never stop being sad that it is this way~
For me, thinking like that just does not happen without LSD, it was a complete accident but it's so pretty!
I'm 53 now and hope that before I die (which I hope isn't anytime soon) there's a safe and legal way for me to try LSD.
That said, I must highly recommend not using in attempt to “fix” one’s aphantasia.
Edit: I recommend to stick to shrooms if you can.
I'd love to tell you, but I don't know how people without aphantasia experience LSD. What I do know is that my inner experience is only (mostly) blind; it has fully developed senses of hearing, touch, smell and taste. I get visual distortions, apparently within normal parameters. Thus far in the few experiences I've had with synesthesia, it's never involved my sense of sight.