I think the magic comes when you discover that something your default mindset was filtering out as noise is actually signal. With that insight, you can more easily make changes to your sober mindset.
What you described sounds very similar to the effects of mindfulness meditation. The idea in a sense is to increase awareness - that is, sensing the world as is, without the filters or influence of mental objects. The theory is that meditation can then help you free yourself of past negative conditioning that is no longer relevant and to be more present.
It would be interesting to see more studies on both, but I understand that many things to do with the mind and subjective experiences are difficult to measure and therefore to study.
check it out
At a cellular level, it temporarily vastly increases the connections between the synapses. A lot of what your brain does is travel along well worn synaptic pathways, like running on autopilot. This is when you feel like you're in a rut, it's because your brain for efficiency reasons is just doing what it always does. The temporary frenzy of increased synaptic connections allows people to break out of those ruts
In it, he talks about how his early experiences with psychedelics turned him on to the fact there was something real worth pursuing in meditation and mindfulness beyond the superstitious bullshit. Most of the book is about using meditation as a tool for spirituality without religious baggage or supernatural explanations.
The way he and others talk about it, is that psychedelics effectively force any mind into an objectively different state of consciousness. Even though it doesn't last long and the memory of it fades, just the awareness that radical mind shifts are possible is enough to help people take mindfulness practice seriously.
Regarding microdosing (as described by the OP) this begs the question: Are there perhaps unwanted side effects for running your brain on permanent "debug mode"?