A big part of that is the regulatory environment. For example - I have the interest and means to get my PPL. I'd love to do so, but the FAA considers ADHD to be a disqualifying medical condition.
If I were to do it again I'd save up and take time off and do all the training in a month or two. Half the cash I burned was relearning the shit I forgot since the last lesson.
(Perpetual student - you can be in control of a plane but not PIC without a medical or anything, so you can literally pay an instructor to be PIC whilst you do the flybits.)