I am over 32 so it does not seem old, still it's a late start even when everything goes well. Many specialists start closer to 40.
Residency starts out with extreme levels of supervision. The end of fellowship is the point where they are considered competent enough to not need direct supervision. Though doctors still have plenty of people looking over there shoulders.
The point is 32 is when doctors can start paying back significant amounts of there student loans and saving for retirement. Including interest it's not hard to be 32 and have 1/2 million in debt.
PS: Also, specialist's can't afford to make the path cheaper for the next person as they need to constrain supply to keep there wages high. Which not only increases cost of healthcare, but also medical research.