RE the budget surplus, the dotcom boom led to a large increase in taxes collected which collapsed from 2000 to 2003. Similarly countries naturally spend more when their population is either skewing old or skewing young. The US hit peak working-age demographics during the period you are describing. The outcome doesn't seem to be "engineered" so much as accidental. In fact taxes on balance were cut during this timeframe while significant contributors to the long term debt / future crisis were unleashed (glass steagall repeal which arguably led to the great financial crisis & federal loans for eduction were dramatically expanded without oversight to ensure the money was being well spent).
Similarly, the covid response caused an explosion in the budget deficit which I wouldn't blame squarely on "a candidate's economic policy". The 2023 deficit in real dollars and as a % of GDP the largest (by a fair amount!) outside of a war, economic crisis, or the covid response. This doesn't even count the ~400? billion in student loan forgiveness that was attempted and failed.
"Generally, looking at the last 30 years or so, it is evident that voters do not care about this issue very much" <- 100% agree.