1) Everything in the universe, including the sun's solar radiation, is finite. It's just a matter of scale. The heat in the earth's core is practically infinite for humanity's current and near-term needs.
2) The heat generated in the earth's core is a combination of fission decay and residual heat from the earth's initial formation. The fission reaction is locked away 5000 KM deep (as in 5 million meters), deep-enough that it poses no threat even if you drill down thousands of meters to tap into that. That is very much not true of man-made nuclear fission.
3) There is still no long term storage mechanism that is guaranteed to keep nuclear waste for the several million years it would take to decay to safe levels, considering possible things that could happen over that time scale.