Wouldn't it be feasible to add a tiny battery or capacitor? Assuming the radio doesn't need to transmit continuously it can be powered via those which are then powered by the "forever battery".

Yeah you'd need a lot of them. Instead if a AA battery maybe you need a truck's worth of this. But if it means you dont need to go back for a while...