I tried using a Game Boy Advance SP with my son. It has no internet connection and he gets to play some fun stuff. It worked really well!
But when it comes to reducing my tech footprint, the only issue I found with going down this path were bank apps. It is basically a requirement to have either an iOS or Android modern device to do anything.