Have you considered a TI-89? They are basically like early microcomputers, frozen in time. They boot to BASIC and can also be programmed in C and assembly with no operating system to speak of. They're handheld and battery powered and durable, they come with useful math software for school, and there's a hobbyist programming scene and a big library of software and games written by kids for kids. That's how I learned to program.