The Peano axioms don't do very much, they just help model the natural numbers. You still need something to model the logic and provide foundations more complex subjects like analysis.

> simplicity

It might be possible to build a set theory and the rest of mathematics based on the integers, but I am sure the result wouldn't look simple.