Statistics has four major branches: inference, exploratory data analysis, experimental design, and visualization.
Your question is probably mostly asking about inference. I like the first few chapters of Kiefer's book to get the context of decision theory. Then you're going to have to wander far and wide, but there have been some good recommendations. Since you're digging into Dedekind on the side, you might like to go back to some of the classics here as well: Wald's "Statistical Decision Functions" and Savage's "Foundations of Statistics."
For exploratory data analysis, Tukey's book "Exploratory Data Analysis" remains the place to start, if you can find a copy that doesn't cost a fortune. Casella wrote a book on experimental design that's solid. For visualizations Wilkinson's "Grammar of Graphics" and Tufte's books are the usual recommendations.