I love the fact that I'm seeing how stuff works. Before theory, people built huge buildings, cars, airplanes, magical electrical circuits etc.
Neural networks are the same thing. Bunch of heuristics that will later be proven in theory. Yet still work incredibly well. Like cathedrals did without any mathematics.
Says a lot about how tinkering with heuristics is powerful.