I'm a bit late, but I wanted to comment on a number of things. The Riemann hypothesis is a conjecture, so I'm not sure what you mean when you say "nothing about the Riemann hypothesis is rigorous." Do you mean 'not rigorous' = 'unproven'? I think your comment that "Physics doesn't lead to insights in maths" is just plain wrong. There a number of topics in mathematics that were hugely influenced by the study of physics. In fact, much of 17th and 18th-century mathematics became inseparable from physics. That said, I'm a bit confused when you try to distinguish the two subjects because I feel that large parts of them are entwined. So, what is 'physics' exactly according to you? For me, the Wikipedia definition suffices:
Physics is the natural science that involves the study of matter and its motion and behavior through space and time, along with related concepts such as energy and force.
To study physics, we need maths. So does your definition of 'physics' include maths?.. or is 'physics' just observations? Take for example the subject of Quantum Mechanics. There are models that we use to make predictions about what we observe. And these models are mathematical. See what I'm trying to get at? The language of physics is inherently mathematical in nature (pun intended).