Just want to say I fully agree with you. react-router seems like a useful library, but I see utterly no reason why you should write your routes in JSX instead of as a simple JS object. Using JSX for anything other than DOM insertion or modification creates an impedance mismatch. The public API makes no real sense to me.
The point of JSX is to more easily build dynamic HTML (and perhaps other markup).