To me, part of the value of Lebesgue integration is in understanding the limitations of Riemann integrals and when they break. Some of this is covered in Stroock's book in chapter 5.1. Alternatively, when in working in function spaces, we may need to integrate in a more general way than Lebesgue integration, so things like Bochner integrals, which require similar theory. This can arise in the theory related to things like PDE constrained optimization, which most of the time is targeted toward physics related models.
All that said, bluntly, I prefer to work with Riemann integrals when at all possible. However, the same question then applies. Do you or someone else have a reference for a rigorous derivation of the divergence theorem or integration by parts in multiple dimensions using Riemann integration? It's not particularly hard in one dimension, but higher dimensions is tricky and it's hard to get the details of integrating on the surface correct. Stroock's book is the only reference that I know of and he does it with Lebesgue integration.
The more practical advantage measure theory provides for probability is you can simultaneously handle continuous and discrete distributions. Most of the time what works for one works for the other but you can get some weird mistakes (Shannon’s differential entropy has a few issues as a measure of information not found in the discrete case because he got lazy and just replaced the sum with an integral).
A good chunk of the time I come across measure theoretic probability papers I feel like they’re making the paper a lot more complicated and messier than it needs to be, but it does serve a purpose.