It's a very easy problem to solve if you do it right - tie Conditional Access policies to you using the correct browser.

This is what the Island browser does - you can't log into any company resource unless you do so from the authorized browser.