Platform native UI conventions are very often sub-optimal, if only because they're old. But sub-optimal standards are very often preferable to unpredictableThe reason that junk like flat design and derivatives like Material Design are awful for usability has nothing to do with being old and everything to do with being unpredictable. Often, a user literally can't tell what parts of an interface are interactive or how they work, because affordances barely exist. It's like the old mystery meat navigation meme for web sites, except they actually did it seriously and thought it was good.
But it still sounds like you're using worthless methodology by focusing only on your one app at a time and ignoring how it fits into its environment and the user's broader workflow. Is that correct, or have you actually quantified the overall productivity loss an app introduces by violating the user's expectations and habits?
Well, firstly, a testing methodology is literally the opposite of worthless if it gives you an objective measure of the increased financial value generated by a change under consideration.
Secondly, you assert without evidence that the kind of change we're talking about does violate the user's expectations and habits, and you further imply that this causes a loss of productivity. As I have argued in earlier comments, the assumption that the user's expectations are governed primarily by their native platform's conventions is not necessarily valid any more, because users spend so much of their time inside a browser using online facilities instead of other native applications.
Moreover, the answer to your other question is yes, we have done many tests over the years that compared options including the native approach on various platforms with some other options we were considering. In the nature of such tests, the outcomes varied. In some cases, we did end up going with presentation similar to the native conventions on one or more platforms; often this coincided with cases where the native conventions across major platforms were similar as well. In other cases, we went with a completely different presentation style, as performance with the native conventions was significantly worse.
The point of all of this is still that ideally you don't want to make UI decisions based on assumptions or dogma if you could try different possibilities with real users and make your decisions based on objective evidence instead.