However, they will likely find it harder to use without an explanation why. If most of the apps a user uses follow the guidelines of Android/iOS, and they are primarily a user of one platform, and your app doesn't follow either, it seems obvious that they won't be able to use their built-up knowledge of how apps work in general to navigate your app.
Most people I know, including myself, have found it initially a little more difficult to navigate around apps from the platform other than the one we are used to using daily.
Apps that insist on creating the same UI on both platforms can be a mixed bag, IMO. Sometimes executed well, sometimes poorly. Because of this, I usually prefer apps that separately comply with each platform.
I am however interested in playing around with Flutter soon!