I switched into the Garmin environment some years ago. Have my second watch now and spread it all over the family. Never had I one person having any kind of issues with it and we have both outdoor and everyday models.
Venu/Vivoactive is a perfectly fine starting point. Move upwards if you need specific sensors, on-device music playback, etc. Upgrade to the Fenix if you want the better battery life and you're not bothered by the dim screen. Upgrade to the Epix if you want the better battery life of the Fenix and you can't live without a bright screen.
When I got my Fenix I thought the screen would bother me, but I got used to it quickly. If you've got a traditional smartwatch and you're moving to a Garmin, the Fenix might be an adjustment. Readability in direct sun can be problematic, (especially trying to read it on a bike) only you know whether or not that matters for your use case.