It's not all good or bad, there's a security issue with side loading, as well as shovelware on the play store. However, there is no world where I would argue that these justify limiting consumer grade hardware to walled gardens.

I recommend getting an Android phone (there are cheap Google Pixels out there) and try to sideload an app. Also browse the web a bit without an adblocker. I'd be surprised if by the end of the experiment you thought that sideloading is the reason their grandma's phone is full of crap.