The reason is that the discussion is nuanced, a smartphone plays a very different role in someone's life than a Nintendo Switch. For better or worse, a smartphone has become a requirement for critical services that we all rely on.
That doesn't really align with the arguments people actually put forth for sideloading on iPhones. "It's my device, I should be able to run whatever I want" is the most common, and doesn't leave room for the nuance you're referencing. Like it or not, most people arguing for iPhone sideloading are using arguments that also demand sideloading on game consoles. It's not at all a straw man.