No, you're not alone. I almost always choose desktop mode where available, primarily because mobile sites disable zoom and choose settings for margins and font sizes that make their text almost unreadably large - frequently only two or three words per line, with masses of whitespace all around.

Reading such articles feels claustrophobic, because you're stuck in this long tunnel of text, with little visual indicator of how long it lasts, inducing anxiety about wasted time. If you're somewhere in the middle, it's a long trip back up to the top.

The close to optimal mobile browser for me was the original (AOSP) Android browser around 2.3. It rewrapped text on double-tap to fit in the visible page area, meaning you could choose what effective font size worked well for you. And zoom still worked, so you had a good handle on where you were on the page, and an easy way out of it.

Neither Firefox nor Chrome on Android are as good a browser for me. Unfortunately, getting the AOSP browser back is awkward, unless you want to use Dolphin or similar, and besides, it no longer has the rewrap functionality.

PS: Google are the evil people here. They deliberately removed this feature from Chrome and AOSP browser, and are punishing sites that don't serve mobile-specific content - i.e. content that is less usable for people who prefer zoom-based navigation.