One handed? So no pinch-to-switch-apps gesture? (it seems odd to point out one handed operation of an iPhone feature when so much of the reason people say it is superior are the things that require two hands to operate).
In this case, the real alternative to the task switcher is flicking through multiple pages of apps, which surely is more difficult than a simple row of recently used apps. It's hard to imagine a pinching gesture that would be usable here, without interfering with the current app.
However, if a user is switching between three or four apps for a given period of time, those apps may very well lie on very different pages of the Springboard. The most frequently used apps are not always the apps that matter now.
Even when you would need to swipe in the task switcher, surely it is easier to find an app among 4 instead of 16?
(Maybe the task switcher really is only useful for a small subset of use cases, in which case your position would seem to be the logical one.)