Up to 1.4 is not a big deal, somewhere around 1.5 or starts to get a little strange. Since I generally have technical or industry related podcasts going while doing other things too high a speed becomes a hindrance.
How precisely rubberband works is something I've not looked into deeply, but the end result is increased CPU load and less choppy audio at high speeds. If you throw together a little script to switch back and forth between rubberband and scaletempo in realtime, I think the difference is pretty dramatic.