The cost of context switching between threads sharing the same address space is much lower than between tasks, because there's no need for a TLB flush. It just comes down to the cost of saving/restoring register state (more or less), and that's a nice known fixed cost on any given processor.
Context switching between tasks has a variable cost, because the implications of the TLB flush depend on the working set size of the switched-to task. If it doesn't touch much memory, the variable cost is low; if it touches a lot, the variable cost can be much, much higher than the register save/restore (fixed cost).