Yes. You do. If I'm casually browsing non-intensive web pages, e-reading, or watching a Netflix movie, or even encoding a video, the background is doing IO rate limited system updates and basic data logging, etc, the vast majority of the time the CPU demands are very low, but frequent enough that putting a CPU to sleep is completely out of the question.
An A53 has a much lower ceiling, but a much better middle tier power usage level, than the A57. Yes, if you want to run a benchmark the A53 is not a good bet (and is generally worse in a workload power usage), but it is a very good bet for most real world usage.