To call that "productivity [that has been] lost", means that there was productivity that should have happened and was somehow stolen.
Sure, there was "productivity potential" in those moments, but that's true for every moment. E.g when we're sleeping, making love or eating. We do we think we should be productive almost every second of our lives, or we "lost" something?
I had a teacher in elementary who didn't like her students staring at the ceiling while she taught. Thinking that we were trying to escape from the lesson, she put several famous paintings (Mona Lisa, Sistine Chapel, etc) on the ceiling so we would at least be learning to appreciate art while in her classroom. There was nowhere in the room to look which wasn't educational.
After iPhone, I read HN, Reddit, or Slashdot. So, at least in my case, there has definitely been a hit to productivity.