You can flip through the pages of a book much faster than it'd take you to actually read them normally. But if you're putting the book in a scanner then you're still extracting information, faster than the "natural speed" of reading them.
My point was that the "natural speed" of video (i.e. the frame rate - clear slow motion requires more frames, while fast blurred motion will generally have fewer) does actually impact how much data must be processed. It's true of audio too (sample rate) and downsampling is a common optimization technique.