This is related to the mechanical design of VHS tapes and the player. VCRs use helical scanning. What this means is that each frame of video is recorded on the VHS tape in a linear track that runs diagonally across the tape. Inside the VHS player, you may remember seeing a drum in the back that seems at an off angle. This is the helical read drum: it spins at a very high speed as the tape is pulled around it, and the combination of the slanted head drum spinning and the tape moving along it at a lower speed results in the heads mounted to the drum tracing these diagonal lines on the tape.
The TV draws to the screen continuously, 29.97 times per second, and draws whatever is coming on the analog video input at that moment. So for the image to remain when you have paused the tape, the VCR must continuously play the same frame back to the television. This requires keeping the drum moving and alternating the tape drive (the capstan motor) back and forth very rapidly to try to keep the same helical track aligned with the head over and over. This never quite works right, leaving a noisy signal that often has a bit of a shake as the tracking goes in and out each frame. Additionally, the capstan rapidly switching directions can damage the tape, which is why most VCRs will switch to Stop after being paused for a few minutes.
Laserdiscs had this problem much worse, as there was simply no practical way to get the disc back to the beginning of a frame quickly enough to rescan it. Higher-end laserdisc players could show a freeze-frame, but they did it with an electronic framebuffer that was digitally replayed to the TV. I wonder if there may have been higher-end VCRs with this same technology, but I haven't seen one.