"A GIF is literally a sequence of independent images squeezed into the same file. An mp4 video can take advantage of all kinds of fancy compression techniques like keyframes and forward-predictive frames."
That's not why the mp4 is smaller. A GIF is not [just] independent images, you can have each frame reuse pixels from previous frames.
The mp4 is smaller because lossy jpeg compression is used on each frame, while the gif stores each frames losslessly.
This has implications from the predictive frames as well - the jpeg is lossy, so two almost similar parts of the images can be considered "the same", while the gif is not, so they must be identical in order to compress that way, and the realities of video noise makes that unlikely.