I bet when you see a red octagon, you just think of it as a thing on the side of the road that means "stop", don't you? But despite your not knowing its origin, not knowing why it means stop, it still works well right?
If stop signs showed the picture of the brake lever from a Ford Model T, you'd have a point.
I agree that the octagon is arbitrary. But that's beside the point, because it's self-evidently arbitrary. Nobody assumes there must be a deeper meaning behind a basic shape. The iconography of a floppy disk is not a basic shape.