To add to your comment…

I thought the use of paper tape was directly related to the teletypes (when used for I/O with a computer). Where the paper tape could operate as both an input and output mechanism, depending on if you wanted to input a program/text or save a program/text. Paper tape wasn’t a computer storage medium, as much as it was a teletype medium.

In this way, the paper tape hijacked the interactive I/O mode on these early computers to supplant it with a stored format.