So I guess it was Open in name only?
The term "open source" didn't come about until much later
I spent a few days peering at an old 'fiche reader figuring out how VMS stored user-defined account text in the process control block. We were doing chargeback you see...
The fun bit was the explanation of how a VAX booted VMS. The Cheshire cat was the mascot: the CPU was set up to have real and virtual addresses line up, then the VM bit was turned on, and...presto, no cat.