You are correct. We're used to thinking of "digital" as implying "electronic", but that's only because our world contains huge numbers of digital electronic systems.
The opposite exists as well, "electronic" is conflated with "digital" quite often when it does not mean that.
> holes are either punched or not
This describes a binary encoding, not a digital process. Otherwise any powerbutton is 'digital', because it's on or off.
You are also correct: power buttons are digital. Contrast to a dimmer switch that can be set to any arbitrary point in a range.