Isn't a stack just a "sequence of terminals and/or non-terminals"? (Or as you suggested, an infinite state machine where the states are all possible sequences of terminals and/or non-terminals.)
I suppose you could represent a PDA as an infinite state machine but the "stack" abstraction is kept around because it's clearer/easier to reason about. See, for example, the proof that CFLs are accepted by NPDAs and its correspondence with LR0 parsing.