What makes NAT the "main factor" for that.

I'm pretty sure every consumer router since at least 2002, if not further back, has had some stateful firewall implementation with default deny rules.