Yeah but the average person wouldn't know to set up a firewall (and can't count on their ISP to have their best interests at heart.) Therefore the general public benefits from the degree of protection that NAT provides.
Obviously, those average people have a suitable firewall provided by default on their routers.