Pumped storage requires a nearby hill with a lake on top of it (otherwise earth and concrete has to be poured to form an artificial reservoir), and a nearby lower reservoir, and a nearby source of water. And if it isn't close to transportation infrastructure, building a facility becomes prohibitively expensive. As the optimal sites are developed, less and less suitable sites need to be developed.

Why is New York building battery storage at $567/KWh [1] if pumped storage is so cheap?

There a vast amount of hydro storage potential in the Himalayas, but good luck developing any of it.

1. https://www.utilitydive.com/news/new-york-battery-storage-co...