What infrastructure is that exactly?
Long Island City and Greenpoint/Williamsburg have already been absurdly built up by developers over the last two decades.
It's hard to believe an influx of new tens of thousands of individuals to fill vacancies at Amazon is going to produce new train tunnels under the East River or new bridges over it. NYC already has large tenants - Google, JP Morgan Chase, Goldman Sachs, Verizon etc, and the city's infrastructure woes have worsened despite their presence.
I would also imagine that much of the gain in the local tax coffers by the additional work force will be offset by whatever tax deals the city and states extends to Amazon.