Oddly, NYC is archipelago city, and the only way from Long Island/Brooklyn/Queens to New Jersey is via Manhattan, hence the congestion. I guess one could take the huge detour via Staten Island, but the Verrazzano already has a big toll.
If only the roads were planned bit better.