I have heard Brooklyn was cheap. I am looking around that area for the fall.
Also, if you're out in NYC on a weekend and want a cab home to somewhere like brooklyn, it'll only cost you like 15 bucks. However, if you want one to Hoboken, it'll cost you around 50-60... Really a pain when the PATH only comes every half hour after midnight.
As long as you can walk to the PATH, the loser factor on weekends (Jersey's finest comes to party) is the only real downside.