So what did I learn? He was a jerk anyway, and it wasn't worth working for him. Fortunately, the economic climate was more forgiving 4 years ago when that happened, and after working in landscaping for 2 weeks, I got a great job that I loved.
If you're really that concerned, go ahead and look elsewhere. If you get some pretty solid connections, go ahead and present your concern to your manager. Depending on his reaction, I don't think you need to feel like you're letting the company down by leaving. If they can't pay you when they said, they at least owe you a very good explanation.