I think that the problem is that we are looking at "willpower" as a general skill when we should see it as more specific to the task. I have built up quite a bit of resistance to certain desires like food and television, but I still flounder when it comes to other areas of my life.
I would be very interested in long-term studies that attempt to build willpower in just one area and then examine the individual's ability to apply that self-control to another, separate task.