Raymond Chen writes a great blog called the OldNewThing that has some good stories. I think asking why it failed is rather hard to explain and I suspect different people who were on the project would have different explanations.

I worked on XP and a little bit of Longhorn. In my opinion one of the reasons was that much of the product direction and leadership shifted. The MSN app team merged with the shell team after XP shipped and JoeB, who was running the shell/UX team moved to a new team.

I wouldn't categorize Longhorn as a failure, but a step in a larger pattern of large release: 95, XP, Win7 and then, typically shorter releases in between that are managed differently: 98, ME, Vista.