They ditched the XServe because people just weren't buying them in any noticeable quantities. Apple sells a lot more Macs now then back when Macs were pretty all Apple made, so why would they decide to leave all that money on the table?
It's a chicken and the egg thing. People weren't buying them because they were not competitive with other rack servers anymore. By using your logic Apple could just not update the MacPro for a few years and then pull line because 'people weren't buying them.' Apple created the people not buying them situation though.
I'm actually a little surprised that they took this long to make the decision to kill off the XServe. Back during the G5 crash-and-burn they stopped publishing separate XServe sales data, merging it into a collective "G5" sales bin, presumably to hide how drastically and quickly G5 sales tailed off after the initial surge of sales, followed by a lot of disappointment when their customers realized that their performance wasn't anywhere near where Apple claimed that it was.