But the thing I don't understand is why Intel gave up on XScale, their ARM-compatible effort
This isn't the first time Intel sold off a slice of itself. I remember AMD doing this many, many times.
For over 40 years, the form factor didn't matter. All that mattered was having the fastest and most performant CPU in the entire world. There was a real demand to crunch numbers and move bits since the 70's, so that's the world Intel, AMD, and everyone else knew.
To sell off a division that sold low-power chips for tiny margins in an industry that never, ever, ever explodes (until it did) was just plain normal. He would have gotten pressure and funny looks from the industry and shareholders if he didn't sell it off.