I'm not really clear on why it's that much worse than having your phone number; with the right access, the phone number is all you need to track a user across global cell networks anyway.

Though the IMEI should be exposed to a Stingray type device, so you'd be able to track users without access to the cell networks, you'd just have to set up a bunch of radios.

And you wouldn't be able to shake it by swapping SIMs. That said, most users wouldn't bother with changing phone numbers or SIMs.

Tracking people (phones) across apps and services.