Most new phones are trackable even if they are off, even.
For anyone wondering how this works: the cellular modem is a separate general-purpose computer that runs code from the manufacturer and the service provider, the only thing needed to allow tracking a phone that's off is circuitry to allow the modem to draw power independent of the rest of the phone.
I guess a faraday pouch might be helpful, but I recall reading these aren't really as effective as many people believe.
I guess the gasket let enough EM through?
Amusingly, crumpled aluminum foil seems to have a better track record.
Same principle applies in the visual spectrum.