You might be allergic, I would get that tested, but it's nothing to worry about, just nice to know. AFAIK most people shouldn't react badly to prolonged skin contact with aluminum in the range that is holding a phone.
Have you travelled to Europe where there's a lot more usage of coins (eg euros/franc)? Do you just have to wear gloves?