Exactly. It does not matter how clever your structure is if the IRS thinks it is fishy you are going to be in for a world of pain.
They also make it more of a pain to deal with financial institutions since the institution needs to ensure you are not a US citizen.