I'm having the opposite problem. Sometimes even my replies to someone with a Gmail address go to their SPAM box. What kind of a filter decides you don't want to see a message from some you messaged first?

FWIW I have my own domain and switched to Google as backend long ago, and yet I still occasionally have this problem.