My problem, and Raganwald's, is that I can generally read the rules of any new game and, based on those rules, formulate a strategy. I might not expect to win, but I at least have enough understanding of the mechanism of the game that I can pick a direction and run with it. (I was going to say, "that I can play passably well", but that really isn't it at all -- it's the formulation of strategy that's important to me, and, I suspect, Raganwald.)
So what happens in Go? Well, you're told the rules, and then you're given a few hints that don't make any sense to you, like the 3-4 stone. Either way, it's impossible to form a strategy. You might as well stand above the board and let stones loose from your fingers to fall randomly upon it.
And my teachers, bless their hearts, took it upon themselves to win by extremely large margins repeatedly, which I would not have taken personally if only I had finished each game with the feeling that I had learned something.
In the end, I became an "eh" player, but found myself dreading Tuesday night Go club, found it occupying too much of my mind space, and found that it wasn't really giving me anything positive in return.
There were very very few long term new recruits in the local club.