I have never heard of that connection. I was under the assumption anti gmo folk were against modified rice that is super drought resistant.
GMO plants are designed to take much higher doses. That is a concern both environmentally and as a potential health issue.
Another use is to engineer plants that produce insecticides themselves, which raises the concern if those insecticides affect other organisms as well.