I wouldn't call UFO conspiracy theories harmless given the past occurrence of UFO suicide cults like Heaven's Gate.
So while most people don't believe this stuff, it's not exactly fringe either. That's maybe a hundred million people who believe aliens are coming to earth, while Heaven's Gate was 41 people. Using belief in UFOs as the basis for a religion has happened a few times, but it's very fringe.