The pendulum rocket fallacy refers to the placement of the engine itself, not center of mass. Mass distribution absolutely matters, because with vectored thrust, your engine can exert a higher torque on the rocket if the center of mass is farther away. You can try it yourself with a long pole and weight (e.g. broom with big brush?). Balancing the pole with the weight on top is easier than with the weight on the bottom.