Ion thrusters basically are low energy particle accelerators already. The problem, aside from energy consumption, is scaling up the thrust. You can't just cram twice as many particles through your particle accelerator without scaling up the mass of the thruster itself, or decreasing the exit velocity. And it does you very little good if your rocket has the fuel to accelerate to .1 c, if it takes you 10,000 years to do so.