Not with antimatter rockets, it's not. Even though antimatter is extremely energy dense, the rocket equation still applies. You need reaction mass.
If you use laser beams as your rocket exhaust, then you get maximal efficiency (exhaust velocity is C), but very low thrust. That's not going to get you anywhere in days or weeks.
If you use stored matter (hydrogen, say) as your rocket exhaust, then you'd get better thrust, but all sorts of other issues come into play. Radiating all that waste heat. Running out of reaction mass. Again, there's not a practical design that could get you anywhere near a nearby star in "days or weeks", even if you assume "perfect" materials, efficiency, etc.
> If you're on a ship with sufficient fuel, you can keep accelerating forever
Um, no.