This is a clever idea, but likely to end badly in practice given the way genetics works. Any neutral trait that doesn't confer increased survivability will be filtered out rapidly and never spread. So you would need to design something that gives the bacteria carrying the trait a slight survival advantage, but at the same time has a hidden "back door" we can exploit.
Furthermore, the back door has to be intimately tied to increased survival and not just along for the ride. Otherwise, the bacteria will snip the back door part out and keep the part that gives them increased survival. Now you have genetically engineered super-bugs. Oops. Even assuming that you have perfectly designed this new trait to meet the needs of having a back door and increased survivability that are inseparable, you still fail. Eventually, the bacteria all drop your fancy new trait and revert to being plain old bacterium because that state is now the one with the survival advantage that allows them to duck your new fancy antibiotics.
Probably, the only angle that is likely to work is immune system modification and enhancement. Something like a vaccine that causes the immune system to go thermonuclear on the pathogen is likely the best route. The immune system is highly adaptable and extremely well-honed. We've successfully driven many viruses to the brink of extinction this way, but the tech we have is not as good for bacteria (though there are vaccines for some bacterial infections).