The only exception that we know of is our own Solar System. Where Jupiter acts like a vacuum cleaner to reduce material that might hit Earth. We knkw of no other system with both Earth-like planets and gas giants. Given how catastrophic the remaining asteroids have been for us (bye bye dinosaurs), this is unlikely to be a coincidence.
Unless we find a similar arrangement around a white dwarf, the time spent in the habitable zone isn't the only important factor for stability over evolutionary time frames.