"No risk" seems a bit strong here. (Or maybe I'm just paranoid. But I was definitely in charge of the litter box when my wife was pregnant).
Could an indoor cat who has at some point shared a litter box with an outdoor cat have picked up spores (in the same way that a human might), for instance? Or, my cats were rescued as young kittens after their (stray) mother died: were they in the wild long enough to be exposed? (And how many cats adopted in the US have similar histories that their owners might not even have heard about?)