I guess there are some qualifications here. If there was a severely low probability of a cyclist there, and especially if cycling was forbidden there, then this does not apply in the same way, although you should always be aware that others might break rules, and you are responsible for minimzing risks even to those. But, you can not slow down for every possibly suividal maniac throwing themselves out in the road where you are not supposed to be.
I still lean towards the car not being cautious enough.