It solves the problem of not letting random strangers into your house.
Even if the portal authenticates everyone... They'll still be one hack away from doing whatever they can to cause something to break for giggles
You sometimes see the idea that if a security feature can potentially be defeated, that is equivalent to that feature not existing.
That's true for a small class of problems, but in general it just shows an inability to see nuance.