It’s more like you have one Lambda instance that is always running (sure think of it like one server) that can serve the first page instantly while the other ten thousands Lambdas take half a second to warm up. That first Lambda responds instantly so it hides the half second response for the rest of the application.
It really makes sense if you have traffic that spikes at certain times per day (think like fantasy football). There’s going to be times when no one is using your application but when they do, there are going to be dozens or hundreds of people suddenly using it. So the first person that logs on hits the provisioned Lambda instantly and then AWS warms up the rest of the Lambdas as more traffic comes in.