If the door closed at flight time then the plane wouldn't be leaving at flight time, it would be leaving at flight time + the time it takes to go from boarding to getting in the air. Flight time is the time the plane is scheduled to take off, which means the flight time would diverge from the boarding time, and then you'd once again be asking why the boarding time is different from the flight time and why they don't let you board until the flight time, rinse and repeat.
They can still go early if all of the passengers have boarded.
I got to Dublin airport once at the time that the gates were supposed to close and they said they would wait for us if we were fast... Got to the concourse and it was the furthest gate possible - the sign said to allow a 15 minute walk! Had to run all the way but we made it.
And there's a time in the US as well, that's called the boarding time. Those there at the boarding time get on the plane, those aren't there at the boarding time missed their flight. Banking on getting on 5-10 minutes after boarding time is gambling that they're running behind schedule.