They can tell people not to impede access to driveways and doors but they can't physically remove them if they're on the public sidewalk.

I generally agree but they should have the right to tell someone not to do something (with threat of calling the cops) but no right to enforce it. For example, it would not be unreasonable to expect the security to ask someone to not urinate or defecate on the sidewalk in front of the entrance.