While I do agree that a "super server" listening to the network could be a new (or old, with (x)inetd) concern, a well-written and well-designed pid1 doing this isn't much more risky than having the services manage themselves. The listener could/should be just as unprivileged as the target daemon, nominally using the same uid/groups and root directory as the target as well. And indeed, systemd's .socket files support the same environment control variables (User=, Group=, SupplementaryGroups=, RootDirectory=, WorkingDirectory=, et al) that services do.
So that boils down to "are people writing socket activation units correctly", which is probably "no", but could be "yes".