This is my main gripe with Prometheus (didn't use it yet).
For backups, you ideally want the backup server to do a pull from the backed-up servers, to avoid that a security incident in one of them could damage your backups.
But for monitoring, I feel that push is the way to go. I want only the minimal, indispensable connections into my production servers.
I like this approach because, for security data, you want it off of the box as soon as possible. In an ideal world a log would go straight from the kernel to another box, or as close to that as possible (to avoid tampering/ DOS'ing). So for your security data you're already going to want to push that latency metric down, and now the question is what to do with the rest of them - obviously your service logs are less sensitive, but at the same time getting things shipped off of a box can save you a lot of headache.
This is hand wavy though, I'm honestly very curious to hear what others think as I haven't built the system I'm describing.
In response to your vector solution:. What I found highly adaptable is having all servers forward to a central vector/fluent-bit agent, have that agent forward to kinesis firehose, then attach a transform lambda to create the final output. You end up with 2 configuration points from which you can control and transform all your logs. Instead of having to push this down to the servers through config management.