autossh + ssh -C + tail -F ?
kafka sounds like an enormous overkill. If you want to store the logs locally while tailing, just add in a tee.
kafka sounds like an enormous overkill. If you want to store the logs locally while tailing, just add in a tee.
But it my case, I have "30 minutes on, 5 minutes off, 90 minutes on, 90 minutes off, 2 minutes on" kind of situations, in which anything that doesn't track what was already transferred and what wasn't, will lose data. (zeromq's buffers also have limited capacity and/or are tied to a process on the other side - if it restarts, buffers are gone).