core0

Ah, thanks - require_ack_response sounds like what I was missing. Some blogs are from before this was added in 0.12 so I didn't know about it.

I am still interested in forwarder failure cases - I have replied to kiyoto's comment talking about the HA docs, which still talk about some other cases that can lose messages.

In this case:

* The process dies immediately after receiving the events, but before writing them into the buffer.

Is it possible to require acknowledgement that the log event has been written to the buffer? Is that separate to what require_ack_response does?

> Where does it say this? I don't think this was ever the case for Fluentd.

It's in http://docs.fluentd.org/articles/high-availability#forwarder..., which says:

However, possible message loss scenarios do exist:

The process [log forwarder’s fluentd from the paragraph above] dies immediately after receiving the events, but before writing them into the buffer.

Is this document out of date?

We'd love to use a ruby-based solution like this, but the docs say it will lose data whenever the receiving end crashes. Any plans to fix that?

The way it was described in the docs gave me the impression there is no acknowledgement of network writes - if that's true won't even clean shutdowns lose data sometimes?