How many retries fluent-bit will try to send to output plugin in case of any failure?

[chillaxd]

When I am trying to test my output plugin in failure scenario I saw fluent-bit is giving me this after 3 retry:

[debug] [task] task_id=0 reached retry-attemps limit 2/1
[ warn] [engine] Task cannot be retried: task_id=0 thread_id=2 output=<my_output_plugin>.0
[debug] [task] destroy task=0x21b4540 (task_id=0)
[debug] [dyntag tail.0] 0x21b44b0 destroy (tag=tail.0)

And after this when I have removed the constraint (which triggers to fail my output plugin), I could not see any flushing done by fluent-bit.

If you ( @edsiper ) please elaborate the failure mechanism of fluent-bit, it will be really helpful for me.

[edsiper]

hi @chillaxd

Every output plugin have an inherit option called Retry_Limit, basically it specify the number of times that the engine can retry to flush a chunk of data in case of failure, by default is always set to 1.

If you want to retry until it succeed you can add the following configuration in the output section, e.g:

[OUTPUT]
    Name es
    Retry_Limit False

Setting the value to false will make it try indefinitely. Note that this can make your memory usage (RAM) go up because the data cannot be flushed, if your sources are log files consider to enabling the Mem_Buf_Limit option.

Let me know how that goes.

[chillaxd]

Hi @edsiper

Thanks for this information. It will help me to design my output plugin with great ease. I will surely update you on this part. Thank you again.

[edsiper]

@chillaxd are you building an output plugin ?

[chillaxd]

@edsiper yes I am building an output plugin

[edsiper]

Since the question is answered I am closing this issue. If you have more question or face any technical issue don't hesitate to ask again.

Fixed.