Logging stops when bufferFileSizeLimitBytes is reached and there is no way to change it to {Hour} logs

[corneliutusnea]

So,

Sending logs to the SeqServer stops when the rolling buffer file reaches the value defined for bufferFileSizeLimitBytes parameter. So by default the logging will stop (hard stop with no way to recover) at 1Gb data / day send through the serilog sink.

The RollingFileSink seems to accept different file patterns (https://github.com/serilog/serilog-sinks-rollingfile/blob/dev/src/Serilog.Sinks.RollingFile/Sinks/RollingFile/Specifier.cs#L22) like Hour, HalfHour or Day but the SeriLog sink automatically configures the file to use -{Date} so the file size limit is set to the date (https://github.com/serilog/serilog-sinks-seq/blob/dev/src/Serilog.Sinks.Seq/Sinks/Seq/DurableSeqSink.cs#L58).

If I try to set the filename to "seqbuffer-{Hour}" for the RollingFileSync crashes because now I have two tokens {Hour} and {Date}.

1. Is there a way (outside or configuring the file size to 5-10Gb to roll the files while sending the data to Seq?
2. Can the -{Date} in the DurrableSeqSink maybe be removed or only appended if there is no other token?

[nblumhardt]

Hi Corneliu, thanks for the note.

The only current option is to increase the buffer file size limit. Since the files aren't designed to be transported over the network, would there be a down-side to this? If the files rolled hourly, you'd still end up with the same disk utilization.

[corneliutusnea]

@nblumhardt I'm not worried about the disk size. I'm worried that because they don't roll, even setting a limit of 2-3Gb can mean that at one point I might have an out-of-average busy-day when I'm at risk for the logs to simply stop with no way to recover.

[nblumhardt]

You're right; I can see how finer-grained rolling files would help minimise disk usage while keeping things moving, by allowing old buffer files to be cleared out faster. I'll have a look at what this would take as soon as I have a chance 👍

[nblumhardt]

Thinking about this some more, there really shouldn't be any reason to use {Date} as the default - {Hour} or even {HalfHour} would function the same way, but with less overhead. Would switching the whole thing over to {Hour} (with some tweaked limits) do the job?

We'd need to make sure it will gracefully roll over for existing implementations, and adjust some limits, but seems okay in theory.

The only real "gotcha" would be the change in semantics where a file size limit is already being specified...

[nblumhardt]

@corneliutusnea I think we've finally got a good solution to this in #92 (WIP). If it's still an issue for you, thoughts/feedback on the PR would be great. Thanks again for the initial nudge on this, sorry it's taken a while.