Feature / fluentD : Support for sub second precision #2538
Conversation
| logDriverFluentdAddress = "fluentd-address" | ||
| dataLogDriverPath = "/data/firelens/" | ||
| logDriverAsyncConnect = "fluentd-async-connect" | ||
| logDriverSubSecondPrecision = "fluentd-sub-second-precision" |
There was a problem hiding this comment.
formatting is incorrect here, this would probably fail our static check
|
Is this something that's transparent to customers? |
|
@yunhee-l Its a non-breaking change, but it is not a hidden change. Its very small though, and I am very confident that if customers notice it, they will be happy (we have some strong requests for this). Many may not notice it at all. Right now, Docker adds timestamps to all logs. If you use the json log driver, like they do in k8s, then you get a timestamp with nanosecond precision. For some reason, when the Fluentd Docker log driver was first implemented, it truncated the timestamps so that they only have seconds. With this change, that truncation will no longer take place, and the full timestamp will be sent to FireLens with full precision. Below are "before" and "after" screenshots, courtesy of @MeghnaPrabhu.
Basically, right now, you get all zeroes for everything after the decimal (just seconds). After this change, you get the real timestamp with full precision. |
|
@PettitWesley thanks for the info, wanted to know whether we need to update our state file version with this change or not (which would be the case if this is something customers have to manually opt it/out). @MeghnaPrabhu please squash all commits into 1. |
MeghnaPrabhu
force-pushed
the
firelens_subsecondprecision
branch
from
f11bd5c
to
7dbade2
Compare
| @@ -1102,6 +1103,7 @@ func getFirelensLogConfig(task *apitask.Task, container *apicontainer.Container, | |||
| logConfig.Config[logDriverTag] = tag | |||
| logConfig.Config[logDriverFluentdAddress] = fluentd | |||
| logConfig.Config[logDriverAsyncConnect] = strconv.FormatBool(true) | |||
| logConfig.Config[logDriverSubSecondPrecision] = strconv.FormatBool(true) | |||
There was a problem hiding this comment.
can you update unit test to include a check for the new option? should be in https://github.com/aws/amazon-ecs-agent/blob/master/agent/engine/docker_task_engine_test.go#L2910-L2915
Update: Support for sub second precision in FluentD Docker Log Driver.
MeghnaPrabhu
force-pushed
the
firelens_subsecondprecision
branch
from
7dbade2
to
9967952
Compare
Update: Support for sub second precision in FluentD Docker Log Driver.
Summary
Adding an option in fluentD Docker Log Driver configuration to enable sub second precision.
This is being enabled to resolve the following request aws/containers-roadmap#839
Implementation
Testing
Manually tested this feature.
Description for the changelog
Feature FluentD: Support for sub second precision.
Licensing
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.