Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Input/output byte metrics for HTTP destination and non-blocking sources/destinations #4440

Merged
merged 10 commits into from
Apr 26, 2023

Conversation

MrAnno
Copy link
Collaborator

@MrAnno MrAnno commented Apr 24, 2023

This PR introduces byte-based input/output metrics for all non-blocking sources and destinations and for the HTTP destination.

On 32-bit architectures, metrics use SCU_KIB and cache smaller values in order to represent values larger than 4 GiB.

These metrics show the serialized message sizes (protocol-specific header/framing length is not included).

The incoming message length is also available as a hard macro: $RAWMSG_SIZE.

@kira-syslogng
Copy link
Contributor

Build FAILURE

MrAnno added a commit to MrAnno/syslog-ng that referenced this pull request Apr 24, 2023
Signed-off-by: László Várady <[email protected]>
@MrAnno
Copy link
Collaborator Author

MrAnno commented Apr 24, 2023

@kira-syslogng do stresstest

@kira-syslogng
Copy link
Contributor

Kira-stress-test: Build SUCCESS

MrAnno added a commit to MrAnno/syslog-ng that referenced this pull request Apr 24, 2023
Signed-off-by: László Várady <[email protected]>
@MrAnno MrAnno marked this pull request as ready for review April 24, 2023 11:45
@kira-syslogng
Copy link
Contributor

Build FAILURE

MrAnno added a commit to MrAnno/syslog-ng that referenced this pull request Apr 24, 2023
Signed-off-by: László Várady <[email protected]>
@MrAnno
Copy link
Collaborator Author

MrAnno commented Apr 24, 2023

@kira-syslogng do stresstest

@kira-syslogng
Copy link
Contributor

Kira-stress-test: Build SUCCESS

Copy link
Collaborator

@alltilla alltilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool feature!

lib/logsource.c Show resolved Hide resolved
lib/logmsg/logmsg.h Show resolved Hide resolved
lib/logsource.c Outdated Show resolved Hide resolved
MrAnno added a commit to MrAnno/syslog-ng that referenced this pull request Apr 25, 2023
Signed-off-by: László Várady <[email protected]>
MrAnno added a commit to MrAnno/syslog-ng that referenced this pull request Apr 25, 2023
Signed-off-by: László Várady <[email protected]>
MrAnno added a commit to MrAnno/syslog-ng that referenced this pull request Apr 25, 2023
Signed-off-by: László Várady <[email protected]>
@MrAnno
Copy link
Collaborator Author

MrAnno commented Apr 25, 2023

@kira-syslogng do stresstest

@kira-syslogng
Copy link
Contributor

Build FAILURE

MrAnno added 2 commits April 25, 2023 20:28
StatsByteCounter allows representing "big" accumulating byte-based metrics
even on 32-bit architectures, where a simple stats-counter would overflow
after 4 GiB.

When 64-bit counters are available, they will be used.
If not, the counters lose precision but are able to represent larger
values.

The current implementation is not thread-safe, reimplement it using a CAS
loop when needed.

Signed-off-by: László Várady <[email protected]>
Signed-off-by: László Várady <[email protected]>
MrAnno added a commit to MrAnno/syslog-ng that referenced this pull request Apr 25, 2023
Signed-off-by: László Várady <[email protected]>
@MrAnno
Copy link
Collaborator Author

MrAnno commented Apr 25, 2023

@kira-syslogng do stresstest

@kira-syslogng
Copy link
Contributor

Build FAILURE

@kira-syslogng
Copy link
Contributor

Kira-stress-test: Build SUCCESS

Note: this metric has been added to the destination worker instead of
the driver, because the API is currently not thread-safe.

Signed-off-by: László Várady <[email protected]>
@alltilla alltilla merged commit d292ca0 into syslog-ng:master Apr 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants