Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add topic tag to producer metrics #6254

Conversation

taylanisikdemir
Copy link
Member

What changed?
Messaging producer (kafka) metrics were missing the topic tag which makes it impossible to make sense of count/latency metrics emitted by this component. In the past we only had visibility topics per environment so topic tag wasn't needed but with the introduction of Async APIs there can be multiple topics per environment.

Changes:

  • Adding optional custom tag support to messaging.NewMetricProducer
  • Pass topic tag option in the places calling this constructor
  • Add unit tests

Misc change:

  • Adding logs for async wf producer creation because producers are lazily initialized by Async API handlers and subject to 5m ttl. These logs will help us debug how often we re-create the producers for the same target topic.

How did you test it?
Unit tests

Copy link

codecov bot commented Aug 26, 2024

Codecov Report

Attention: Patch coverage is 85.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 73.00%. Comparing base (fbae51a) to head (1c18eee).
Report is 1 commits behind head on master.

Files Patch % Lines
common/messaging/kafka/client_impl.go 0.00% 2 Missing ⚠️
common/messaging/metrics_producer.go 92.85% 1 Missing ⚠️
Additional details and impacted files
Files Coverage Δ
common/asyncworkflow/queue/kafka/queue.go 82.60% <100.00%> (+1.21%) ⬆️
common/messaging/metrics_producer.go 81.81% <92.85%> (ø)
common/messaging/kafka/client_impl.go 18.29% <0.00%> (-0.23%) ⬇️

... and 7 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fbae51a...1c18eee. Read the comment docs.

metricsProducer struct {
producer Producer
metricsClient metrics.Client
type MetricsProducer struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we export this type?

@taylanisikdemir taylanisikdemir merged commit ac349fc into cadence-workflow:master Aug 26, 2024
20 checks passed
@taylanisikdemir taylanisikdemir deleted the taylan/kafka_producer_metric_topic_tag_support branch August 26, 2024 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants