Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Differentiate metrics for domainTagged metrics and task operation metrics #3467

Merged

Conversation

anish531213
Copy link
Contributor

@anish531213 anish531213 commented Aug 20, 2020

What changed?
Right now when metrics is emitted with domain tag, our current dashboard and alerts will look at the metrics (p99/mean) only for the one domain due to maxSeries. So, in order to solve this problem, I introduced emitting both domainTagged metrics and operation metrics separately with different metics name. This change is only made for new task processing.

Why?

How did you test it?
Haven't tested it yet.

Potential risks
No risk.

@anish531213 anish531213 requested review from meiliang86, yycptt and a team August 20, 2020 23:42
@anish531213 anish531213 force-pushed the aanish/emit-domain-metrics branch from 3995fdd to 5c4281e Compare August 20, 2020 23:44
Copy link
Contributor

@yycptt yycptt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please take a look at how metricRollupName works: https://github.com/uber/cadence/blob/master/common/metrics/defs.go#L2098

It can make code much cleaner as we only need to call IncCounter() or RecordTimer() once, instead of copy-pasting code.

common/metrics/defs.go Outdated Show resolved Hide resolved
service/history/task/task.go Outdated Show resolved Hide resolved
@anish531213 anish531213 requested a review from yycptt August 21, 2020 17:54
Copy link
Contributor

@yycptt yycptt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also update the metrics used in history/taskProcessor.go

common/metrics/defs.go Outdated Show resolved Hide resolved
service/history/task/task.go Show resolved Hide resolved
@anish531213 anish531213 requested a review from yycptt August 21, 2020 19:44
Copy link
Contributor

@yycptt yycptt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.1%) to 66.936% when pulling 383f6f6 on anish531213:aanish/emit-domain-metrics into 15eed39 on uber:master.

@anish531213 anish531213 merged commit fa57460 into cadence-workflow:master Aug 21, 2020
mkolodezny pushed a commit to mkolodezny/cadence that referenced this pull request Aug 24, 2020
)

* Differentiate metrics for domainTagged metrics and task operation metrics
anish531213 added a commit to anish531213/cadence that referenced this pull request Aug 25, 2020
)

* Differentiate metrics for domainTagged metrics and task operation metrics
yux0 pushed a commit to yux0/cadence that referenced this pull request May 4, 2021
)

* Differentiate metrics for domainTagged metrics and task operation metrics
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants