Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[spanmetrics] Do not drop metric data points on attributes cache miss. #18725

Merged
merged 2 commits into from
Feb 20, 2023

Conversation

kovrus
Copy link
Member

@kovrus kovrus commented Feb 16, 2023

Description:

Components:

  • spanmetrics connector
  • spanmetrics processor

The metric key to attributes cache should improve performance but should result in data loss when on cache misses. This change addresses the issue with dropped data points.

At this point, the core logic of the components is the same, so it should be fine to fix this issue for both components in the same PR.

Testing:
Existing unit tests

@github-actions github-actions bot added the processor/spanmetrics Span Metrics processor label Feb 16, 2023
@runforesight
Copy link

runforesight bot commented Feb 16, 2023

Foresight Summary

    
Major Impacts

build-and-test duration(44 minutes 27 seconds) has decreased 22 minutes 16 seconds compared to main branch avg(1 hour 6 minutes 43 seconds).
View More Details

⭕  build-and-test-windows workflow has finished in 4 seconds (41 minutes 37 seconds less than main branch avg.) and finished at 18th Feb, 2023.


Job Failed Steps Tests
windows-unittest-matrix -     🔗  N/A See Details
windows-unittest -     🔗  N/A See Details

✅  check-links workflow has finished in 52 seconds (1 minute 39 seconds less than main branch avg.) and finished at 18th Feb, 2023.


Job Failed Steps Tests
changed files -     🔗  N/A See Details
check-links -     🔗  N/A See Details

✅  telemetrygen workflow has finished in 1 minute 1 second (2 minutes 3 seconds less than main branch avg.) and finished at 18th Feb, 2023.


Job Failed Steps Tests
build-dev -     🔗  N/A See Details
publish-latest -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

✅  changelog workflow has finished in 3 minutes 24 seconds (⚠️ 49 seconds more than main branch avg.) and finished at 18th Feb, 2023.


Job Failed Steps Tests
changelog -     🔗  N/A See Details

✅  prometheus-compliance-tests workflow has finished in 3 minutes 39 seconds (5 minutes 12 seconds less than main branch avg.) and finished at 18th Feb, 2023.


Job Failed Steps Tests
prometheus-compliance-tests -     🔗  ✅ 21  ❌ 0  ⏭ 0    🔗 See Details

✅  load-tests workflow has finished in 13 minutes 57 seconds (2 minutes 53 seconds less than main branch avg.) and finished at 18th Feb, 2023.


Job Failed Steps Tests
loadtest (TestIdleMode) -     🔗  ✅ 1  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestBallastMemory|TestLog10kDPS) -     🔗  ✅ 18  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceAttributesProcessor) -     🔗  ✅ 3  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetric10kDPS|TestMetricsFromFile) -     🔗  ✅ 6  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetricResourceProcessor|TestTrace10kSPS) -     🔗  ✅ 12  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceBallast1kSPSWithAttrs|TestTraceBallast1kSPSAddAttrs) -     🔗  ✅ 10  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceNoBackend10kSPS|TestTrace1kSPSWithAttrs) -     🔗  ✅ 8  ❌ 0  ⏭ 0    🔗 See Details
setup-environment -     🔗  N/A See Details

✅  build-and-test workflow has finished in 44 minutes 27 seconds (22 minutes 16 seconds less than main branch avg.) and finished at 18th Feb, 2023.


Job Failed Steps Tests
unittest-matrix (1.19, internal) -     🔗  ✅ 561  ❌ 0  ⏭ 0    🔗 See Details
correctness-metrics -     🔗  ✅ 2  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, extension) -     🔗  ✅ 537  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, processor) -     🔗  ✅ 1509  ❌ 0  ⏭ 0    🔗 See Details
correctness-traces -     🔗  ✅ 17  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, internal) -     🔗  ✅ 561  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, extension) -     🔗  ✅ 537  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, processor) -     🔗  ✅ 1509  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-0) -     🔗  ✅ 2575  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, exporter) -     🔗  ✅ 2455  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, receiver-0) -     🔗  ✅ 2575  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, exporter) -     🔗  ✅ 2455  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, other) -     🔗  ✅ 4745  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-1) -     🔗  ✅ 1928  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, receiver-1) -     🔗  ✅ 1928  ❌ 0  ⏭ 0    🔗 See Details
integration-tests -     🔗  ✅ 55  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, other) -     🔗  ✅ 4745  ❌ 0  ⏭ 0    🔗 See Details
setup-environment -     🔗  N/A See Details
check-collector-module-version -     🔗  N/A See Details
check-codeowners -     🔗  N/A See Details
lint-matrix (receiver-0) -     🔗  N/A See Details
lint-matrix (receiver-1) -     🔗  N/A See Details
lint-matrix (processor) -     🔗  N/A See Details
lint-matrix (exporter) -     🔗  N/A See Details
lint-matrix (extension) -     🔗  N/A See Details
lint-matrix (internal) -     🔗  N/A See Details
lint-matrix (other) -     🔗  N/A See Details
checks -     🔗  N/A See Details
build-examples -     🔗  N/A See Details
lint -     🔗  N/A See Details
unittest (1.20) -     🔗  N/A See Details
unittest (1.19) -     🔗  N/A See Details
cross-compile (darwin, amd64) -     🔗  N/A See Details
cross-compile (darwin, arm64) -     🔗  N/A See Details
cross-compile (linux, 386) -     🔗  N/A See Details
cross-compile (linux, amd64) -     🔗  N/A See Details
cross-compile (linux, arm) -     🔗  N/A See Details
cross-compile (linux, arm64) -     🔗  N/A See Details
cross-compile (linux, ppc64le) -     🔗  N/A See Details
cross-compile (windows, 386) -     🔗  N/A See Details
cross-compile (windows, amd64) -     🔗  N/A See Details
build-package (deb) -     🔗  N/A See Details
build-package (rpm) -     🔗  N/A See Details
windows-msi -     🔗  N/A See Details
publish-check -     🔗  N/A See Details
publish-dev -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

✅  e2e-tests workflow has finished in 11 minutes 10 seconds (4 minutes 35 seconds less than main branch avg.) and finished at 18th Feb, 2023.


Job Failed Steps Tests
kubernetes-test -     🔗  N/A See Details

🔎 See details on Foresight

*You can configure Foresight comments in your organization settings page.

@kovrus kovrus force-pushed the spanmetrics-caching branch 2 times, most recently from 445ca87 to 8923a3c Compare February 16, 2023 15:59
@kovrus kovrus force-pushed the spanmetrics-caching branch from 8923a3c to 626f775 Compare February 16, 2023 16:01
@kovrus kovrus marked this pull request as ready for review February 16, 2023 16:01
@kovrus kovrus requested review from a team and dashpole February 16, 2023 16:01
@kovrus
Copy link
Member Author

kovrus commented Feb 16, 2023

cc: owner - @albertteoh

Copy link
Contributor

@albertteoh albertteoh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏼

@kovrus kovrus force-pushed the spanmetrics-caching branch from 626f775 to 72ecb76 Compare February 17, 2023 13:43
@kovrus
Copy link
Member Author

kovrus commented Feb 17, 2023

cc: @Cluas

@kovrus kovrus requested a review from albertteoh February 17, 2023 13:46
@kovrus kovrus force-pushed the spanmetrics-caching branch from 65e1ae7 to 95e66bd Compare February 17, 2023 14:58
Copy link
Contributor

@Cluas Cluas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice job!

@kovrus kovrus force-pushed the spanmetrics-caching branch from 95e66bd to 994ccf7 Compare February 18, 2023 00:19
@kovrus kovrus requested review from djaglowski and removed request for albertteoh February 18, 2023 00:29
@djaglowski djaglowski merged commit 104bc96 into open-telemetry:main Feb 20, 2023
@kovrus kovrus deleted the spanmetrics-caching branch February 20, 2023 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants