signalfx exporter: Add exclude_properties config option for limiting dimension updates #18464

rmfitzpatrick · 2023-02-08T18:10:40Z

Description:
Adding a feature - These changes add exclude_properties support to* the signalfx exporter to filter which properties/dimensions are used in dimension update content. This work is based on existing Smart Agent functionality: https://github.com/signalfx/signalfx-agent/blob/main/docs/config-schema.md#propertiestoexclude

Testing:
Added and updated existing tests.

Documentation:
Documented config field in readme

runforesight · 2023-02-08T18:11:54Z

Foresight Summary

Major Impacts

build-and-test-windows duration(4 seconds) has decreased 40 minutes 45 seconds compared to main branch avg(40 minutes 49 seconds).

View More Details

⭕ build-and-test-windows workflow has finished in 4 seconds (40 minutes 45 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

Job	Failed Steps	Tests
windows-unittest-matrix	- 🔗	N/A	See Details
windows-unittest	- 🔗	N/A	See Details

✅ telemetrygen workflow has finished in 1 minute 26 seconds (1 minute 32 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

Job	Failed Steps	Tests
build-dev	- 🔗	N/A	See Details
publish-latest	- 🔗	N/A	See Details
publish-stable	- 🔗	N/A	See Details

✅ changelog workflow has finished in 1 minute 51 seconds (1 minute 53 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

Job	Failed Steps	Tests
changelog	- 🔗	N/A	See Details

✅ check-links workflow has finished in 2 minutes 48 seconds and finished at 9th Feb, 2023.

Job	Failed Steps	Tests
changed files	- 🔗	N/A	See Details
check-links	- 🔗	N/A	See Details

✅ prometheus-compliance-tests workflow has finished in 3 minutes 37 seconds (5 minutes 7 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

Job	Failed Steps	Tests
prometheus-compliance-tests	- 🔗	✅ 21 ❌ 0 ⏭ 0 🔗	See Details

✅ load-tests workflow has finished in 9 minutes 10 seconds (7 minutes 33 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

Job	Failed Steps	Tests
loadtest (TestIdleMode)	- 🔗	✅ 1 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestTraceAttributesProcessor)	- 🔗	✅ 3 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestMetric10kDPS\|TestMetricsFromFile)	- 🔗	✅ 6 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestMetricResourceProcessor\|TestTrace10kSPS)	- 🔗	✅ 12 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestTraceBallast1kSPSWithAttrs\|TestTraceBallast1kSPSAddAttrs)	- 🔗	✅ 10 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestTraceNoBackend10kSPS\|TestTrace1kSPSWithAttrs)	- 🔗	✅ 8 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestBallastMemory\|TestLog10kDPS)	- 🔗	✅ 19 ❌ 0 ⏭ 0 🔗	See Details
setup-environment	- 🔗	N/A	See Details

✅ build-and-test workflow has finished in 43 minutes 27 seconds (14 minutes 56 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

Job	Failed Steps	Tests
unittest-matrix (1.19, internal)	- 🔗	✅ 561 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, internal)	- 🔗	✅ 561 ❌ 0 ⏭ 0 🔗	See Details
correctness-metrics	- 🔗	✅ 2 ❌ 0 ⏭ 0 🔗	See Details
correctness-traces	- 🔗	✅ 17 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, extension)	- 🔗	✅ 546 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, extension)	- 🔗	✅ 546 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, processor)	- 🔗	✅ 1472 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, processor)	- 🔗	✅ 1472 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, receiver-0)	- 🔗	✅ 2574 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, receiver-0)	- 🔗	✅ 2574 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, exporter)	- 🔗	✅ 2452 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, exporter)	- 🔗	✅ 2452 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, receiver-1)	- 🔗	✅ 1958 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, receiver-1)	- 🔗	✅ 1958 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, other)	- 🔗	✅ 4672 ❌ 0 ⏭ 0 🔗	See Details
integration-tests	- 🔗	✅ 55 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, other)	- 🔗	✅ 4672 ❌ 0 ⏭ 0 🔗	See Details
setup-environment	- 🔗	N/A	See Details
check-collector-module-version	- 🔗	N/A	See Details
check-codeowners	- 🔗	N/A	See Details
checks	- 🔗	N/A	See Details
build-examples	- 🔗	N/A	See Details
lint-matrix (receiver-0)	- 🔗	N/A	See Details
lint-matrix (receiver-1)	- 🔗	N/A	See Details
lint-matrix (processor)	- 🔗	N/A	See Details
lint-matrix (extension)	- 🔗	N/A	See Details
lint-matrix (exporter)	- 🔗	N/A	See Details
lint-matrix (internal)	- 🔗	N/A	See Details
lint-matrix (other)	- 🔗	N/A	See Details
lint	- 🔗	N/A	See Details
unittest (1.19)	- 🔗	N/A	See Details
unittest (1.18)	- 🔗	N/A	See Details
cross-compile (darwin, amd64)	- 🔗	N/A	See Details
cross-compile (darwin, arm64)	- 🔗	N/A	See Details
cross-compile (linux, 386)	- 🔗	N/A	See Details
cross-compile (linux, amd64)	- 🔗	N/A	See Details
cross-compile (linux, arm)	- 🔗	N/A	See Details
cross-compile (linux, arm64)	- 🔗	N/A	See Details
cross-compile (linux, ppc64le)	- 🔗	N/A	See Details
cross-compile (windows, 386)	- 🔗	N/A	See Details
cross-compile (windows, amd64)	- 🔗	N/A	See Details
build-package (deb)	- 🔗	N/A	See Details
build-package (rpm)	- 🔗	N/A	See Details
windows-msi	- 🔗	N/A	See Details
publish-check	- 🔗	N/A	See Details
publish-stable	- 🔗	N/A	See Details
publish-dev	- 🔗	N/A	See Details

✅ e2e-tests workflow has finished in 11 minutes 26 seconds (9 minutes 9 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

Job	Failed Steps	Tests
kubernetes-test	- 🔗	N/A	See Details

🔎 See details on Foresight

^{*You can configure Foresight comments in your organization settings page.}

exporter/signalfxexporter/exporter_test.go

dmitryax · 2023-02-08T20:17:14Z

exporter/signalfxexporter/internal/translation/dpfilters/propertyfilter.go

+)
+
+var anyREFilter = func() *StringFilter {
+	if sf, err := NewStringFilter([]string{"/^.*$/"}); err != nil {


I would expect NewStringFilter(nil) to have the same effect... Do you know why it's not the case by a chance? If it had such behavior, we wouldn't need the custom unmarshaller on PropertyFilter I beleive

I'm not sure I agree that I'd expect that to be the case since NewStringFilter(nil) to me suggests a new string filter that matches nothing. Added a unit test to confirm this is the existing behavior.

I just tried adding a nil pointer Matches() true default to make a custom PropertyFilter unmarshaller unneeded since it is a little wonky as is. I'm not super confident it won't change existing behavior though.

Sounds good. I also like this more than the PropertyFilter unmarshaller. The existing behavior should not be affected, otherwise, we would encounter nil pointer panics.

…dimension updates

dmitryax

LGTM

rmfitzpatrick requested a review from a team February 8, 2023 18:10

rmfitzpatrick requested a review from dmitryax as a code owner February 8, 2023 18:10

github-actions bot added the exporter/signalfx label Feb 8, 2023

github-actions bot requested a review from pmcollins February 8, 2023 18:12

github-actions bot assigned Aneurysm9 Feb 8, 2023

rmfitzpatrick force-pushed the signalfxexporterfilterproperties branch 5 times, most recently from 29eb9f9 to 39dea84 Compare February 8, 2023 20:09

dmitryax reviewed Feb 8, 2023

View reviewed changes

rmfitzpatrick force-pushed the signalfxexporterfilterproperties branch from 39dea84 to 8cac5c9 Compare February 8, 2023 23:21

signalfx exporter: Add exclude_properties config option for limiting …

01c46bc

…dimension updates

rmfitzpatrick force-pushed the signalfxexporterfilterproperties branch from 8cac5c9 to 01c46bc Compare February 9, 2023 15:28

set nil StringFilter.Matches() to true

5c08e78

dmitryax approved these changes Feb 9, 2023

View reviewed changes

dmitryax merged commit 0040467 into open-telemetry:main Feb 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

signalfx exporter: Add exclude_properties config option for limiting dimension updates #18464

signalfx exporter: Add exclude_properties config option for limiting dimension updates #18464

rmfitzpatrick commented Feb 8, 2023 •

edited

Loading

runforesight bot commented Feb 8, 2023 •

edited

Loading

`build-and-test-windows` duration(4 seconds) has decreased 40 minutes 45 seconds compared to main branch avg(40 minutes 49 seconds).

⭕ build-and-test-windows workflow has finished in 4 seconds (40 minutes 45 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ telemetrygen workflow has finished in 1 minute 26 seconds (1 minute 32 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ changelog workflow has finished in 1 minute 51 seconds (1 minute 53 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ check-links workflow has finished in 2 minutes 48 seconds and finished at 9th Feb, 2023.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 37 seconds (5 minutes 7 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ load-tests workflow has finished in 9 minutes 10 seconds (7 minutes 33 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ build-and-test workflow has finished in 43 minutes 27 seconds (14 minutes 56 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ e2e-tests workflow has finished in 11 minutes 26 seconds (9 minutes 9 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

dmitryax Feb 8, 2023 •

edited

Loading

rmfitzpatrick Feb 9, 2023

dmitryax Feb 9, 2023

dmitryax left a comment

signalfx exporter: Add exclude_properties config option for limiting dimension updates #18464

signalfx exporter: Add exclude_properties config option for limiting dimension updates #18464

Conversation

rmfitzpatrick commented Feb 8, 2023 • edited Loading

runforesight bot commented Feb 8, 2023 • edited Loading

Foresight Summary

build-and-test-windows duration(4 seconds) has decreased 40 minutes 45 seconds compared to main branch avg(40 minutes 49 seconds).

⭕ build-and-test-windows workflow has finished in 4 seconds (40 minutes 45 seconds less than main branch avg.) and finished at 9th Feb, 2023.

✅ telemetrygen workflow has finished in 1 minute 26 seconds (1 minute 32 seconds less than main branch avg.) and finished at 9th Feb, 2023.

✅ changelog workflow has finished in 1 minute 51 seconds (1 minute 53 seconds less than main branch avg.) and finished at 9th Feb, 2023.

✅ check-links workflow has finished in 2 minutes 48 seconds and finished at 9th Feb, 2023.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 37 seconds (5 minutes 7 seconds less than main branch avg.) and finished at 9th Feb, 2023.

✅ load-tests workflow has finished in 9 minutes 10 seconds (7 minutes 33 seconds less than main branch avg.) and finished at 9th Feb, 2023.

✅ build-and-test workflow has finished in 43 minutes 27 seconds (14 minutes 56 seconds less than main branch avg.) and finished at 9th Feb, 2023.

✅ e2e-tests workflow has finished in 11 minutes 26 seconds (9 minutes 9 seconds less than main branch avg.) and finished at 9th Feb, 2023.

dmitryax Feb 8, 2023 • edited Loading

Choose a reason for hiding this comment

rmfitzpatrick Feb 9, 2023

Choose a reason for hiding this comment

dmitryax Feb 9, 2023

Choose a reason for hiding this comment

dmitryax left a comment

Choose a reason for hiding this comment

rmfitzpatrick commented Feb 8, 2023 •

edited

Loading

runforesight bot commented Feb 8, 2023 •

edited

Loading

`build-and-test-windows` duration(4 seconds) has decreased 40 minutes 45 seconds compared to main branch avg(40 minutes 49 seconds).

⭕ build-and-test-windows workflow has finished in 4 seconds (40 minutes 45 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ telemetrygen workflow has finished in 1 minute 26 seconds (1 minute 32 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ changelog workflow has finished in 1 minute 51 seconds (1 minute 53 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 37 seconds (5 minutes 7 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ load-tests workflow has finished in 9 minutes 10 seconds (7 minutes 33 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ build-and-test workflow has finished in 43 minutes 27 seconds (14 minutes 56 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

✅ e2e-tests workflow has finished in 11 minutes 26 seconds (9 minutes 9 seconds less than `main` branch avg.) and finished at 9th Feb, 2023.

dmitryax Feb 8, 2023 •

edited

Loading