From 013aff723f07bf2ff34a627529279ccbfb7ce1cd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20=C5=9Awi=C4=85tek?= Date: Fri, 22 Apr 2022 18:03:43 +0200 Subject: [PATCH] docs: describe metadata_attributes breaking change Co-authored-by: Dominik Rosiek <58699848+sumo-drosiek@users.noreply.github.com> --- CHANGELOG.md | 4 +-- docs/Upgrading.md | 64 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index adacbe24d9..cba5e58d1e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Breaking changes - chore: bump OT core to v0.49.0 [#550][#550] ([upgrade guide][upgrade-guide-log-collection]) +- fix!(sumologicexporter): send resource attributes as fields for non-otlp, removing metadata_attributes [#549][#549] ([upgrade-guide][upgrade-guide-metadata]) ### Changed @@ -22,6 +23,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 [Unreleased]: https://github.com/SumoLogic/sumologic-otel-collector/compare/v0.48.0-sumo-0...main [upgrade-guide-log-collection]: docs/Upgrading.md#several-changes-to-receivers-using-opentelemetry-log-collection +[upgrade-guide-metadata]: docs/Upgrading.md#sumo-logic-exporter-metadata-handling [#546]: https://github.com/SumoLogic/sumologic-otel-collector/pull/546 [#550]: https://github.com/SumoLogic/sumologic-otel-collector/pull/550 [#553]: https://github.com/SumoLogic/sumologic-otel-collector/pull/553 @@ -39,7 +41,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed - fix(sumologicexporter): treat resource attributes as fields for otlp #536 -- fix!(sumologicexporter): send resource attributes as fields for non-otlp, removing metadata_attributes #549 ### Other @@ -52,7 +53,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 [#530]: https://github.com/SumoLogic/sumologic-otel-collector/pull/530 [#534]: https://github.com/SumoLogic/sumologic-otel-collector/pull/534 [#536]: https://github.com/SumoLogic/sumologic-otel-collector/pull/536 -[#549]: https://github.com/SumoLogic/sumologic-otel-collector/pull/549 ## [v0.47.0-sumo-0] diff --git a/docs/Upgrading.md b/docs/Upgrading.md index b2416acc31..41719b6800 100644 --- a/docs/Upgrading.md +++ b/docs/Upgrading.md @@ -12,3 +12,67 @@ Please refer to the [official upgrade guide][opentelemetry-log-collection-upgrad [opentelemetry-log-collection]: https://github.com/open-telemetry/opentelemetry-log-collection [opentelemetry-log-collection-upgrade-guide]: https://github.com/open-telemetry/opentelemetry-log-collection/releases/tag/v0.29.0 + +### Sumo Logic exporter metadata handling + +The [OpenTelemetry data format][ot-data-format] makes a distinction between record-level attributes and +resource-level attributes. The `metadata_attributes` configuration option in the [`sumologicexporter`][sumologicexporter] +allowed setting metadata for records sent to the Sumo Logic backend based on both record and resource-level +attributes. Only attributes matching the supplied regular expressions were sent. + +However, this is conceptually incompatible with OpenTelemetry. Our intent with the exporter is to use OpenTelemetry +conventions as much as we can, to the point where it should eventually be possible to export data to Sumo using the +upstream OTLP exporter. This is why we are changing the behaviour. From now on: + +1. `metadata_attributes` no longer exists. +1. Metadata for sent records is based on resource-level attributes. + +In order to retain current behaviour, processors should be used to transform the data before it is exported. This +potentially involves two transformations: + +#### Removing unnecessary metadata using the [resourceprocessor][resourceprocessor] + +`metadata_attributes` allowed filtering based on regular expressions. An equivalent processor doesn't yet +exist, but resource-level attributed can be dropped using the [resourceprocessor][resourceprocessor]. For example: + +```yaml +processors: + resource: + attributes: + - pattern: ^k8s\.pod\..* + action: delete +``` + +will delete all attributes starting with `k8s.pod.`. + +**NOTE**: The ability to delete attributes based on a regular expression is currently unique to our fork of the +[resourceprocessor][resourceprocessor], and isn't available in upstream. + +#### Moving record-level attributes used for metadata to the resource level + +This can be done using the [Group by Attributes processor][groupbyattrsprocessor]. If you were using the Sumo Logic +exporter to export data with a `host` record-level attribute: + +```yaml +exporters: + sumologicexporter: + ... + metadata_attributes: + - host +``` + +You can achieve the same effect with the following processor configuration: + +```yaml +processors: + groupbyattrsprocessor: + keys: + - host +``` + +Keep in mind that your attribute may already be resource-level, in which case no changes are necessary. + +[ot-data-format]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/common/README.md +[groupbyattrsprocessor]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/groupbyattrsprocessor +[resourceprocessor]: https://github.com/SumoLogic/opentelemetry-collector-contrib/tree/2ae9e24dc7efd940e1aa2f6efb288504b591af9b/processor/resourceprocessor +[sumologicexporter]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.48.0-sumo-0/pkg/exporter/sumologicexporter