Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sumologicexporter): remove translating attributes #672

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- feat(filterprocessor): drop custom changes ([upgrade guide][upgrade_guide_v0_55_0_expr_support]) [#709] [#714]
- feat(sumologicexporter): remove translating telegraf metric names ([upgrade guide][upgrade_guide_unreleased_moved_telegraf_translation]) [#678]
- feat(sumologicexporter): remove translating attributes ([upgrade guide][upgrade_guide_unreleased_moved_translation]) [#672]

[Unreleased]: https://github.com/SumoLogic/sumologic-otel-collector/compare/v0.57.2-sumo-0...main
[#672]: https://github.com/SumoLogic/sumologic-otel-collector/pull/672
[#678]: https://github.com/SumoLogic/sumologic-otel-collector/pull/678
[#709]: https://github.com/SumoLogic/sumologic-otel-collector/pull/709
[#714]: https://github.com/SumoLogic/sumologic-otel-collector/pull/714
Expand Down
53 changes: 1 addition & 52 deletions pkg/exporter/sumologicexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,13 +86,6 @@ exporters:
# default = false
flatten_body: {true, false}

# DEPRECATED
# translate_attributes specifies whether attributes should be translated
# from OpenTelemetry to Sumo Logic conventions;
# see "Attribute translation" documentation chapter from this document,
# default = true
translate_attributes: {true, false}

# instructs sumologicexporter to use an edpoint automatically generated by
# sumologicextension;
# to use direct endpoint, set it `auth` to `null` and set the endpoint configuration
Expand Down Expand Up @@ -138,48 +131,6 @@ exporters:

[sumologicextension]: ./../../extension/sumologicextension

## Attribute translation

**Note**: This functionality has been moved to the [sumologicschemaprocessor](../../processor/sumologicschemaprocessor/) and is now deprecated.
Please check the [upgrade guide](../../../docs/Upgrading.md#sumologic-exporter-drop-support-for-translating-attributes) for migrating instructions.

Attribute translation changes some of the attribute keys from OpenTelemetry convention to Sumo Logic convention.
For example, OpenTelemetry convention for the attribute containing Kubernetes pod name is `k8s.pod.name`,
but Sumo Logic expects it to be in attribute named `pod`.

If attribute with target name eg. `pod` already exists,
translation is not being done for corresponding attribute (`k8s.pod.name` in this example).

This feature is turned on by default.
To turn it off, set the `translate_attributes` configuration option to `false`.
Note that this may cause some of Sumo Logic apps, built-in dashboards to not work correctly.

Below is a list of all attribute keys that are being translated.

| OTC key name | Sumo Logic key name |
|---------------------------|---------------------|
| `cloud.account.id` | `AccountId` |
| `cloud.availability_zone` | `AvailabilityZone` |
| `cloud.platform` | `aws_service` |
| `cloud.region` | `Region` |
| `host.id` | `InstanceId` |
| `host.name` | `host` |
| `host.type` | `InstanceType` |
| `k8s.cluster.name` | `Cluster` |
| `k8s.container.name` | `container` |
| `k8s.daemonset.name` | `daemonset` |
| `k8s.deployment.name` | `deployment` |
| `k8s.namespace.name` | `namespace` |
| `k8s.node.name` | `node` |
| `k8s.service.name` | `service` |
| `k8s.pod.hostname` | `host` |
| `k8s.pod.name` | `pod` |
| `k8s.pod.uid` | `pod_id` |
| `k8s.replicaset.name` | `replicaset` |
| `k8s.statefulset.name` | `statefulset` |
| `service.name` | `service` |
| `log.file.path_resolved` | `_sourceName` |

## Source Templates

You can specify a template with an attribute for `source_category`, `source_name`,
Expand All @@ -188,9 +139,7 @@ can be used this way.

For example, when there is an attribute `my_attr`: `my_value`, `metrics/%{my_attr}`
would be expanded to `metrics/my_value`.
Use OpenTelemetry attribute names like `k8s.pod.name` instead of `pod`,
even when [attribute translation](#attribute-translation)
is turned on.
Use OpenTelemetry attribute names like `k8s.pod.name` instead of `pod`.

If an attribute is not found, it is replaced with `undefined`.
For example, `%{existing_attr}/%{nonexistent_attr}` becomes `value-of-existing-attr/undefined`.
Expand Down
15 changes: 7 additions & 8 deletions pkg/exporter/sumologicexporter/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,9 @@ type Config struct {
// The format of traces you will be sending, currently only otlp format is supported
TraceFormat TraceFormatType `mapstructure:"trace_format"`

// DEPRECATED
// Specifies whether attributes should be translated
// from OpenTelemetry standard to Sumo conventions (for example `cloud.account.id` => `accountId`
// `k8s.pod.name` => `pod` etc).
TranslateAttributes bool `mapstructure:"translate_attributes"`

// DEPRECATED: The below attributes only exist so we can print a nicer error
// message about not supporting them anymore.
TranslateAttributes bool `mapstructure:"translate_attributes"`
TranslateTelegrafMetrics bool `mapstructure:"translate_telegraf_attributes"`
MetadataAttributes []string `mapstructure:"metadata_attributes"`

Expand Down Expand Up @@ -139,6 +134,12 @@ Please consult the changelog at https://github.com/SumoLogic/sumologic-otel-coll

if cfg.TranslateTelegrafMetrics {
return fmt.Errorf(`*Deprecation warning*: translate_telegraf_attributes is not supported anymore.
Please consult the changelog at https://github.com/SumoLogic/sumologic-otel-collector/releases/tag/v0.59.0-sumo-0`,
)
}

if cfg.TranslateAttributes {
return fmt.Errorf(`*Deprecation warning*: translate_attributes is not supported anymore.
Please consult the changelog at https://github.com/SumoLogic/sumologic-otel-collector/releases/tag/v0.59.0-sumo-0`,
)
}
Expand Down Expand Up @@ -266,8 +267,6 @@ const (
DefaultSourceHost string = ""
// DefaultClient defines default Client
DefaultClient string = "otelcol"
// DefaultTranslateAttributes defines default TranslateAttributes
DefaultTranslateAttributes bool = true
// DefaultClearTimestamp defines default ClearLogsTimestamp value
DefaultClearLogsTimestamp bool = true
// DefaultLogKey defines default LogKey value
Expand Down
25 changes: 0 additions & 25 deletions pkg/exporter/sumologicexporter/exporter.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,6 @@ const (
tracesDataUrl = "/api/v1/collector/traces"
)

const translationDeprecationBanner = `
***********************************************************************************************************************************************************
*** Translating attributes is deprecated and is going to be dropped soon. Please see the migration document: ***
*** https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/Upgrading.md#sumologic-exporter-drop-support-for-translating-attributes. ***
***********************************************************************************************************************************************************
`

const sourceTemplatesDeprecationBanner = `
***********************************************************************************************************************************************************
*** Adding source headers is deprecated and is going to be dropped soon. Please see the migration document: ***
Expand Down Expand Up @@ -77,14 +70,6 @@ type sumologicexporter struct {
}

func initExporter(cfg *Config, createSettings component.ExporterCreateSettings) (*sumologicexporter, error) {
if cfg.TranslateAttributes {
createSettings.Logger.Warn(translationDeprecationBanner)

cfg.SourceCategory = translateConfigValue(cfg.SourceCategory)
cfg.SourceHost = translateConfigValue(cfg.SourceHost)
cfg.SourceName = translateConfigValue(cfg.SourceName)
}

if cfg.SourceCategory != "" || cfg.SourceHost != "" || cfg.SourceName != "" {
createSettings.Logger.Warn(sourceTemplatesDeprecationBanner)
}
Expand Down Expand Up @@ -243,9 +228,6 @@ func (se *sumologicexporter) pushLogsData(ctx context.Context, ld plog.Logs) err

se.dropRoutingAttribute(rl.Resource().Attributes())
currentMetadata := newFields(rl.Resource().Attributes())
if se.config.TranslateAttributes {
currentMetadata.translateAttributes()
}

if droppedRecords, err := sdr.sendNonOTLPLogs(ctx, rl, currentMetadata); err != nil {
dropped = append(dropped, droppedResourceRecords{
Expand Down Expand Up @@ -311,13 +293,6 @@ func (se *sumologicexporter) pushMetricsData(ctx context.Context, md pmetric.Met
rm := rms.At(i)

se.dropRoutingAttribute(rm.Resource().Attributes())

// TODO: Move these modifications to the Sumo schema processor
// we shouldn't modify data in an exporter, but these modifications are idempotent and therefore harmless
if se.config.TranslateAttributes {
translateAttributes(rm.Resource().Attributes()).
CopyTo(rm.Resource().Attributes())
}
}

var droppedMetrics pmetric.Metrics
Expand Down
Loading