Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sumologicexporter)!: attribute translation: change file.path.resolved to log.file.path_resolved #579

Merged
merged 1 commit into from
May 19, 2022
Merged

feat(sumologicexporter)!: attribute translation: change file.path.resolved to log.file.path_resolved #579

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Breaking changes

- fix(k8sprocessor)!: deprecate clusterName setting [#578]
- feat(sumologicexporter)!: attribute translation: change `file.path.resolved` to `log.file.path_resolved` [#579]

### Added

Expand All @@ -23,6 +24,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
[#576]: https://github.com/SumoLogic/sumologic-otel-collector/pull/576
[#575]: https://github.com/SumoLogic/sumologic-otel-collector/pull/575
[#578]: https://github.com/SumoLogic/sumologic-otel-collector/pull/578
[#579]: https://github.com/SumoLogic/sumologic-otel-collector/pull/579

## [v0.50.0-sumo-0]

Expand Down
8 changes: 4 additions & 4 deletions docs/Configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -386,11 +386,11 @@ receivers:
include_file_path_resolved: true
```

The `include_file_name: false` prevents the receiver from adding `file.name` attribute to the logs.
The `include_file_name: false` prevents the receiver from adding `log.file.name` attribute to the logs.
Instead, we are using `include_file_path_resolved: true`,
which adds a `file.path.resolved` attribute to the logs
which adds a `log.file.path_resolved` attribute to the logs
that contains the whole path of the file, as opposed to just the name of the file.
What's more, the `file.path.resolved` attribute is automatically recognized by the `sumologicexporter`
What's more, the `log.file.path_resolved` attribute is automatically recognized by the `sumologicexporter`
and translated to `_sourceName` attribute in Sumo Logic.

### Keeping track of position in files
Expand Down Expand Up @@ -462,7 +462,7 @@ Body: {
-> key: STRING(val)
}
Attributes:
-> file.name: STRING(example.log)
-> log.file.name: STRING(example.log)
Trace ID:
Span ID:
Flags: 0
Expand Down
24 changes: 12 additions & 12 deletions otelcolbuilder/cmd/testdata/filelog_reallife_complicated_sumologicexporter.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,8 @@ receivers:
to: body.log
- type: regex_parser
regex: (?P<timestamp>\d{4}-\d{2}-\d{2}-\d{2}\d{2}-\d{2})\d+
parse_from: attributes['file.path.resolved']
preserve_to: attributes['file.path.resolved']
parse_from: attributes['log.file.path_resolved']
preserve_to: attributes['log.file.path_resolved']
timestamp:
parse_from: body.timestamp
layout_type: gotime
Expand Down Expand Up @@ -123,8 +123,8 @@ receivers:
to: body.log
- type: regex_parser
regex: (?P<timestamp>\d{4}-\d{2}-\d{2}-\d{2}\d{2}-\d{2})\d+
parse_from: attributes['file.path.resolved']
preserve_to: attributes['file.path.resolved']
parse_from: attributes['log.file.path_resolved']
preserve_to: attributes['log.file.path_resolved']
timestamp:
parse_from: body.timestamp
layout_type: gotime
Expand Down Expand Up @@ -225,8 +225,8 @@ receivers:
to: body.log
- type: regex_parser
regex: (?P<timestamp>\d{4}-\d{2}-\d{2}-\d{2}\d{2}-\d{2})\d+
parse_from: attributes['file.path.resolved']
preserve_to: attributes['file.path.resolved']
parse_from: attributes['log.file.path_resolved']
preserve_to: attributes['log.file.path_resolved']
timestamp:
parse_from: body.timestamp
layout_type: gotime
Expand Down Expand Up @@ -262,8 +262,8 @@ receivers:
to: body.log
- type: regex_parser
regex: (?P<timestamp>\d{4}-\d{2}-\d{2}-\d{2}\d{2}-\d{2})\d+
parse_from: attributes['file.path.resolved']
preserve_to: attributes['file.path.resolved']
parse_from: attributes['log.file.path_resolved']
preserve_to: attributes['log.file.path_resolved']
timestamp:
parse_from: body.timestamp
layout_type: gotime
Expand Down Expand Up @@ -640,8 +640,8 @@ receivers:
to: body.log
- type: regex_parser
regex: (?P<timestamp>\d{4}-\d{2}-\d{2}-\d{2}\d{2}-\d{2})\d+
parse_from: attributes['file.path.resolved']
preserve_to: attributes['file.path.resolved']
parse_from: attributes['log.file.path_resolved']
preserve_to: attributes['log.file.path_resolved']
timestamp:
parse_from: body.timestamp
layout_type: gotime
Expand Down Expand Up @@ -677,8 +677,8 @@ receivers:
to: body.log
- type: regex_parser
regex: (?P<timestamp>\d{4}-\d{2}-\d{2}-\d{2}\d{2}-\d{2})\d+
parse_from: attributes['file.path.resolved']
preserve_to: attributes['file.path.resolved']
parse_from: attributes['log.file.path_resolved']
preserve_to: attributes['log.file.path_resolved']
timestamp:
parse_from: body.timestamp
layout_type: gotime
Expand Down
2 changes: 1 addition & 1 deletion pkg/exporter/sumologicexporter/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ Below is a list of all attribute keys that are being translated.
| `k8s.replicaset.name` | `replicaset` |
| `k8s.statefulset.name` | `statefulset` |
| `service.name` | `service` |
| `file.path.resolved` | `_sourceName` |
| `log.file.path_resolved` | `_sourceName` |

## Source Templates

Expand Down
4 changes: 2 additions & 2 deletions pkg/exporter/sumologicexporter/exporter_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -410,7 +410,7 @@ func TestPushOTLPLogs_AttributeTranslation(t *testing.T) {
resourceAttrs := logs.ResourceLogs().At(0).Resource().Attributes()
resourceAttrs.InsertString("host.name", "harry-potter")
resourceAttrs.InsertString("host.type", "wizard")
resourceAttrs.InsertString("file.path.resolved", "/tmp/log.log")
resourceAttrs.InsertString("log.file.path_resolved", "/tmp/log.log")
return logs
}

Expand Down Expand Up @@ -466,7 +466,7 @@ func TestPushOTLPLogs_AttributeTranslation(t *testing.T) {
body := extractBody(t, req)

//nolint:lll
assert.Equal(t, "\n\xd4\x01\n\xb8\x01\n\x1b\n\thost.name\x12\x0e\n\fharry-potter\n\x15\n\thost.type\x12\b\n\x06wizard\n$\n\x12file.path.resolved\x12\x0e\n\f/tmp/log.log\n\x1d\n\v_sourceHost\x12\x0e\n\fharry-potter\n=\n\x0f_sourceCategory\x12*\n(category_with_host_template_harry-potter\x12\x17\n\x00\x12\x13*\r\n\vExample logJ\x00R\x00", body)
assert.Equal(t, "\n\xd8\x01\n\xbc\x01\n\x1b\n\thost.name\x12\x0e\n\fharry-potter\n\x15\n\thost.type\x12\b\n\x06wizard\n(\n\x16log.file.path_resolved\x12\x0e\n\f/tmp/log.log\n\x1d\n\v_sourceHost\x12\x0e\n\fharry-potter\n=\n\x0f_sourceCategory\x12*\n(category_with_host_template_harry-potter\x12\x17\n\x00\x12\x13*\r\n\vExample logJ\x00R\x00", body)

assert.Empty(t, req.Header.Get("X-Sumo-Fields"),
"We should not get X-Sumo-Fields header when sending data with OTLP",
Expand Down
2 changes: 1 addition & 1 deletion pkg/exporter/sumologicexporter/translate_attributes.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ var attributeTranslations = map[string]string{
"k8s.replicaset.name": "replicaset",
"k8s.statefulset.name": "statefulset",
"service.name": "service",
"file.path.resolved": "_sourceName",
"log.file.path_resolved": "_sourceName",
}

func translateAttributes(attributes pcommon.Map) pcommon.Map {
Expand Down