fix: fix bug in query result marshaling for invalid utf8 characters #14585
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…d utf8 characters are present Signed-off-by: Callum Styan <[email protected]>
cstyan
changed the title
Signed-off-by: Callum Styan <[email protected]>
vlad-diachenko
approved these changes
Oct 23, 2024
pkg/util/marshal/query.go
Outdated
| } | ||
| return r | ||
| } | ||
| stream.Labels = string(bytes.Map(removeInvalidUtf, []byte(stream.Labels))) |
There was a problem hiding this comment.
it's better to use bytes.Map() only if []byte(stream.Labels) contains utf8.RuneError .
more details here
pkg/util/marshal/query.go
Outdated
| @@ -77,6 +79,14 @@ func NewStreams(s logqlmodel.Streams) (loghttp.Streams, error) { | |||
| ret := make([]loghttp.Stream, len(s)) | |||
|
|
|||
| for i, stream := range s { | |||
| // The rune error replacement is rejected by Prometheus hence replacing them with space. | |||
| removeInvalidUtf := func(r rune) rune { | |||
There was a problem hiding this comment.
can be moved to global vars to initialize once
| Labels: "{asdf=\"�\"}", | ||
| }, | ||
| }) | ||
| require.NoError(t, err) |
There was a problem hiding this comment.
it would be useful to assert the result as well
Signed-off-by: Callum Styan <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Potentially we want to fix this by actually sanitizing data for OTLP ingestion/structured metadata, but I think we should include a fix to allow users with data that is already stored in Loki to query it out.
The bug is present for streams which have structured metadata which contains invalid UTF-8 characters. When you write a query like
{label="x"}there's no problem, but as soon as you dolabel="x"} |= "some filter"the bug kicks in because we pull in the structured metadata.When we then go to marshal the labelset on the query result we run into an error here because the underlying call to
NewLabelSetcalls Prometheus'ParseMetricwhich enforces "any valid Unicode character" in the label value. This fails if there's an invalid character. I'm not entirely sure what the invalid character is or where it's coming from, but we have precedent for checking for it already asRuneError(here) in other places. For this PR I've simply copied what we do for the JSONParser from here.