Fix Deadlock in groupbytrace #1846
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chris-smith-zocdoc
requested review from
bogdandrutu,
dmitryax,
james-bebbington,
nilebox,
owais,
pjanotti and
tigrannajaryan
as code owners
|
Opps wrong branch, heres the real one #1847 |
MovieStoreGuy
pushed a commit
to atlassian-forks/opentelemetry-collector
that referenced
this pull request
Nov 11, 2021
* Move the Event type from the API to the SDK This type is not used in the API. It is used in the SDK and then through the processing pipelines. Move it to the package that originates its use. * Add changes to changelog * Update CHANGELOG.md Co-authored-by: Gustavo Silva Paiva <[email protected]> Co-authored-by: Gustavo Silva Paiva <[email protected]>
hughesjj
pushed a commit
to hughesjj/opentelemetry-collector
that referenced
this pull request
Apr 27, 2023
…y#1846) Bumps [boto3](https://github.com/boto/boto3) from 1.24.45 to 1.24.46. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](boto/boto3@1.24.45...1.24.46) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Troels51
pushed a commit
to Troels51/opentelemetry-collector
that referenced
this pull request
Jul 5, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description: <Describe what has changed.
Fixes #1811
The deadlock in the groupbytrace processor was caused by the consumer goroutine (the one consuming from the event machine) trying to write data to the channel. This would work sometimes because of the channel buffer but deadlock once the buffer became full. I've refactored the design so that the event machine is much simpler and only has two events
traceReceivedandtraceExpired. By doing this I've removed the cases where the reader would need to write events, removing the possibility of deadlocks. This also moved all modifications to the datastructures to happen under the consumer goroutine which fixes some other synchronization bugs with concurrent access to the storage becoming out of sync with the ringbuffer.A second deadlock was discovered in the shutdown procedure where the channel and mutex locks could be obtained in the inverse order, leading to deadlock. I replaced the mutex with a channel and select which will avoid this.
Testing:
Added test cases
@jpkrohling