Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

grpc OTLPSpanExporter max retry timeout is not configurable #4331

Open
Dan-Knott opened this issue Nov 28, 2024 · 1 comment · May be fixed by #4333
Open

grpc OTLPSpanExporter max retry timeout is not configurable #4331

Dan-Knott opened this issue Nov 28, 2024 · 1 comment · May be fixed by #4333

Comments

@Dan-Knott
Copy link

Dan-Knott commented Nov 28, 2024 •
edited
Loading

The exponential backoff in the grpc version of the OTLPSpanExporter has its max retry timeout hard coded, meaning there is no way to configure the retry behaviour of this exporter. The value should be configurable like with the http version of the OTLPSpanExporter.

opentelemetry-python/exporter/opentelemetry-exporter-otlp-proto-grpc/src/opentelemetry/exporter/otlp/proto/grpc/exporter.py

Lines 271 to 293 in 2ddb076

def _export(
self, data: Union[TypingSequence[ReadableSpan], MetricsData]
) -> ExportResultT:
# After the call to shutdown, subsequent calls to Export are
# not allowed and should return a Failure result.
if self._shutdown:
logger.warning("Exporter already shutdown, ignoring batch")
return self._result.FAILURE
# FIXME remove this check if the export type for traces
# gets updated to a class that represents the proto
# TracesData and use the code below instead.
# logger.warning(
# "Transient error %s encountered while exporting %s, retrying in %ss.",
# error.code(),
# data.__class__.__name__,
# delay,
# )
max_value = 64
# expo returns a generator that yields delay values which grow
# exponentially. Once delay is greater than max_value, the yielded
# value will remain constant.
for delay in _create_exp_backoff_generator(max_value=max_value):

@awoimbee
Copy link

At $dayjob this is causing issues where a non critical OTLP endpoint not being available causes a critical API endpoint to not be available.

This was referenced Nov 28, 2024
Closed
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@awoimbee @Dan-Knott