-
Notifications
You must be signed in to change notification settings - Fork 805
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix reset reapply #3252
Fix reset reapply #3252
Conversation
service/history/historyEngine.go
Outdated
executionInfo := mutableState.GetExecutionInfo() | ||
maxAllowedSignals := e.config.MaximumSignalsPerExecution(domainEntry.GetInfo().Name) | ||
if maxAllowedSignals > 0 && (int(executionInfo.SignalCount)+len(toReapplyEvents)) >= maxAllowedSignals { | ||
e.logger.Info("Execution limit reached for maximum signals during signal re-application", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like a quite critical error, as it's non-retryable and the passive side doesn't even know the reapply has failed. Shall we use Error
level log and probably emit some metrics?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
signal reapplication should ignore this limit.
or at least return error
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove this check in this PR. If it returns an error, the caller side has. to handle this error or it will retry forever. I will include it in a different PR.
What changed?
Why?
This is to fix the feedback loop on reset and re-apply
How did you test it?
Local test
Potential risks
This feedback loop issue is not fixed and causes backlog in replication stack