Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update mutable state to generate workflow requests #5821

Merged
merged 2 commits into from
Apr 17, 2024

Conversation

Shaddoll
Copy link
Member

@Shaddoll Shaddoll commented Mar 27, 2024

What changed?
This PR is built on top of #5826.
In this PR, we generate workflow requests from external API requests and replication events and store them in database to detect duplicated requests. If a duplicated requests is detected, a DuplicateRequestError is returned by persistence layer with the run_id telling upper layer which run the request has been applied to. And when this error is returned, the API does no-op and return the run_id to the caller.

Why?
To improve idempotency of Cadence APIs

How did you test it?
unit tests

Potential risks
We have a feature flag to turn on/off this feature. And we'll rollout this feature at domain level.

Release notes

Documentation Changes

Copy link

codecov bot commented Mar 27, 2024

Codecov Report

Merging #5821 (ddb429c) into master (6d8466c) will increase coverage by 0.04%.
The diff coverage is 86.30%.

Additional details and impacted files
Files Coverage Δ
service/history/execution/mutable_state_util.go 72.75% <100.00%> (+0.51%) ⬆️
service/history/execution/state_rebuilder.go 72.13% <ø> (ø)
service/history/ndc/activity_replicator.go 85.36% <100.00%> (+0.11%) ⬆️
service/history/ndc/events_reapplier.go 90.00% <100.00%> (ø)
...story/ndc/existing_workflow_transaction_manager.go 84.83% <100.00%> (+0.04%) ⬆️
...ce/history/ndc/new_workflow_transaction_manager.go 84.57% <100.00%> (+0.29%) ⬆️
service/history/ndc/transaction_manager.go 72.81% <100.00%> (+0.13%) ⬆️
service/history/reset/resetter.go 60.05% <100.00%> (+0.10%) ⬆️
service/history/shard/context.go 30.30% <ø> (ø)
service/history/workflow/util.go 48.57% <100.00%> (+0.99%) ⬆️
... and 5 more

... and 5 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d8466c...ddb429c. Read the comment docs.

@coveralls
Copy link

coveralls commented Mar 27, 2024

Pull Request Test Coverage Report for Build 018eea45-745a-45b9-9dc1-c4e30fd5979f

Details

  • 165 of 187 (88.24%) changed or added relevant lines in 15 files are covered.
  • 63 unchanged lines in 12 files lost coverage.
  • Overall coverage increased (+0.03%) to 67.41%

Changes Missing Coverage Covered Lines Changed/Added Lines %
service/history/config/config.go 2 3 66.67%
common/log/tag/tags.go 0 3 0.0%
service/history/execution/mutable_state_decision_task_manager.go 31 35 88.57%
service/history/execution/context.go 48 54 88.89%
service/history/engine/engineimpl/historyEngine.go 24 32 75.0%
Files with Coverage Reduction New Missed Lines %
common/task/weighted_round_robin_task_scheduler.go 2 89.05%
common/task/fifo_task_scheduler.go 2 85.57%
common/persistence/sql/sqlplugin/mysql/task.go 2 73.68%
service/matching/db.go 2 73.23%
service/matching/matcher.go 2 90.72%
common/persistence/sql/sqlplugin/mysql/db.go 2 79.49%
common/log/tag/tags.go 3 50.46%
service/history/task/fetcher.go 3 85.57%
common/types/shared.go 4 40.23%
common/persistence/nosql/nosql_task_store.go 5 61.95%
Totals Coverage Status
Change from base Build 018ee46d-6495-47ba-900b-2030e24c5f31: 0.03%
Covered Lines: 98662
Relevant Lines: 146362

💛 - Coveralls

@Shaddoll Shaddoll force-pushed the request branch 6 times, most recently from 1497967 to 5899eb4 Compare April 15, 2024 19:19
@Shaddoll Shaddoll marked this pull request as ready for review April 15, 2024 19:47
@Shaddoll Shaddoll changed the title WIP: Update mutable state to generate and replicate requestIDs Update mutable state to generate workflow requests Apr 15, 2024
@Shaddoll Shaddoll force-pushed the request branch 3 times, most recently from a2f6f2d to efb6994 Compare April 16, 2024 22:45
)
if t, ok := err.(*persistence.DuplicateRequestError); ok {
if t.RequestType == persistence.WorkflowRequestTypeStart || (isSignalWithStart && t.RequestType == persistence.WorkflowRequestTypeSignal) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we check (!isSignalWithStart && t.RequestType == persistence.WorkflowRequestTypeStart) || (isSignalWithStart && t.RequestType == persistence.WorkflowRequestTypeSignal)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, start is included in signalwithstart request

)
if t, ok := err.(*persistence.DuplicateRequestError); ok {
Copy link
Member

@davidporter-id-au davidporter-id-au Apr 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: prefer errors.Is(err, persistence.DuplicateRequestError). That'll allow us to provide further information for the error in the future without affecting the control flow

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need type assertion, because we need to retrieve runID from the error

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

errors.As would do the job then

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, my apologies, errors.As, though be careful to test it, it's double-pointer API is kinda confusing.

var e *persistence.DuplicateRequestError
if errors.As(err, &e) {
  ...
}

@@ -3101,3 +3112,75 @@ func TestUpdateWorkflowExecutionWithNewAsPassive(t *testing.T) {
})
}
}

func TestValidateWorkflowRequestsAndMode(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will you have a unit test that covers the error case in the conflict resolution flow for this change when that's updated?

Copy link
Member

@davidporter-id-au davidporter-id-au left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm to the extent that I've been following + some nits

@Shaddoll Shaddoll merged commit baac621 into cadence-workflow:master Apr 17, 2024
20 checks passed
@Shaddoll Shaddoll deleted the request branch April 17, 2024 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants