Add shard not owned error

[jakobht]

NOTE: This PR: cadence-workflow/cadence-idl#179 needs to be merged before this one. This one then needs to be updated to reference the correct commit on the main branch in the IDL repo.

What changed?

• Added a new error type returned from matching, TaskListNotOwnedByHostError in thrift and proto IDLs
• We do not log errors on this error as it's an expected transient error

Why?

• We have to propagate the error type over the wire so we can check the error type on the history side, to be able to handle it gracefully.
• We should not have error logs on this error as it is expected and transient.

How did you test it?

• Unit tests
• Deployment in staging environment

Potential risks

Release notes

Documentation Changes

[codecov]

Codecov Report

Attention: Patch coverage is 88.88889% with 5 lines in your changes missing coverage. Please review.

Project coverage is 73.02%. Comparing base (ac349fc) to head (facd131).
Report is 2 commits behind head on master.

Files	Patch %	Lines
service/matching/handler/context.go	0.00%	3 Missing ⚠️
common/util.go	71.42%	2 Missing ⚠️

Additional details and impacted files

Files	Coverage Δ
common/types/mapper/proto/errors.go	100.00% <100.00%> (ø)
common/types/mapper/thrift/errors.go	100.00% <100.00%> (ø)
common/types/mapper/thrift/shared.go	97.71% <100.00%> (+<0.01%)	⬆️
service/history/task/task.go	79.52% <100.00%> (+0.39%)	⬆️
common/util.go	81.16% <71.42%> (-0.26%)	⬇️
service/matching/handler/context.go	43.63% <0.00%> (-2.52%)	⬇️

... and 4 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ac349fc...facd131. Read the comment docs.

[dkrotx]

with nit

[dkrotx]

@jakobht does it require specific order or rollout? Like frontend,history being rolled out before matching?
How Cadence clients react to this error? I expect this to be only within history<->matching and frontend<->matching. What frontend returns to clients as the result of hitting this error twice - no tasks?

[jakobht]

@dkrotx Ideally it would be frontend -> history -> matching.

But we will just get some log spam if not - so it's safe to do in any order. I don't think it's worth it to do any custom rollout.

[3vilhamster]

Looks good. This is a clear miss