Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

consumer(cdc): fix unstable test move_table #5086

Merged
merged 32 commits into from
Apr 7, 2022
Merged

consumer(cdc): fix unstable test move_table #5086

merged 32 commits into from
Apr 7, 2022

Conversation

3AceShowHand
Copy link
Contributor

@3AceShowHand 3AceShowHand commented Mar 31, 2022 •
edited by Rustin170506
Loading

What problem does this PR solve?

Issue Number: close #2887, close #5101, close #5090

https://github.com/Shopify/sarama/blob/main/config.go#L38-L44
https://github.com/Shopify/sarama/blob/main/config.go#L461

https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html#producerconfigs_max.in.flight.requests.per.connection

What is changed and how it works?

Consumer may receive RowChangedEvent out of order by the commitTs, may caused by the the issues mentioned above.

  • Emit row changed event if resolved ts received, just like what cdc do .

revert sarama to 1.29.0, to prevent data race introduced by sarama v1.32.0

Check List

Tests

  • No code

Code changes

Side effects

Related changes

Release note 

`None`.

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Mar 31, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • overvenus
  • sdojjy

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Mar 31, 2022
@ti-chi-bot ti-chi-bot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Mar 31, 2022
@ti-chi-bot ti-chi-bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Mar 31, 2022
@3AceShowHand 3AceShowHand added the type/enhancement The issue or PR belongs to an enhancement. label Mar 31, 2022
@ti-chi-bot ti-chi-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Mar 31, 2022
@3AceShowHand
Copy link
Contributor Author

/run-kafka-integration-test

@ti-chi-bot ti-chi-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 1, 2022
@3AceShowHand
Copy link
Contributor Author

/run-kafka-integration-test

@3AceShowHand
Copy link
Contributor Author

/run-kafka-integration-test

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. and removed release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Apr 1, 2022
@ti-chi-bot ti-chi-bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Apr 6, 2022
@3AceShowHand
Copy link
Contributor Author

/run-all-tests

@3AceShowHand 3AceShowHand force-pushed the fix-kafka-consumer-commitTs-regression branch from 7df7c96 to 48c8d39 Compare April 6, 2022 10:27
@ti-chi-bot ti-chi-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 6, 2022
@3AceShowHand
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 48c8d39

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 6, 2022
@wjhuang2016
Copy link
Member

/run-verify

@lance6716
Copy link
Contributor

/hold

@ti-chi-bot ti-chi-bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. and removed status/can-merge Indicates a PR has been approved by a committer. labels Apr 6, 2022
@lance6716 lance6716 force-pushed the fix-kafka-consumer-commitTs-regression branch from 23da98e to cca6726 Compare April 6, 2022 12:22
@wjhuang2016
Copy link
Member

/run-verify

3 similar comments
@wjhuang2016
Copy link
Member

/run-verify

@wjhuang2016
Copy link
Member

/run-verify

@lance6716
Copy link
Contributor

/run-verify

@3AceShowHand 3AceShowHand force-pushed the fix-kafka-consumer-commitTs-regression branch from 9d00fd6 to 48c8d39 Compare April 7, 2022 08:21
@3AceShowHand
Copy link
Contributor Author

/unhold

@ti-chi-bot ti-chi-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 7, 2022
@3AceShowHand
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 88e3293

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 7, 2022
@3AceShowHand
Copy link
Contributor Author

/run-kafka-integration-test

@3AceShowHand
Copy link
Contributor Author

/run-integration-test

@ti-chi-bot ti-chi-bot merged commit 361aa92 into pingcap:master Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@overvenus overvenus overvenus approved these changes

@sdojjy sdojjy sdojjy approved these changes

@zhaoxinyu zhaoxinyu Awaiting requested review from zhaoxinyu

@Rustin170506 Rustin170506 Awaiting requested review from Rustin170506

Assignees
No one assigned
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
8 participants
@3AceShowHand @ti-chi-bot @codecov-commenter @Rustin170506 @wjhuang2016 @lance6716 @sdojjy @overvenus