Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][txn] Transaction cumulative ack redeliver change #14371

Conversation

congbobo184
Copy link
Contributor

@congbobo184 congbobo184 commented Feb 18, 2022

#10478

Motivation

since #10478 merged, we should change the cumulative ack with transaction abort redeliver logic. We can't redeliver unCumulativeAck message by the server because the client will receive the new message and ack then will receive the old message they abort.

in this case:

  1. we have 5 message
  2. cumulative ack 3 messages with the transaction
  3. we abort this transaction
  4. server redeliver message by the current consumer_epoch
  5. the client will not filter the 4 or 5 messages, because in [Broker] PIP:84 Redeliver command add epoch. #10478 we don't change the client consumer epoch
  6. client cumulative ack 4 5 with transaction and commit will lose the 1 2 3 messages and the consume message, not in order.

Modifications

don't redeliver any cumulative ack messages, it will do by user self

Verifying this change

don't need to add a test, the original test has been overwritten

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

  • Does this pull request introduces a new feature? (yes)

  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

  • If a feature is not applicable for documentation, explain why?

  • If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation

  • doc-not-needed

@github-actions
Copy link

@congbobo184:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

@congbobo184 congbobo184 added the doc-not-needed Your PR changes do not impact docs label Feb 18, 2022
@github-actions
Copy link

@congbobo184:Thanks for providing doc info!

eolivelli
eolivelli previously approved these changes Feb 18, 2022
@github-actions
Copy link

The pr had no activity for 30 days, mark with Stale label.

@congbobo184 congbobo184 changed the title [Transaction] Transaction cumulative ack redeliver change [fix][txn] Transaction cumulative ack redeliver change Apr 18, 2022
Copy link
Contributor

@gaoran10 gaoran10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense, LGTM, left a comment about the test.

…tion_cumulative_abort_redeliver_message

# Conflicts:
#	pulsar-broker/src/test/java/org/apache/pulsar/client/impl/TransactionEndToEndTest.java
Copy link
Contributor

@gaoran10 gaoran10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

congbobo184 added 2 commits July 12, 2022 20:12
@congbobo184 congbobo184 merged commit e6396bb into apache:master Jul 13, 2022
@codelipenghui codelipenghui added the release/important-notice The changes which are important should be mentioned in the release note label Jul 14, 2022
wuxuanqicn pushed a commit to wuxuanqicn/pulsar that referenced this pull request Jul 14, 2022
apache#10478

### Motivation
since apache#10478 merged, we should change the cumulative ack with transaction abort redeliver logic. We can't redeliver unCumulativeAck message by the server because the client will receive the new message and ack then will receive the old message they abort. 

in this case:
1. we have 5 message
2. cumulative ack 3 messages with the transaction
3. we abort this transaction
4. server redeliver message by the current consumer_epoch
5. the client will not filter the 4 or 5 messages, because in apache#10478 we don't change the client consumer epoch
6. client cumulative ack 4 5 with transaction and commit will lose the 1 2 3 messages and the consume message, not in order.
### Modifications
don't redeliver any cumulative ack messages, it will do by user self
congbobo184 added a commit that referenced this pull request Jul 14, 2022
…ge by users (#16592)

### Motivation
since #14371 merged, when the client abort txn with cumulative ack, We need to the redeliver message manually

### Modifications
change test when transaction abort with cumulative ack, redeliver message manually
### Verifying this change
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/transaction doc-not-needed Your PR changes do not impact docs lifecycle/stale release/important-notice The changes which are important should be mentioned in the release note
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants