-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[refactor][java] Unify the acknowledge process for batch and non-batch message IDs #17833
Merged
BewareMyPower
merged 3 commits into
apache:master
from
BewareMyPower:bewaremypower/refactor-ack
Oct 6, 2022
Merged
[refactor][java] Unify the acknowledge process for batch and non-batch message IDs #17833
BewareMyPower
merged 3 commits into
apache:master
from
BewareMyPower:bewaremypower/refactor-ack
Oct 6, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BewareMyPower
added
area/client
type/refactor
Code or documentation refactors. e.g. refactor code structure or methods to improve code readability
release/2.9.4
release/2.8.5
release/2.11.1
release/2.10.3
labels
Sep 25, 2022
BewareMyPower
requested review from
merlimat,
Technoboy-,
eolivelli,
codelipenghui,
congbobo184,
michaeljmarshall and
mattisonchao
September 25, 2022 12:43
BewareMyPower
added
type/bug
The PR fixed a bug or issue reported a bug
and removed
type/bug
The PR fixed a bug or issue reported a bug
labels
Sep 25, 2022
BewareMyPower
force-pushed
the
bewaremypower/refactor-ack
branch
from
September 25, 2022 16:28
d4d5425
to
45b86e3
Compare
eolivelli
reviewed
Sep 25, 2022
pulsar-client/src/main/java/org/apache/pulsar/client/impl/BatchMessageIdImpl.java
Outdated
Show resolved
Hide resolved
shoothzj
previously approved these changes
Sep 26, 2022
13 tasks
lhotari
reviewed
Sep 26, 2022
...nt/src/main/java/org/apache/pulsar/client/impl/PersistentAcknowledgmentsGroupingTracker.java
Outdated
Show resolved
Hide resolved
…h message IDs ### Motivation When I read the acknowledge related code, I found much repeat code and the inconsistent code style. Generally we should handle the following cases for a given message ID: 1. The message ID doesn't contain a valid batch index, i.e. `messageId instanceof BatchMessageIdImpl` returns false. 2. Otherwise, it's a `BatchMessageIdImpl`. The behavior is determined by whether the batch index ACK is enabled. For individual ACKs, the stats and unacked message tracker must be updated according to the type of message ID. Since `MessageIdImpl` is used as the key, for a `BatchMessageIdImpl`, we must create a `MessageIdImpl` from it as the key of these hash maps. The existing methods `modifyBatchMessageIdAndStatesInConsumer` and `modifyMessageIdStatesInConsumer` are confused. So I decide to make the code simple and clear. ### Modifications - Add a `getMessageIdImpl` method to `BatchMessageIdImpl` to create a `MessageIdImpl` object from it. - Add an `addAcknowledgement` overload to accept an extra `BatchMessageIdImpl` argument, which is used to compute the number of ACK commands sent and perform batch related ACKs. The `MessageIdImpl` argument is used to update the DLQ and unacked tracker. - Add a `addIndividualAcknowledgement` method for individual ACKs. This method accept two functions to handle ACK for batch and non-batch message IDs because when a list of message IDs are acknowledeged, `doIndividualAckAsync` and `doIndividualBatchAck` will be called instead of `doIndividualAck` and `doIndividualBatchAck`. The previous `modify...InConsumer` methods will be merged into the following case in `addIndividualAcknowledgement`: ```java if (batchMessageId == null || batchMessageId.ackIndividual()) ```
BewareMyPower
force-pushed
the
bewaremypower/refactor-ack
branch
from
September 27, 2022 16:13
5607203
to
dfb79ba
Compare
2 tasks
lhotari
approved these changes
Sep 28, 2022
/pulsarbot rerun-failure-checks |
@eolivelli @shoothzj Could you take a second look? |
shoothzj
approved these changes
Oct 2, 2022
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
Dec 6, 2022
…h message IDs (apache#17833) (cherry picked from commit 83309ed) (cherry picked from commit 331008c)
congbobo184
pushed a commit
to congbobo184/pulsar
that referenced
this pull request
Dec 7, 2022
…h message IDs (apache#17833) (cherry picked from commit 83309ed)
Technoboy-
pushed a commit
that referenced
this pull request
Feb 8, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area/client
cherry-picked/branch-2.9
Archived: 2.9 is end of life
cherry-picked/branch-2.10
cherry-picked/branch-2.11
doc-not-needed
Your PR changes do not impact docs
release/2.8.5
release/2.9.4
release/2.10.3
release/2.11.1
type/refactor
Code or documentation refactors. e.g. refactor code structure or methods to improve code readability
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
When I read the acknowledge related code, I found much repeat code and
the inconsistent code style. Generally we should handle the following
cases for a given message ID:
messageId instanceof BatchMessageIdImpl
returns false.BatchMessageIdImpl
. The behavior is determined bywhether the batch index ACK is enabled.
For individual ACKs, the stats and unacked message tracker must be
updated according to the type of message ID. Since
MessageIdImpl
isused as the key, for a
BatchMessageIdImpl
, we must create aMessageIdImpl
from it as the key of these hash maps.The existing methods
modifyBatchMessageIdAndStatesInConsumer
andmodifyMessageIdStatesInConsumer
are confused. So I decide to make thecode simple and clear.
Modifications
toMessageIdImpl
method toBatchMessageIdImpl
to create aMessageIdImpl
object from it.addAcknowledgement
overload to accept an extraBatchMessageIdImpl
argument, which is used to compute the number ofACK commands sent and perform batch related ACKs. The
MessageIdImpl
argument is used to update the DLQ and unacked tracker.
addIndividualAcknowledgement
method for individual ACKs. Thismethod accept two functions to handle ACK for batch and non-batch
message IDs because when a list of message IDs are acknowledeged,
doIndividualAckAsync
anddoIndividualBatchAck
will be calledinstead of
doIndividualAck
anddoIndividualBatchAck
.The previous
modify...InConsumer
methods will be merged into thefollowing case in
addIndividualAcknowledgement
:Verifying this change
Rename
ConsumerAckResponseTest
toConsumerAckTest
. Then add some tests to verify the following states will be modified as expected.onAcknowledge
andonAcknowledgeCumulative
methods.UnAckedMessageTracker
instance in the consumer.numAcksSent
consumer stats.In three cases:
acknowledge
a single message IDacknowledge
a list of message IDsacknowledgeCumulative
a single message IDDocumentation
doc-required
(Your PR needs to update docs and you will update later)
doc-not-needed
(Please explain why)
doc
(Your PR contains doc changes)
doc-complete
(Docs have been already added)
Matching PR in forked repository
PR in forked repository: BewareMyPower#4