Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] consumer backlog eviction policy should not reset read position of the consumer #18036

Closed
2 tasks done
HQebupt opened this issue Oct 13, 2022 · 0 comments · Fixed by HQebupt/pulsar#5 or #18037
Closed
2 tasks done

[Bug] consumer backlog eviction policy should not reset read position of the consumer #18036

HQebupt opened this issue Oct 13, 2022 · 0 comments · Fixed by HQebupt/pulsar#5 or #18037
Assignees
@HQebupt
Labels
type/bug The PR fixed a bug or issue reported a bug

Comments

@HQebupt
Copy link
Contributor

HQebupt commented Oct 13, 2022 •
edited
Loading

Search before asking

  • I searched in the issues and found nothing similar.

Version

  • Pulsar: latest
  • JDK: 17

Minimal reproduce step

  • Run unit test: org.apache.pulsar.broker.service.BacklogQuotaManagerTest#testConsumerBacklogEvictionTimeQuota

What did you expect to see?

  1. The backlog eviction policy should not the reset read position of the consumer. It is due to wrong method for resetting mark delete position.
  2. Reduce the resetting mark delete position request.

What did you see instead?

  1. When I run the unit test, I found the cursor c1 's read position was reset by BacklogQuotaManager. The backlog eviction policy should not the reset read position of the consumer. See the following log.
2022-10-13T17:50:37,488 - INFO  - [pulsar-backlog-quota-checker-64-1:BacklogQuotaManager@89] - Backlog quota type message_age exceeded for topic [persistent://prop/ns-quota/topic3c8347406-bbe5-40e9-9f08-eced5d838a42]. Applying [consumer_backlog_eviction] policy
2022-10-13T17:50:37,492 - INFO  - [BookKeeperClientWorker-OrderedExecutor-1-0:ManagedCursorImpl@1193] - [prop/ns-quota/persistent/topic3c8347406-bbe5-40e9-9f08-eced5d838a42] Initiate reset position to 1:0 on cursor c1

2022-10-13T17:50:37,628 - INFO  - [bookkeeper-ml-scheduler-OrderedScheduler-1-0:ManagedCursorImpl$30@2963] - [prop/ns-quota/persistent/topic3c8347406-bbe5-40e9-9f08-eced5d838a42] Updated cursor c1 with ledger id 3 md-position=0:-1 rd-position=2:4
2022-10-13T17:50:37,795 - INFO  - [BookKeeperClientWorker-OrderedExecutor-1-0:ManagedCursorImpl@1193] - [prop/ns-quota/persistent/topic3c8347406-bbe5-40e9-9f08-eced5d838a42] Initiate reset position to 2:0 on cursor c1
2022-10-13T17:50:37,799 - INFO  - [BookKeeperClientWorker-OrderedExecutor-1-0:ManagedCursorImpl$9@1245] - [prop/ns-quota/persistent/topic3c8347406-bbe5-40e9-9f08-eced5d838a42] reset position to 2:0 skipping from current read position 1:0 on cursor c1
  1. There are a lot of redundant reset mark delete position requests in the log.

image

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@HQebupt HQebupt added the type/bug The PR fixed a bug or issue reported a bug label Oct 13, 2022
@HQebupt HQebupt self-assigned this Oct 13, 2022
@sijie sijie mentioned this issue Oct 13, 2022
Open
2 tasks
This was referenced Oct 13, 2022
Merged
Merged
HQebupt added a commit to HQebupt/pulsar that referenced this issue Oct 19, 2022
@HQebupt
[fix][broker]consumer backlog eviction policy should not reset read p…
98e9089 
…osition for consumer (#5)

### Motivation
fix apache#18036

### Modifications
- The backlog eviction policy should use `asyncMarkDelete` instead of `resetCursor` in order to move the mark delete position.
HQebupt added a commit that referenced this issue Oct 19, 2022
@HQebupt
[improve][broker]consumer backlog eviction policy should not reset re…
7404e0d 
…ad position for consumer (#18037)

### Motivation
Fixes #18036

### Modifications
- The backlog eviction policy should use `asyncMarkDelete` instead of `resetCursor` in order to move the mark delete position.
Technoboy- pushed a commit that referenced this issue Nov 4, 2022
@HQebupt @Technoboy-
[improve][broker]consumer backlog eviction policy should not reset re…
ea22bec 
…ad position for consumer (#18037)

### Motivation
Fixes #18036

### Modifications
- The backlog eviction policy should use `asyncMarkDelete` instead of `resetCursor` in order to move the mark delete position.
Technoboy- pushed a commit that referenced this issue Nov 5, 2022
@HQebupt @Technoboy-
[improve][broker]consumer backlog eviction policy should not reset re…
0b7140b 
…ad position for consumer (#18037)

Fixes #18036

- The backlog eviction policy should use `asyncMarkDelete` instead of `resetCursor` in order to move the mark delete position.
@Technoboy- Technoboy- mentioned this issue Nov 5, 2022
Merged
5 tasks
nicoloboschi pushed a commit to datastax/pulsar that referenced this issue Nov 9, 2022
@HQebupt @nicoloboschi
[improve][broker]consumer backlog eviction policy should not reset re…
22371c2 
…ad position for consumer (apache#18037)

Fixes apache#18036

- The backlog eviction policy should use `asyncMarkDelete` instead of `resetCursor` in order to move the mark delete position.

(cherry picked from commit 0b7140b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@HQebupt HQebupt

Labels
type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Milestone
No milestone
1 participant
@HQebupt