Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][broker]add active status into cursor stats #17884

Merged
merged 1 commit into from
Oct 1, 2022

Conversation

HQebupt
Copy link
Contributor

@HQebupt HQebupt commented Sep 29, 2022

Motivation

The active state indicates whether a cursor is active. And it affects the cache hit rate, because inactive cursors will be evicted from the entry cache. This active state helps to troubleshoot issues with low cache hit rate.
Also it is meaningful for configuring a suitable values for managedLedgerCursorBackloggedThreshold in ServiceConfiguration

# Configure the threshold (in number of entries) from where a cursor should be considered 'backlogged'
# and thus should be set as inactive.
managedLedgerCursorBackloggedThreshold=1000

Modifications

Add active status into cursor stats.
It will get the following sample data when getting the internalStats of a partitioned topic.

{
    "entriesAddedCounter": 6,
    "numberOfEntries": 10302152,
    "totalSize": 63786554161,
    "currentLedgerEntries": 6,
    "currentLedgerSize": 31182,
    "lastLedgerCreatedTimestamp": "2022-09-29T14:48:52.825+08:00",
    "waitingCursorsCount": 0,
    "pendingAddEntriesCount": 0,
    "lastConfirmedEntry": "1359228:5",
    "lastIndex": 1165055506,
    "state": "LedgerOpened",
    "ledgers": [
        {
            "ledgerId": 1356881,
            "entries": 50000,
            "size": 309705120,
            "offloaded": false,
            "underReplicated": false
        }
    ],
    "cursors": {
        "cg_test_map_cache1": {
            "markDeletePosition": "1356881:3399",
            "readPosition": "1356881:3400",
            "waitingReadOp": false,
            "pendingReadOps": 0,
            "messagesConsumedCounter": -10298746,
            "cursorLedger": -1,
            "cursorLedgerLastEntry": -1,
            "individuallyDeletedMessages": "[]",
            "lastLedgerSwitchTimestamp": "2022-09-29T14:48:52.859+08:00",
            "state": "NoLedger",
            "numberOfEntriesSinceFirstNotAckedMessage": 1,
            "totalNonContiguousDeletedMessagesRange": 0,
            "subscriptionHavePendingRead": false,
            "subscriptionHavePendingReplayRead": false,
            "active": false,
            "properties": {
                "index": 846056487
            }
        },
        "cg_test_map_cache2": {
            "markDeletePosition": "1359208:79",
            "readPosition": "1359228:6",
            "waitingReadOp": false,
            "pendingReadOps": 0,
            "messagesConsumedCounter": -8,
            "cursorLedger": 1359234,
            "cursorLedgerLastEntry": 1,
            "individuallyDeletedMessages": "[]",
            "lastLedgerSwitchTimestamp": "2022-09-29T14:48:52.859+08:00",
            "state": "Open",
            "numberOfEntriesSinceFirstNotAckedMessage": 15,
            "totalNonContiguousDeletedMessagesRange": 0,
            "subscriptionHavePendingRead": false,
            "subscriptionHavePendingReplayRead": false,
            "active": true,
            "properties": {
                "index": 1165055008
            }
        }
    },
    "schemaLedgers": [],
    "compactedLedger": {
        "ledgerId": -1,
        "entries": -1,
        "size": -1,
        "offloaded": false,
        "underReplicated": false
    }
}

Verifying this change

  • Make sure that the change passes the CI checks.

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

Check the box below or label this PR directly.

Need to update docs?

  • doc-required
  • doc-not-needed
  • doc
  • doc-complete

Matching PR in forked repository

PR in forked repository: HQebupt#1

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Sep 29, 2022
@HQebupt
Copy link
Contributor Author

HQebupt commented Sep 29, 2022

@AnonHxy @codelipenghui @Jason918 @Technoboy- @mattisonchao PTAL thanks. The matching PR pass the tests.

@AnonHxy
Copy link
Contributor

AnonHxy commented Sep 30, 2022

LGTM

@HQebupt
Copy link
Contributor Author

HQebupt commented Oct 1, 2022

/pulsarbot ready-to-test

@HQebupt
Copy link
Contributor Author

HQebupt commented Oct 1, 2022

/pulsarbot run-failure-checks

1 similar comment
@HQebupt
Copy link
Contributor Author

HQebupt commented Oct 1, 2022

/pulsarbot run-failure-checks

@Jason918 Jason918 added this to the 2.12.0 milestone Oct 1, 2022
@Jason918 Jason918 merged commit 8c1152c into apache:master Oct 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-not-needed Your PR changes do not impact docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants