[run-ci][improve][txn]PIP-160 make txn components supports buffered writer metrics #3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
poorbarcode
force-pushed
the
pip/160-18
branch
3 times, most recently
from
2ca47e0 to
398a6e7
Compare
poorbarcode
force-pushed
the
pip/160-18
branch
from
f51ed87 to
bf7c69e
Compare
poorbarcode
changed the title
Merged
4 tasks
poorbarcode
force-pushed
the
pip/160-18
branch
2 times, most recently
from
d20eba4 to
355212b
Compare
Signed-off-by: tison <[email protected]> Signed-off-by: tison <[email protected]>
…e#17723) - The approval solution doesn't work as expected by approving the PR or by adding the ready-to-test label and adding a comment "/pulsarbot rerun-failure-checks". - Fix the ready-to-test label check: - Refresh PR labels when re-running workflow - when re-running, the event JSON remains the same. The API must be used to fetch the up-to-date JSON for the PR. - Fix the PR approval check: - set GITHUB_TOKEN for script so that retrieving the approval status could work
### Motivation in the schema update, will create a `ledgerHandle` and write data to BK, after that `ledgerHandle` is no longer useful and no other object holds references to it. `ledgerHandle` will be recycled with GC, but `ledgerHandle` also hold external connections, which will cause leakage. https://github.com/apache/pulsar/blob/40b9d7ea50cef54becb09f2543193e08375abe0b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/BookkeeperSchemaStorage.java#L452-L456 ### Modifications after the schema is updated, close the `ledgerHandle`, just like schema-read: https://github.com/apache/pulsar/blob/40b9d7ea50cef54becb09f2543193e08375abe0b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/BookkeeperSchemaStorage.java#L519-L525
…hapter (apache#17615) * Sidebar re-org * Separate TLS encryption/authentication using Keystore to parent topics * use language-specific tabs to show code snippets * streamline headings * Update client-libraries-java.md * Add code snippet for python clients to support apache#17482 * Add code snippet for go clients
…ception` (apache#17700) ### Motivation In the production environment, we found two brokers holding the same valid locks. and one has an exceptional revalidate future with `lockBusyException`. after reading the code, there may forget the reset the cache and complete expire exception when getting lockBusyException.
… cookbook (apache#17666) * remove duplicate file `cookbooks-encryption.md` * Revert "remove duplicate file `cookbooks-encryption.md`" This reverts commit a1e5a54. * streamline encryption cookbook and make it single-sourced * Add code snippets for Java/C++/Go clients. * use one svg image to replace two jpg files * client version and title updates * improve security overview * Add note and link for TLS encryption
…spatchThrottlingTest (apache#17702)
…ation (apache#16917) * [improve][txn][PIP-196] Segmented transaction buffer snapshot Master apache#16913 ### Motivation 1. Add configurations. 2. Implement the system topic client for snapshot topic and index topic. 3. Implement AbortedTxnProcessor. 4. Add transaction buffer snapshot metrics. ### Modification Add configuration for segment snaopshot.
…on (apache#16003) ### Motivation Now, the MongoDB source connector only supports the incremental message synchronization. This PR adds support for full message synchronization. Since MongDB 4.0, we can set the starting point for the change stream by the `startAtOperationTime` field. So, we can set it to `0` to make start point the earliest. See https://www.mongodb.com/docs/v4.2/reference/method/db.collection.watch/ for more information.
### Motivation The current `bin/pulsar version` script only prints out the version string. When running custom builds of Pulsar, this is sometimes not enough to know which version you have. Since the `PulsarVersion` class already has more information, I propose we print out all of its information in the same format that the broker logs the information. Here is a sample output for this specific PR: ```shell $ bin/pulsar version Current version of pulsar is: 2.11.0-SNAPSHOT Git Revision 4414e3d Git Branch print-more-version-info Built by Michael Marshall <[email protected]> on mmarshall-rmbp16 at 2020-01-22T09:10:15-0600 ``` ### Modifications * Add more output to the `bin/pulsar version` command. ### Verifying this change This change is a trivial rework / code cleanup without any test coverage. ### Documentation - [x] `doc-not-needed` (Please explain why) ### Matching PR in forked repository PR in forked repository: michaeljmarshall#3
…pache#17743) - when setup is called explicitly, cleanup should be issued before it - remove unnecessary "resetConfig" method calls - resetConfig gets called as part of internalCleanup - when resetConfig is in cleanup, it's possible to override config before calling setup - this allows using setup & cleanup methods instead of using error prone internalSetup and internalCleanup methods which might leave something behind
…mes topics in one broker (apache#17526)
* fix entryFilter stats * fix test * add test comment * review fix
### Motivation Capturing shared ptr in the timer function will cause a circular reference. ### Modifications - Use weak ptr instead shared ptr.
… C++-owned objects (apache#16535) Fixes apache#16527
…axMessageSize (apache#17289) ### Motivation See: apache#17188 ### Modifications Support include message header size when check maxMessageSize for cpp client
### Motivation
now `org.apache.pulsar.client.api.transaction.Transaction` dont have a interface for user to get the transaction state.
user can get the transaction state to do user's own op.
### Modifications
1. add the interface in `org.apache.pulsar.client.api.transaction.Transaction` `getState`
2. TransactionImpl implement the interface
```
* Get transaction state.
*
* @return {@link State} the state of the transaction.
*/
State getState();
```
### Verifying this change
add the test
* remove unused code * check style * remove unused bundleFactory Co-authored-by: leolinchen <[email protected]>
…pache#17456) Support lower boundary shedding for ThresholdShedder (apache#17456) The existing ThresholdShedder has the following problems, for example: There are 11 Brokers, of which 10 are loaded at 80% and 1 is loaded at 0%. The average load is 80 * 10 / 11 = 72.73, and the threshold to unload is 72.73 + 10 = 82.73. Since 80 < 82.73, unload will not be trigger, and there is one idle Broker with load of 0%. On the basis of ThresholdShedder, we adds the lower boundary judgment of the load. When 【current usage < average usage - threshold】, the broker with the highest load will be triggered to unload
poorbarcode
force-pushed
the
pip/160-18
branch
from
7ff2165 to
4fd5fa1
Compare
congbobo184
pushed a commit
to apache/pulsar
that referenced
this pull request
Sep 29, 2022
…trics (#17701) Master Issue: #15370 ### Modifications - Make transaction `MLTransactionMetadataStoreProvider` & `MLPendingAckStoreProvider` support buffered writer metrics. - Motivation: #15370 ---- - Delete constructor of `TxnLogBufferedWriter` without parameter `metrics`. - Motivation: it is unnecessary. ---- - Add a default `DisabledTxnLogBufferedWriterMetricsStats` implementation. ---- - Previous PR remaining code to optimize: remove the check code `if (metrics != null)`. The motivation see: - Motivation: #16758 (comment) ---- - Make transaction log buffered writer only create by the `MLTransactionMetadataStoreProvider` & `MLPendingAckStoreProvider`. - Motivation: #16758 (comment) ### Documentation - [ ] `doc-required` - [x] `doc-not-needed` - [ ] `doc` - [ ] `doc-complete` ### Matching PR in forked repository PR in forked repository: - poorbarcode#3
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
relate to: