-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feat] Add scripts for updating BK RocksDB ini files #23178
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #23178 +/- ##
============================================
+ Coverage 73.57% 74.58% +1.01%
- Complexity 32624 33555 +931
============================================
Files 1877 1921 +44
Lines 139502 144480 +4978
Branches 15299 15809 +510
============================================
+ Hits 102638 107761 +5123
+ Misses 28908 28461 -447
- Partials 7956 8258 +302
Flags with carried forward coverage won't be shown. Click here to find out more. |
(cherry picked from commit 3ada566)
(cherry picked from commit 3ada566)
One of the tunings that is lost in Pulsar 2.11 / Bookkeeper 4.15 is the automatic adjustment of
since Pulsar 2.11 / Bookkeeper 4.15, this isn't automatically adjusted. the value is fixed to 206150041 which is less than 200MB. |
Fixes #23176
Motivation
The way RocksDB is configured changed in Pulsar 2.11 / Bookkeeper 4.15 .
The settings are handled in
conf/entry_location_rocksdb.conf
file anddbStorage_rocksDB_*
keys in bookkeeper.conf have no effect. This is mainly a problem for Kubernetes deployments since there's not an easy way to override the configs.Modifications
update-ini-from-env.py
for updating keys in RocksDB ini files by environment variablesupdate-rocksdb-conf-from-env.py
for updatingconf/entry_location_rocksdb.conf
file withdbStorage_rocksDB_*
keys in the environment.Additional context
With these changes, it's possible to modify a Pulsar Helm chart's bookkeeper/autorecovery commands to include
&& (([ -x bin/update-rocksdb-conf-from-env.py ] && bin/update-rocksdb-conf-from-env.py) || true)
afterbin/apply-config-from-env.py conf/bookkeeper.conf
Documentation
doc
doc-required
doc-not-needed
doc-complete