Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Periodically pause copying ledger nodes during online_delete #4907

Draft
wants to merge 59 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
777365c
Periodically pause copying ledger nodes during online_delete
ximinez Jan 18, 2024
f9799af
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Feb 8, 2024
6f32213
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Feb 21, 2024
7c71d9e
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Feb 29, 2024
4f02b6e
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Mar 1, 2024
7f42b59
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Mar 7, 2024
811e7e5
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Mar 13, 2024
287dc29
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Mar 14, 2024
0d8b5bd
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Mar 19, 2024
11de364
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Mar 26, 2024
8001755
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Mar 28, 2024
56e1e51
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Apr 3, 2024
fccad25
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Apr 4, 2024
a8ee8d3
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Apr 5, 2024
e3624e3
Drastically change the node copy timing parameters:
ximinez Apr 11, 2024
66560d7
fixup! Drastically change the node copy timing parameters:
ximinez Apr 11, 2024
0cd1a5c
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Apr 19, 2024
ae0d69b
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Apr 19, 2024
0b5eac3
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Apr 19, 2024
9971265
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Apr 24, 2024
9539d23
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Apr 26, 2024
50a1317
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez May 1, 2024
bb4c426
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez May 7, 2024
fd02554
Merge commit 'c706926' into shamapstore-backoff
ximinez Jul 1, 2024
1e1b5f0
Merge commit 'f6879da' into shamapstore-backoff
ximinez Jul 1, 2024
da80396
Move CMake directory
Jul 1, 2024
72a1e89
Rearrange sources
Jul 1, 2024
e2a1cf7
Rewrite includes
Jul 1, 2024
91e5792
Recompute loops
Jul 1, 2024
fbaf9d0
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Jul 1, 2024
a857c2c
Fix formatting
ximinez Jul 2, 2024
88ce966
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Jul 5, 2024
10057ba
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Jul 10, 2024
49235da
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Jul 19, 2024
bdbea31
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Jul 24, 2024
a6043f1
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Jul 25, 2024
43d2e17
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Jul 29, 2024
4e59c22
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Aug 2, 2024
3344e62
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Aug 6, 2024
5d85323
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Aug 7, 2024
c7e496d
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Aug 19, 2024
f2bec52
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Sep 5, 2024
c09d1d2
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Sep 11, 2024
69f0435
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Sep 11, 2024
ed4574e
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Sep 25, 2024
06e70f8
* Stolen from or inspired by @jfreeman
ximinez Oct 15, 2024
2fa9f0a
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Oct 18, 2024
d49e9c6
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Oct 31, 2024
4622136
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Oct 31, 2024
3206f07
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Nov 4, 2024
1f1353e
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Nov 5, 2024
93f8267
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Nov 8, 2024
b44168f
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Nov 13, 2024
7c79952
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Nov 13, 2024
18bbf3d
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Nov 27, 2024
7abbef1
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Dec 3, 2024
071d70f
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Dec 4, 2024
b6ed609
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Dec 16, 2024
5e5ba5a
Merge remote-tracking branch 'upstream/develop' into shamapstore-backoff
ximinez Dec 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions src/ripple/app/misc/SHAMapStoreImp.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,12 @@ SHAMapStoreImp::copyNode(std::uint64_t& nodeCount, SHAMapTreeNode const& node)
if (healthWait() == stopping)
return false;
}
if (!(nodeCount % backoffInterval_))
{
JLOG(journal_.trace())
<< "copyNode backing off after " << nodeCount << " node writes";
std::this_thread::sleep_for(backOff_);
}

return true;
}
Expand Down
3 changes: 3 additions & 0 deletions src/ripple/app/misc/SHAMapStoreImp.h
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,9 @@ class SHAMapStoreImp : public SHAMapStore
std::string const dbPrefix_ = "rippledb";
// check health/stop status as records are copied
std::uint64_t const checkHealthInterval_ = 1000;
// back off periodically to reduce write contention as records are copied
std::uint64_t const backoffInterval_ =
ripple::NodeStore::batchWriteLimitSize / 2;
// minimum # of ledgers to maintain for health of network
static std::uint32_t const minimumDeletionInterval_ = 256;
// minimum # of ledgers required for standalone mode.
Expand Down
Loading