[Enhancement] auto change replication_num of system tables #51799

murphyatwork · 2024-10-11T10:58:16Z

Why I'm doing:

When scale-in the cluster from 3 nodes to 1 node:

Since the system tables in _statistics_ database will have 3 replicas, they would stop the scale-in
The current solution is to drop this database and recreate it, but which is not safe actually

What I'm doing:

Auto change replication_num of several system tables

When scaling out to more than 3 nodes, change the replication_num to 3
When scaling in to 1 node, change the replication_num to 1, otherwise scale-in would never succeed

Affected system tables:

column_statistics
histogram_statistics
table_statistic_v1
external_column_statistics
external_histogram_statistics
3.2: pipe_file_list
3.3: loads_history
3.3: task_run_history

Behavior change:

before: the replication_num will only be increased but not decreased
now: the replication_num will be increased when scale-out, and decreased when scale-in

Fixes #issue

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
This is a backport pr

Bugfix cherry-pick branch check:

Signed-off-by: Murphy <[email protected]>

fe/fe-core/src/main/java/com/starrocks/statistic/StatisticsMetaManager.java

fe/fe-core/src/main/java/com/starrocks/scheduler/history/TableKeeper.java

fe/fe-core/src/main/java/com/starrocks/load/loadv2/LoadsHistorySyncer.java

Signed-off-by: Murphy <[email protected]>

nshangyiming · 2024-10-12T02:00:32Z

fe/fe-core/src/main/java/com/starrocks/statistic/StatisticUtils.java

+     * 1. When scale-out to greater than 3 nodes, change the replication_num to 3
+     * 3. When scale-in to less than 3 node, change it to retainedBackendNum
+     */
+    public static boolean alterSystemTableReplicationNumIfNecessary(String tableName) {


Behavior change: before: the replication_num will only be increased but not decreased now: the replication_num will be increased when scale-out, and decreased when scale-in

plz add test case for this behavior change

Signed-off-by: Murphy <[email protected]>

sonarcloud · 2024-10-14T07:39:03Z

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

github-actions · 2024-10-14T09:06:38Z

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2024-10-14T09:07:14Z

[FE Incremental Coverage Report]

✅ pass : 45 / 50 (90.00%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	com/starrocks/load/pipe/filelist/FileListTableRepo.java	1	2	50.00%	[164]
🔵	com/starrocks/load/loadv2/LoadsHistorySyncer.java	2	3	66.67%	[157]
🔵	com/starrocks/system/SystemInfoService.java	4	5	80.00%	[799]
🔵	com/starrocks/scheduler/history/TableKeeper.java	7	8	87.50%	[158]
🔵	com/starrocks/statistic/StatisticsMetaManager.java	13	14	92.86%	[303]
🔵	com/starrocks/load/pipe/filelist/RepoCreator.java	4	4	100.00%	[]
🔵	com/starrocks/statistic/StatisticUtils.java	14	14	100.00%	[]

github-actions · 2024-10-14T09:07:25Z

[BE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2024-10-14T11:17:43Z

@Mergifyio backport branch-3.3

github-actions · 2024-10-14T11:17:44Z

@Mergifyio backport branch-3.2

mergify · 2024-10-14T11:17:58Z

backport branch-3.3

✅ Backports have been created

#51885 [Enhancement] auto change replication_num of system tables (backport #51799) has been created for branch branch-3.3 but encountered conflicts

mergify · 2024-10-14T11:18:00Z

backport branch-3.2

✅ Backports have been created

#51886 [Enhancement] auto change replication_num of system tables (backport #51799) has been created for branch branch-3.2 but encountered conflicts

Signed-off-by: Murphy <[email protected]> (cherry picked from commit 0c0ea45) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/load/loadv2/LoadsHistorySyncer.java # fe/fe-core/src/main/java/com/starrocks/load/pipe/filelist/RepoCreator.java # fe/fe-core/src/main/java/com/starrocks/statistic/StatisticsMetaManager.java

Signed-off-by: Murphy <[email protected]> (cherry picked from commit 0c0ea45) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/load/loadv2/LoadsHistorySyncer.java # fe/fe-core/src/main/java/com/starrocks/load/pipe/filelist/RepoCreator.java # fe/fe-core/src/main/java/com/starrocks/scheduler/history/TableKeeper.java # fe/fe-core/src/main/java/com/starrocks/scheduler/history/TaskRunHistoryTable.java # fe/fe-core/src/main/java/com/starrocks/statistic/StatisticUtils.java # fe/fe-core/src/main/java/com/starrocks/statistic/StatisticsMetaManager.java # fe/fe-core/src/test/java/com/starrocks/scheduler/history/TaskRunHistoryTest.java

Signed-off-by: Murphy <[email protected]> (cherry picked from commit 0c0ea45) Signed-off-by: Murphy <[email protected]> # Conflicts: # fe/fe-core/src/main/java/com/starrocks/load/loadv2/LoadsHistorySyncer.java # fe/fe-core/src/main/java/com/starrocks/load/pipe/filelist/RepoCreator.java # fe/fe-core/src/main/java/com/starrocks/statistic/StatisticsMetaManager.java

…#51799) Signed-off-by: Murphy <[email protected]>

…#51799) Signed-off-by: Murphy <[email protected]> Signed-off-by: zhiminr.ren <[email protected]>

…51799) (#51886) Signed-off-by: Murphy <[email protected]> Co-authored-by: Murphy <[email protected]> Co-authored-by: Murphy <[email protected]>

Signed-off-by: Murphy <[email protected]> (cherry picked from commit 0c0ea45) Signed-off-by: Murphy <[email protected]>

…51799) (#52766) Signed-off-by: Murphy <[email protected]>

murphyatwork added 3 commits October 11, 2024 18:23

fix load_history & task_run_history

c64306e

Signed-off-by: Murphy <[email protected]>

fix pipe_file_list

cea63b9

Signed-off-by: Murphy <[email protected]>

fix statistics

09f1e5a

Signed-off-by: Murphy <[email protected]>

murphyatwork requested review from a team as code owners October 11, 2024 10:58

mergify bot assigned murphyatwork Oct 11, 2024

github-actions bot added 3.3 3.2 labels Oct 11, 2024

starrocks-cr bot reviewed Oct 11, 2024

View reviewed changes

fe/fe-core/src/main/java/com/starrocks/statistic/StatisticsMetaManager.java Show resolved Hide resolved

starrocks-cr bot reviewed Oct 11, 2024

View reviewed changes

fe/fe-core/src/main/java/com/starrocks/scheduler/history/TableKeeper.java Show resolved Hide resolved

starrocks-cr bot reviewed Oct 11, 2024

View reviewed changes

fe/fe-core/src/main/java/com/starrocks/load/loadv2/LoadsHistorySyncer.java Show resolved Hide resolved

refactor

56af81f

Signed-off-by: Murphy <[email protected]>

satanson previously approved these changes Oct 11, 2024

View reviewed changes

kangkaisen previously approved these changes Oct 11, 2024

View reviewed changes

LiShuMing previously approved these changes Oct 12, 2024

View reviewed changes

nshangyiming reviewed Oct 12, 2024

View reviewed changes

add test

a31931c

Signed-off-by: Murphy <[email protected]>

murphyatwork dismissed stale reviews from LiShuMing, kangkaisen, and satanson via a31931c October 14, 2024 07:31

github-actions bot added the behavior_changed label Oct 14, 2024

kangkaisen approved these changes Oct 14, 2024

View reviewed changes

satanson approved these changes Oct 14, 2024

View reviewed changes

nshangyiming approved these changes Oct 14, 2024

View reviewed changes

nshangyiming merged commit 0c0ea45 into StarRocks:main Oct 14, 2024
54 checks passed

github-actions bot removed the 3.3 label Oct 14, 2024

github-actions bot removed the 3.2 label Oct 14, 2024

mergify bot mentioned this pull request Oct 14, 2024

[Enhancement] auto change replication_num of system tables (backport #51799) #51885

Closed

42 tasks

mergify bot mentioned this pull request Oct 14, 2024

[Enhancement] auto change replication_num of system tables (backport #51799) #51886

Merged

42 tasks

ZiheLiu pushed a commit to ZiheLiu/starrocks that referenced this pull request Oct 31, 2024

[Enhancement] auto change replication_num of system tables (StarRocks…

b0c3747

…#51799) Signed-off-by: Murphy <[email protected]>

renzhimin7 pushed a commit to renzhimin7/starrocks that referenced this pull request Nov 7, 2024

[Enhancement] auto change replication_num of system tables (StarRocks…

4fc07d7

…#51799) Signed-off-by: Murphy <[email protected]> Signed-off-by: zhiminr.ren <[email protected]>

murphyatwork mentioned this pull request Nov 9, 2024

[Enhancement] auto change replication_num of system tables (backport #51799) #52766

Merged

60 tasks

github-actions bot added the 3.2-merged label Nov 9, 2024

murphyatwork added a commit that referenced this pull request Nov 11, 2024

[Enhancement] auto change replication_num of system tables (#51799)

49dde9b

Signed-off-by: Murphy <[email protected]> (cherry picked from commit 0c0ea45) Signed-off-by: Murphy <[email protected]>

wanpengfei-git pushed a commit that referenced this pull request Nov 11, 2024

[Enhancement] auto change replication_num of system tables (backport #…

d48afa3

…51799) (#52766) Signed-off-by: Murphy <[email protected]>

github-actions bot added the 3.3-merged label Nov 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Enhancement] auto change replication_num of system tables #51799

[Enhancement] auto change replication_num of system tables #51799

murphyatwork commented Oct 11, 2024 •

edited

Loading

nshangyiming Oct 12, 2024

murphyatwork Oct 14, 2024

sonarcloud bot commented Oct 14, 2024

github-actions bot commented Oct 14, 2024

github-actions bot commented Oct 14, 2024

github-actions bot commented Oct 14, 2024

github-actions bot commented Oct 14, 2024

github-actions bot commented Oct 14, 2024

mergify bot commented Oct 14, 2024 •

edited

Loading

mergify bot commented Oct 14, 2024 •

edited

Loading

[Enhancement] auto change replication_num of system tables #51799

[Enhancement] auto change replication_num of system tables #51799

Conversation

murphyatwork commented Oct 11, 2024 • edited Loading

Why I'm doing:

What I'm doing:

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

nshangyiming Oct 12, 2024

Choose a reason for hiding this comment

murphyatwork Oct 14, 2024

Choose a reason for hiding this comment

sonarcloud bot commented Oct 14, 2024

Quality Gate passed

github-actions bot commented Oct 14, 2024

[Java-Extensions Incremental Coverage Report]

github-actions bot commented Oct 14, 2024

[FE Incremental Coverage Report]

file detail

github-actions bot commented Oct 14, 2024

[BE Incremental Coverage Report]

github-actions bot commented Oct 14, 2024

github-actions bot commented Oct 14, 2024

mergify bot commented Oct 14, 2024 • edited Loading

✅ Backports have been created

mergify bot commented Oct 14, 2024 • edited Loading

✅ Backports have been created

murphyatwork commented Oct 11, 2024 •

edited

Loading

mergify bot commented Oct 14, 2024 •

edited

Loading

mergify bot commented Oct 14, 2024 •

edited

Loading