[Enhancement] Add some logs to help online businesses quickly find the queries that cause FE oom

[before-Sunrise]

Why I'm doing:

What I'm doing:

Fixes #issue

What type of PR is this:

• BugFix
• Feature
• Enhancement
• Refactor
• UT
• Doc
• Tool

Does this PR entail a change in behavior?

• Yes, this PR will result in a change in behavior.
• No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• Interface/UI changes: syntax, type conversion, expression evaluation, display information
• Parameter changes: default values, similar parameters but with different default values
• Policy changes: use new policy to replace old one, functionality automatically enabled
• Feature removed
• Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• I have added test cases for my bug fix or my new feature
• This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function
• This is a backport pr

Bugfix cherry-pick branch check:

• I have checked the version labels which the pr will be auto-backported to the target branch
  • 3.3
  • 3.2
  • 3.1
  • 3.0
  • 2.5

[sonarcloud]

Quality Gate passed

Issues
12 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[starrocks-cr]

The most risky bug in this code is:
A potential integer overflow when calculating totalScanRangeBytes, which could lead to incorrect memory checks and potential OOM issues.

You can modify the code like this:

@@ -188,6 +189,11 @@ public class OlapScanNode extends ScanNode {
 
     private VectorSearchOptions vectorSearchOptions = new VectorSearchOptions();
 
+    private boolean calculatedScanRange = false;
+
+    private long totalScanRangeBytes = 0;
+
+    // Constructs node to scan given data files of table 'tbl'.
     // Constructs node to scan given data files of table 'tbl'.
     public OlapScanNode(PlanNodeId id, TupleDescriptor desc, String planNodeName) {
         super(id, desc, planNodeName);
@@ -641,6 +647,11 @@ public void addScanRangeLocations(Partition partition,
             scanRangeLocations.setScan_range(scanRange);
 
             bucketSeq2locations.put(tabletId2BucketSeq.get(tabletId), scanRangeLocations);
+            if (!calculatedScanRange && scanRangeLocations != null) {
+                long scanRangeSize = SizeEstimator.estimate(scanRangeLocations);
+                checkIfScanRangeNumSafe(scanRangeSize);
+                calculatedScanRange = true;
+            }
 
             result.add(scanRangeLocations);
         }
@@ -731,6 +742,32 @@ private void computeTabletInfo() throws UserException {
         }
     }
 
+    public void checkIfScanRangeNumSafe(long scanRangeSize) {
+        long totalPartitionNum = 0;
+        long totalTabletsNum = 0;
+        for (long partitionId : selectedPartitionIds) {
+            final Partition partition = olapTable.getPartition(partitionId);
+            List<PhysicalPartition> physicalPartitions = (List<PhysicalPartition>) partition.getSubPartitions();
+            totalPartitionNum += physicalPartitions.size();
+            for (PhysicalPartition physicalPartition : physicalPartitions) {
+                final MaterializedIndex selectedTable = physicalPartition.getIndex(selectedIndexId);
+                totalTabletsNum += selectedTable.getTablets().size();
+            }
+        }
+
+        totalScanRangeBytes = Math.multiplyExact(scanRangeSize, totalTabletsNum); // Prevent overflow
+        if (totalScanRangeBytes > 1024 * 1024) {
+            Runtime runtime = Runtime.getRuntime();
+            long freeMemory = runtime.freeMemory();
+            if (totalScanRangeBytes > freeMemory / 2) {
+                LOG.warn(
+                        "Try to allocate too many scan ranges for table {}, which may cause FE OOM, Partition Num:{}, tablet Num:{}, Scan Range Total Bytes:{}",
+                        olapTable.getName(), totalPartitionNum, totalTabletsNum, totalScanRangeBytes);
+            }
+        }
+
+    }
+
     /**
      * We query meta to get request's data location
      * extra result info will pass to backend ScanNode

[github-actions]

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

[github-actions]

[FE Incremental Coverage Report]

✅ pass : 41 / 51 (80.39%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	com/starrocks/sql/StatementPlanner.java	1	3	33.33%	[155, 156]
🔵	com/starrocks/http/rest/ExecuteSqlAction.java	1	2	50.00%	[134]
🔵	com/starrocks/qe/ConnectProcessor.java	1	2	50.00%	[311]
🔵	com/starrocks/qe/ConnectContext.java	5	7	71.43%	[940, 941]
🔵	com/starrocks/statistic/StatisticsCollectJob.java	4	5	80.00%	[153]
🔵	com/starrocks/statistic/StatisticExecutor.java	5	6	83.33%	[434]
🔵	com/starrocks/planner/OlapScanNode.java	23	25	92.00%	[763, 765]
🔵	com/starrocks/common/Config.java	1	1	100.00%	[]

[github-actions]

[BE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

[github-actions]

@Mergifyio backport branch-3.3

[github-actions]

@Mergifyio backport branch-3.2

[mergify]

backport branch-3.3

✅ Backports have been created

• #51531 [Enhancement] Add some logs to help online businesses quickly find the queries that cause FE oom (backport #51528) has been created for branch branch-3.3 but encountered conflicts

[mergify]

backport branch-3.2

✅ Backports have been created

• #51532 [Enhancement] Add some logs to help online businesses quickly find the queries that cause FE oom (backport #51528) has been created for branch branch-3.2 but encountered conflicts