Skip to content

Latest commit

 

History

History
38 lines (24 loc) · 2.34 KB

pip-358.md

File metadata and controls

38 lines (24 loc) · 2.34 KB
Raw

PIP-358: let resource weight work for OverloadShedder, LeastLongTermMessageRate, ModularLoadManagerImpl.

Background knowledge

Initially, we introduce loadBalancerCPUResourceWeight, loadBalancerBandwidthInResourceWeight, loadBalancerBandwidthOutResourceWeight, loadBalancerMemoryResourceWeight, loadBalancerDirectMemoryResourceWeight in ThresholdShedder to control the resource weight for different resources when calculating the load of the broker. Then we let it work for LeastResourceUsageWithWeight for better bundle placement policy.

But #19559 and #21168 have pointed out that the actual load of the broker is not related to the memory usage and direct memory usage, thus we have changed the default value of loadBalancerMemoryResourceWeight, loadBalancerDirectMemoryResourceWeight to 0.0.

There are still some places where memory usage and direct memory usage are used to calculate the load of the broker, such as OverloadShedder, LeastLongTermMessageRate, ModularLoadManagerImpl. We should let the resource weight work for these places so that we can set the resource weight to 0.0 to avoid the impact of memory usage and direct memory usage on the load of the broker.

Motivation

The actual load of the broker is not related to the memory usage and direct memory usage, thus we should let the resource weight work for OverloadShedder, LeastLongTermMessageRate, ModularLoadManagerImpl so that we can set the resource weight to 0.0 to avoid the impact of memory usage and direct memory usage on the load of the broker.

Detailed Design

Let resource weight work for OverloadShedder, LeastLongTermMessageRate, ModularLoadManagerImpl.

  • For OverloadShedder, LeastLongTermMessageRate, we replace getMaxResourceUsage() with getMaxResourceUsageWithWeight() in the calculation of the load of the broker.
  • For ModularLoadManagerImpl, we replace getMaxResourceUsage() with getMaxResourceUsageWithWeight() when checking if the broker is overloaded and decide whether to update the broker data to metadata store.

Backward & Forward Compatibility

Links