[FG:InPlacePodVerticalScaling] A proposal for CPU allocated strategy when Pod scale up and down

[Chunxia202410]

What type of PR is this?

/kind cleanup

This PR is based on the kubernetes#123319.

What this PR does / why we need it:

When Guaranteed QoS Class Pod scale up and down with Static CPU management policy alongside InPlacePodVerticalScaling, the allocated CPU should meet the kubernetes CPUs allocation rules.

For scaling up, keep the original CPUs, and only allocate the additional number of CPUs. when allocate additional CPUs, it is necessary to ensure that the combination of the original CPUs and the additional CPUs complies with the Kubernetes CPU allocation rules.
For takeByTopologyNUMAPacked：
For example:
Assuming the CPU topology is as follows, and kubernetes CPUs allocation rules is Packed(takeByTopologyNUMAPacked):

Allocated CPUs of Pod0 is {2,3}, Allocated CPUs of Pod1 is {13}. Other CPUs are free.
-When scale up of Pod 1 from 1 CPU to 2 CPUs. ==> CPU {12} should to be allocated to Pod1, so CPUset of Pod1 is {12,13}
-When scale up of Pod 1 from 1 CPU to 4 CPUs. ==> CPU {12,14,15} should to be allocated to Pod1, so CPUset of Pod1 is {12,13,14,15}
-When scale up of Pod 1 from 1 CPU to 8 CPUs. ==> CPU {8-12,14,15} should to be allocated to Pod1, so CPUset of Pod1 is {8-15}

For scaling down, if there are mustKeepCPUsForResize, when allocate remained CPUs, it is necessary to ensure that the combination of the mustKeepCPUsForResize CPUs and the allocated remained CPUs complies with the Kubernetes CPU allocation rules.

Which issue(s) this PR fixes:

The step as follows for allocate additional CPUs when scale up with CPU allocated packet(takeByTopologyNUMAPacked) rule, and assume that the NUMA nodes are higher than Sockets in the memory hierarchy.

1. Keep the original CPUs.
2. Take remain CPUs in the numa which only allocated CPUs to this Pod. (newly add step)
3. Take full numa nodes.
4. Take remain CPUs in the socket which only allocated CPUs to this Pod, than take full sockets in numa nodes which allocated CPUs to this Pod. (newly add step)
5. Take full sockets.
6. Take remain CPUs in the physical cores which only allocated CPUs to this Pod, than take full cores in sockets which allocated CPUs to this Pod, than take remain full cores in numa which allocated CPUs to this Pod. (newly add step)
7. Take full physical cores.
8. Take remain CPUs in the physical cores which only allocated CPUs to this Pod, than take remain CPUs in sockets which allocated CPUs to this Pod, than take remain CPUs in numa which allocated CPUs to this Pod. (newly add step)
9. Take remain CPUs

For takeByTopologyNUMADistributed：
The allocated should be a subset of the combo. and the number of allocated CPU is not more than distribute number.

Special notes for your reviewer:
UncoreCaches need to be considerd

Does this PR introduce a user-facing change?

NONE

[esotsal]

Thanks @Chunxia202410 for the proposal, can you please update PR using kubernetes#129719 as base ? Thanks

[Chunxia202410]

Yes~

[Chunxia202410]

This PR is replaced by esotsal#3

[esotsal]

Thanks !