feat: Add support for limiting the number of CPU cores calcualted for parallelisation #129
Conversation
theofidry
changed the title
|
|
||
| public function test_it_can_describe_itself(): void | ||
| { | ||
| $finder = new EnvVariableFinder('CI_CPU_LIMIT'); |
There was a problem hiding this comment.
How CI_CPU_LIMIT relates to KUBERNETES_CPU_LIMIT? Is this refactor leftover?
There was a problem hiding this comment.
It doesn't directly relate, it's just a generic finder I've made available which tries to get back a positive int from the environment variable. So it can be (and is) used for KUBERNETES_CPU_LIMIT or any other.
| "CastInt": { | ||
| "ignore": [ | ||
| // This is a bug or case handled by strict types. Not sure why | ||
| // infection can't detect it.V |
There was a problem hiding this comment.
| // infection can't detect it.V | |
| // Infection can't detect it. |
| @@ -113,4 +126,11 @@ public function getFinderAndCores(): array | |||
|
|
|||
| throw NumberOfCpuCoreNotFound::create(); | |||
| } | |||
|
|
|||
| public static function getKubernetesLimit(): ?int | |||
There was a problem hiding this comment.
Is Kubernetes the only environment with this issue? Maybe other infra orchestrators also apply here, so maybe more generic name like getLogicalCpuLimit() or getInfrastructureCpuLimit() would be better?
Anyway, I've just entered debug mode on our CI runner in Gitlab (which is run on Kubernetes) and I don't see KUBERNETES_CPU_LIMIT there. How is it supposed to work in practice? I don't think setting this manually is good, as we have runner tiers and these have different CPU limits. It should be somehow detected automatically what is the actual CPU limit inside container.
There was a problem hiding this comment.
FYI: being inside container on k8s it's possible to do cat /sys/fs/cgroup/cpu.max and you get something like 400000 100000, where the first value is a quota, the second is period. Basically this is alternative way of retrieving data described here. Actual available CPUs can be calculated by dividing quota / period. Just tested it again on our cluster and it works, but cgroup must be available in order to provide this data.
There was a problem hiding this comment.
Is Kubernetes the only environment with this issue?
so far yes: I didn't see any other, and worst case you can always pass the $limit
parameter.
so maybe more generic name like getLogicalCpuLimit() or getInfrastructureCpuLimit() would be better?
I will introduce such a method if there is more coming, meanwhile the current one is strictly about kubernetes. The new one would likely leverage the existing kubernetes one and potentially more I guess.
Anyway, I've just entered debug mode on our CI runner in Gitlab (which is run on Kubernetes) and I don't see KUBERNETES_CPU_LIMIT there. How is it supposed to work in practice?
You tell me 😅 . I've never used kubernetes... so I don't know. It was my understanding from the reported issue that it was available as an environment variable as per the following quote:
it is actually overridden and limited to 2 in gitlab CI via the KUBERNETES_CPU_LIMIT environment variable
Closes #125