Skip to content

Commit

Permalink
Improve detection of CPU limits when running inside a Container
Browse files Browse the repository at this point in the history
This focuses on better supporting Docker CLI's parameter `--cpus`, which limits the amount of CPU time available to the container (ex: 1.8 means 180% CPU time, ie on 2 cores 90% for each core, on 4 cores 45% on each core, etc.)

All the runtime components depending on the number of processors available are:
 - ThreadPool
 - GC
 - `Environment.ProcessorCount` via `SystemNative::GetProcessorCount`
 - `SimpleRWLock::m_spinCount`
 - `BaseDomain::m_iNumberOfProcessors` (it's used to determine the GC heap to affinitize to)

All the above components take advantage of `--cpus` via `CGroup::GetCpuLimit` with dotnet#12797, allowing to optimize performance in a container/machine with limited resources. This makes sure the runtime components makes the best use of available resources.

In the case of `Environment.ProcessorCount`, the behavior is such that passing `--cpus=1.5` on a machine with 8 processors will return `1`  as shown in https://github.com/dotnet/coreclr/issues/22302#issuecomment-459092299. This behavior is not consistent with [Windows Job Objects](https://docs.microsoft.com/en-us/windows/desktop/api/winnt/ns-winnt-jobobject_cpu_rate_control_information) which still returns the number of processors for the container/machine even if it only gets parts of the total number of cycles.

This behavior is erroneous because the container still has access to the full range of processors on the machine, and only its _processor time_ is limited. For example, in the case of a 4 processors machine, with a value of `--cpus=1.8`, there can be 4 threads running in parallel even though each thread will only get `1.8 / 8 = .45` or 45% of all cycles of each processor.

The work consist in reverting the behavior of `SystemNative::GetProcessorCount` to pre dotnet#12797.
  • Loading branch information
luhenry committed Mar 29, 2019
1 parent e6c49f7 commit 8f206df
Show file tree
Hide file tree
Showing 6 changed files with 26 additions and 22 deletions.
7 changes: 0 additions & 7 deletions src/classlibnative/bcltype/system.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -343,13 +343,6 @@ INT32 QCALLTYPE SystemNative::GetProcessorCount()
processorCount = systemInfo.dwNumberOfProcessors;
}

#ifdef FEATURE_PAL
uint32_t cpuLimit;

if (PAL_GetCpuLimit(&cpuLimit) && cpuLimit < (uint32_t)processorCount)
processorCount = cpuLimit;
#endif

END_QCALL;

return processorCount;
Expand Down
7 changes: 4 additions & 3 deletions src/gc/unix/cgroup.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ class CGroup
{
long long quota;
long long period;
long long cpu_count;
double cpu_count;

quota = ReadCpuCGroupValue(CFS_QUOTA_FILENAME);
if (quota <= 0)
Expand All @@ -119,10 +119,11 @@ class CGroup
return true;
}

cpu_count = quota / period;
cpu_count = (double) quota / period;
if (cpu_count < UINT32_MAX)
{
*val = cpu_count;
// round up
*val = (uint32_t)(cpu_count + 0.999999999);
}
else
{
Expand Down
9 changes: 5 additions & 4 deletions src/pal/src/misc/cgroup.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ class CGroup
{
long long quota;
long long period;
long long cpu_count;
double cpu_count;

quota = ReadCpuCGroupValue(CFS_QUOTA_FILENAME);
if (quota <= 0)
Expand All @@ -106,11 +106,12 @@ class CGroup
*val = 1;
return true;
}
cpu_count = quota / period;

cpu_count = (double) quota / period;
if (cpu_count < UINT_MAX)
{
*val = cpu_count;
// round up
*val = (UINT)(cpu_count + 0.999999999);
}
else
{
Expand Down
6 changes: 6 additions & 0 deletions src/pal/src/thread/process.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2524,6 +2524,12 @@ PAL_GetCPUBusyTime(
{
return 0;
}

UINT cpuLimit;
if (PAL_GetCpuLimit(&cpuLimit) && cpuLimit < dwNumberOfProcessors)
{
dwNumberOfProcessors = cpuLimit;
}
}

if (getrusage(RUSAGE_SELF, &resUsage) == -1)
Expand Down
7 changes: 0 additions & 7 deletions src/utilcode/util.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1219,13 +1219,6 @@ int GetCurrentProcessCpuCount()
count = 64;
}

#ifdef FEATURE_PAL
uint32_t cpuLimit;

if (PAL_GetCpuLimit(&cpuLimit) && cpuLimit < count)
count = cpuLimit;
#endif

cCPUs = count;

return count;
Expand Down
12 changes: 11 additions & 1 deletion src/vm/gcenv.os.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -400,7 +400,17 @@ uint32_t GCToOSInterface::GetCurrentProcessCpuCount()
{
LIMITED_METHOD_CONTRACT;

return ::GetCurrentProcessCpuCount();
uint32_t cpuCount = ::GetCurrentProcessCpuCount();

#ifdef FEATURE_PAL
uint32_t cpuLimit;
if (PAL_GetCpuLimit(&cpuLimit) && cpuLimit < cpuCount)
{
cpuCount = cpuLimit;
}
#endif

return cpuCount;
}

// Return the size of the user-mode portion of the virtual address space of this process.
Expand Down

0 comments on commit 8f206df

Please sign in to comment.