Skip to content

Commit

Permalink
Improve detection of CPU limits when running inside a Container
Browse files Browse the repository at this point in the history
This focuses on better supporting Docker CLI's parameter `--cpus`, which limits the amount of CPU time available to the container (ex: 1.8 means 180% CPU time, ie on 2 cores 90% for each core, on 4 cores 45% on each core, etc.)

All the runtime components depending on the number of processors available are:
 - ThreadPool
 - GC
 - `Environment.ProcessorCount` via `SystemNative::GetProcessorCount`
 - `SimpleRWLock::m_spinCount`
 - `BaseDomain::m_iNumberOfProcessors` (it's used to determine the GC heap to affinitize to)

All the above components take advantage of `--cpus` via `CGroup::GetCpuLimit` with dotnet#12797, allowing to optimize performance in a container/machine with limited resources. This makes sure the runtime components makes the best use of available resources.

In the case of `Environment.ProcessorCount`, the behavior is such that passing `--cpus=1.5` on a machine with 8 processors will return `1`  as shown in https://github.com/dotnet/coreclr/issues/22302#issuecomment-459092299. This behavior is not consistent with [Windows Job Objects](https://docs.microsoft.com/en-us/windows/desktop/api/winnt/ns-winnt-jobobject_cpu_rate_control_information) which still returns the number of processors for the container/machine even if it only gets parts of the total number of cycles.

This behavior is erroneous because the container still has access to the full range of processors on the machine, and only its _processor time_ is limited. For example, in the case of a 4 processors machine, with a value of `--cpus=1.8`, there can be 4 threads running in parallel even though each thread will only get `1.8 / 8 = .45` or 45% of all cycles of each processor.

The work consist in reverting the behavior of `SystemNative::GetProcessorCount` to pre dotnet#12797.
  • Loading branch information
luhenry committed Apr 5, 2019
1 parent 98d2b48 commit 32bbe0e
Show file tree
Hide file tree
Showing 7 changed files with 10 additions and 9 deletions.
2 changes: 1 addition & 1 deletion src/gc/env/gcenv.os.h
Original file line number Diff line number Diff line change
Expand Up @@ -372,7 +372,7 @@ class GCToOSInterface
// Get number of processors assigned to the current process
// Return:
// The number of processors
static uint32_t GetCurrentProcessCpuCount();
static uint32_t GetCurrentProcessCpuCount(bool withCpuLimit = false);

// Sets the calling thread's affinity to only run on the processor specified.
// Parameters:
Expand Down
2 changes: 1 addition & 1 deletion src/gc/gc.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -34103,7 +34103,7 @@ HRESULT GCHeap::Initialize()

nhp_from_config = static_cast<uint32_t>(GCConfig::GetHeapCount());

uint32_t nhp_from_process = GCToOSInterface::GetCurrentProcessCpuCount();
uint32_t nhp_from_process = GCToOSInterface::GetCurrentProcessCpuCount(true);

if (nhp_from_config)
{
Expand Down
2 changes: 1 addition & 1 deletion src/gc/windows/gcenv.windows.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1011,7 +1011,7 @@ const AffinitySet* GCToOSInterface::SetGCThreadsAffinitySet(uintptr_t configAffi
// Get number of processors assigned to the current process
// Return:
// The number of processors
uint32_t GCToOSInterface::GetCurrentProcessCpuCount()
uint32_t GCToOSInterface::GetCurrentProcessCpuCount(bool withCpuLimit)
{
static int cCPUs = 0;

Expand Down
2 changes: 1 addition & 1 deletion src/inc/utilcode.h
Original file line number Diff line number Diff line change
Expand Up @@ -1413,7 +1413,7 @@ class CPUGroupInfo
}
};

int GetCurrentProcessCpuCount();
int GetCurrentProcessCpuCount(bool withCpuLimit = false);
DWORD_PTR GetCurrentProcessCpuMask();

uint32_t GetOsPageSize();
Expand Down
4 changes: 2 additions & 2 deletions src/utilcode/util.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1191,7 +1191,7 @@ BOOL CPUGroupInfo::GetCPUGroupRange(WORD group_number, WORD* group_begin, WORD*
//******************************************************************************
// Returns the number of processors that a process has been configured to run on
//******************************************************************************
int GetCurrentProcessCpuCount()
int GetCurrentProcessCpuCount(bool withCpuLimit)
{
CONTRACTL
{
Expand Down Expand Up @@ -1236,7 +1236,7 @@ int GetCurrentProcessCpuCount()
#ifdef FEATURE_PAL
uint32_t cpuLimit;

if (PAL_GetCpuLimit(&cpuLimit) && cpuLimit < count)
if (withCpuLimit && PAL_GetCpuLimit(&cpuLimit) && cpuLimit < count)
count = cpuLimit;
#endif

Expand Down
4 changes: 2 additions & 2 deletions src/vm/gcenv.os.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -476,14 +476,14 @@ const AffinitySet* GCToOSInterface::SetGCThreadsAffinitySet(uintptr_t configAffi
// Get number of processors assigned to the current process
// Return:
// The number of processors
uint32_t GCToOSInterface::GetCurrentProcessCpuCount()
uint32_t GCToOSInterface::GetCurrentProcessCpuCount(bool withCpuLimit)
{
LIMITED_METHOD_CONTRACT;

// GetCurrentProcessCpuCount only returns up to 64 procs.
return CPUGroupInfo::CanEnableGCCPUGroups() ?
GCToOSInterface::GetTotalProcessorCount():
::GetCurrentProcessCpuCount();
::GetCurrentProcessCpuCount(withCpuLimit);
}

// Return the size of the user-mode portion of the virtual address space of this process.
Expand Down
3 changes: 2 additions & 1 deletion src/vm/simplerwlock.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,8 @@ class SimpleRWLock
} CONTRACTL_END;

m_RWLock = 0;
m_spinCount = (GetCurrentProcessCpuCount() == 1) ? 0 : 4000;
// Passing false here reduces ASP.NET Core Plaintext benchmark results from 1.2M to 0.8M RPS.
m_spinCount = (GetCurrentProcessCpuCount(true) == 1) ? 0 : 4000;
m_WriterWaiting = FALSE;

#ifdef _DEBUG
Expand Down

0 comments on commit 32bbe0e

Please sign in to comment.