JDK-8140793 : getAvailableProcessors may incorrectly report the number of cpus in Docker container
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 8,9
  • Priority: P4
  • Status: Resolved
  • Resolution: Duplicate
  • Submitted: 2015-10-29
  • Updated: 2024-06-04
  • Resolved: 2015-12-23
Related Reports
Duplicate :  
JDK-6515172 - Runtime.availableProcessors() ignores Linux taskset command
Relates :  
JDK-8146115 - Improve docker container detection and resource configuration usage
Description
OperatingSystemMXBean.getAvailableProcessors should return the number of processors made available to the process.  Docker can place limits on it.
Mis-reporting the number can throw off heuristics for optimizing computations based on memory size and available processors.  
It is hinted that the number of processors reported from /proc on linux reports the hardware processors not those permitted to the process.

sysconf(_SC_NPROCESSORS_CONF) may be more reliable in this case.
Comments
Closed as a duplicate; assuming JDK-6515172 is fixed in 9. +1 on the backport to 8u.
23-12-2015
Looks like this is covered in JDK-6515172 which is a P3 which is a more appropriate priority here. The number of GC threads gets determined based on processors. This should also be back ported to JDK8uX. Many customers, including our upper stack will be running on JDK8 in Docker environments.
23-12-2015
If this can be fixed as in 6515172, then that's a plus. But it should be fixed in 9 and its priority changed to at least a 4. It is not low impact issue since many applications configure their work loads based on an accurate count of the number of processors.
14-12-2015
Okay I've paged in my memory from 2010. If Docker is using cgroups then we can use sched_getaffinity. This request is a duplicate of JDK-6515172. I suggest closing this, targetting JDK-6515172 to 9 and assigning it to me. Do we have any way to test this under Docker?
14-12-2015
There is mention of the nproc command. It seems to call a library function num_processors(). On Ubuntu, the nproc command calls sched_getaffinity(). There is a CPU_COUNT macro which counts the number of cpus in the affinity set.
11-12-2015
@Dan: Determining active CPU count is a runtime issue AFAIK - one I've worked on a bit in the past :) @Roger: cgroups seem similar to Solaris resource pools (which we also do not handle), and after looking at the documentation: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt it isn't at all clear to me that: (a) there is an efficient API we can query to determine the number of CPUS; and (b) that the number of CPUs is a meaningful value if the CPU shares facility is being used. Both the number of physical processors available to a cgroup and the share of CPU time available to a process in the cgroup may vary dynamically. And in addition to cgroups simple cpusets may also be being used. Someone familiar with Docker and cgroups and the various ways in which CPU limitations can be imposed, needs to define the policies by which we use the available information to report a meaningful value for "available processors". Reading and parsing /proc/<pid>/cgroup does not seem like a very efficient way to have to implement this. It is a pity that cgroups do not seem to be integrated into the sysconf facility such that :sysconf(_SC_NPROCESSORS_ONLN) "just works".
02-11-2015
Docker uses cgroups but more investigation is needed to determine what really limits the cpus. The VM internal use of the number of cpus should be examined also since it has ergonomics that depend on the number of cpus. So it may not be just a serviceability issue.
30-10-2015
Does this belong in hotspot/svc? The HotSpot glue that supports OperatingSystemMXBean is typically supported by the Serviceability team...
30-10-2015
The number of active processors reported on linux is not taken from /proc information but from sysconf: int online_cpus = ::sysconf(_SC_NPROCESSORS_ONLN); I'm not familiar with Docker - how does it limit the available processors? Does it employ cpusets?
30-10-2015