The VM currently starts a large amount of compiler threads on systems with many CPUs regardless of the code cache size. They consume memory even if they are idle almost all of the time.
This doesn't make sense for very small code cache sizes. A simple approach would be to limit the number of threads dependent on the code cache size.
A more complex approach is to allocate the compiler threads on demand depending on the compile queue lengths.
New implementation: -XX:+UseDynamicNumberOfCompilerThreads, active by default
It only starts 1 compiler thread per type (C1 and C2/Graal) at startup and the value determined by CICompilerCount is only used as an upper limit for compiler threads.
Additional threads get started depending on the compile queue sizes and the available memory. C2/Graal threads get started up to a number of half of their compile queue length. The start gets prevented when the operating system reports that the available memory is less than 200MB per C2/Graal compiler thread.
Similarly, C1 threads get started up to a number of one fourth of their compile queue length and the start gets prevented when the available memory is less than 100MB per C1 compiler thread.
Idle compiler threads die after some time (500 ms for C1, 100 ms for C2/Graal) in reverse order of their creation. One thread of each type is kept alive. It's also possible to keep all started ones alive by using the diagnostic flag -XX:-ReduceNumberOfCompilerThreads.
CompilerThread creation and death can be traced by activating the diagnostic flag -XX:+TraceCompilerThreads.