JDK-7021979 : rapid sustained monitor circulation causes asymptotic increase in # of extant monitors
  • Type: Enhancement
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 7
  • Priority: P4
  • Status: Resolved
  • Resolution: Duplicate
  • OS: generic
  • CPU: generic
  • Submitted: 2011-02-24
  • Updated: 2020-11-11
  • Resolved: 2020-11-11
Related Reports
Duplicate :  
Relates :  
Relates :  
Description
I've attached a test case, Inflation.java, that shows a growth in the number of extant monitors asympotitically up to T * MAXPRIVATE where T is the number of live threads and MAXPRIVATE is a constant defined in synchronizer.cpp, currently 1024.   Instructions for running the program are in comments at the top of Inflation.java.   More details about the pathology are comments in that file.   

An extremely simple fix that I tested is to :

(a) add a new Thread.omFreeEpoch field, initialized to 0 in the thread ctor

(b) in synchronizer.cpp omAlloc(), we change 

  Self->omFreeProvision += 1 + (Self->omFreeProvision/2) ;

to
  int Epoch = GVars.stwCycle ;
  if (Self->omFreeEpoch == Epoch) {
    Self->omFreeProvision += 1 + (Self->omFreeProvision/2) ;
    if (Self->omFreeProvision > MAXPRIVATE ) Self->omFreeProvision = MAXPRIVATE ;
  } else {
    // Avoid the scenario where we can have asymptotic growth
    // of the number of monitors up to T * MAXPRIVATE where T is the 
    // number of extant threads.  This can occur if threads
    // were to loop, inflating, and then sleeping, and during the sleep period
    // the monitor was idle and GC activity cause safepoints.
    // In that case monitors rapidly circulate from the global free list to local lists,
    // to then be inflated, to then be default in deflate_idle_monitors and then
    // to return to the global free list.  Such circulation tends to cause
    // omFreeProvision to creep up to MAXPRIVATE, which in turn causes more 
    // monitors to be created as threads draw larger segments from the global list.
    // Either decay (halve) or reset omFreeProvision.  
    // In addition, we might consider an enhancement to trim or cull the 
    // thread-local free lists at STW-time.  
    Self->omFreeProvision = 16 ;
  }
  Self->omFreeEpoch = Epoch ;

-Dave

Comments
JDK-8253064 has been integrated. Closing this bug as a duplicate. [~redestad] and [~rehn] - thanks for confirming.
11-11-2020

Sure
11-11-2020

Thanks [~redestad]!
05-11-2020

Seems to me that JDK-8253064 (which looks great - just skimmed it) should be a completely satisfactory solution to this issue, yes.
05-11-2020

Erik and Dan's work on this bug: JDK-8253064 monitor list simplifications and getting rid of TSM will delete the per-thread in-use and free lists so this bug can probably be closed as a duplicate if/when JDK-8253064 gets integrated. [~rehn] and [~redestad] - what do you folks think?
05-11-2020

Concurrent and more eager cleanup is great, and while the latency issue is thus diminished that is somewhat orthogonal to the issue where every thread in an app holds onto up to MAXPRIVATE number of monitors, pushing the total population of monitors up. I've also not seen any evidence that a MAXPRIVATE value larger than ~8 brings any measurable reduction in coherency traffic on modern (dual socket) Intel systems, thus I think reducing the max number of monitors a thread will provision drastically (some value between 8 and 128) might be a sufficient solution to the footprint concern.
28-02-2018

I didn't see any issue in JDK 11 - tip, I suspect it's due to now we trigger a cleanup safepoint if monitors number are 'big'. We are also about to start looking into concurrent deflation.
28-02-2018

This is not simple a footprint problem, but the cause of significant latency during safepointing. Even simpler programs can cause the monitor population to rise over 250k (over extended time periods), e.g: public class Inflatey { public static void inflatey() { try { String lockyLocky = new String("I'm going to be a lock when I grow up !"); synchronized (lockyLocky) { lockyLocky.wait(0, 1); } } catch (InterruptedException intExc) {} } public static void main(String[] args) { while (true) { inflatey(); } } } At 250k monitors, deflate_idle_monitors() during STW in SafepointSynchronize::begin() starts taking around 8ms on our "perf aurora" Linux x64 servers.
18-09-2015

While variants of Dice's patch have a marked impact on monitor population on some benchmarks, there are some question marks on choice of "tick" to use for backing off monitor provisioning value. Using the GC epoch might be too coarse or too fine grained depending on application behavior, and needs more analysis. For JDK9 an appropriate low-risk improvement could be to make the MAX_PRIVATE and the implicit initial constant (16) tunable using flags and/or reduce their default values. Either of these changes would cooperate well with the existing ways to limit monitor population growth (-XX:MonitorBound in particular) since existing options have no current means of freeing up provisioned monitors (leading to starvation and excessive safepointing), while on a sample of hardware it's not been possible to show a significant degradation in major benchmark throughput unless reducing MAX_PRIVATE to near-0 values.
21-08-2015

We don't currently have a real world scenario where this is a problem, closing as WNF.
12-02-2015