The SuspendibleThreadSet class currently lives in concurrentGCThread.hpp and has a somewhat strange relationship with ConcurrentGCThread. Even if the ConcurrentGCThread class, its sub-classes, are the main users of the SuspendibleThreadSet there are still other non-ConcurrentGCThread users (like by SharedHeap's FlexibleWorkGang returned by workers()).

ConcurrentGCThread owns the single instance of SuspendibleThreadSet and it's made accessible to its sub-classes via the _sts member. However, to allow other threads to use this all functions on the _sts instance are also exposed through a static API on the ConcurrentGCThread (the stsJoin(), stsLeave(), etc). So there are two APIs exposing for the same functions.

I suggest we break out and detach SuspendibleThreadSet from ConcurrentGCThread to make the APIs cleaner. I also suggest we add a "SuspendibleThreadSetJoiner" class (with similar semantics as a MutexLocker, with the difference that it joins/leaves the set), since the following pattern is fairly common:

if (....) {
   _sts.join();
   <do something>
   _sts.leave();
}

which would the instead read:

if (....) {
   SuspendibleThreadSetJoiner sts;
   <do something>
}