In 6739920, networking change to address the specific symptom is in progress.  However, we need JVM ObjectMonitor management code change too so that customers can avoid or work-around similar situations.

Problem from the original CR:
On RHEL 4.6 64-bit 4G mem, two instances of jakarta-jmeter with 6000 threads worked fine on 6u3 64-bit.
But after upgrading to 6u6, java processes soon cause memory thrashing and become very slow.

By monitoring virtual and resident size of java processes, 6u4 and later use larger memory.  In a case,
6u7 uses 600MB or larger C heap.

By checking malloc related path on gdb, it turned out the most memory allocations come from 
ObjectMonitor allocation.

On 6u4, per-thread ObjectMonitor freelist was introdued (_omFreeList).  In some test, I saw more than
800 ObjectMonitors in average on 6u7.  ObjectMonitor size is 168 bytes on 64-bit JVM for Linux.

6000*800*168=806400000

6u3 can run the same app with 100m to 200m C heap, but 6u4 and later require much larger C heap.

C heap usage levels off at some point, but we cannot know how much memory is really needed to
a specific run of application.  Currently there is no clear limit for the ObjectMonitors on
omFreeList.  

To suppress the excessive C heap use, we need a switch to turn off the per-thread ObjectMonitor
free list or a good mechanism to control the memory usage (Or simply a better implementation).
Switching off the per-thread object monitor caches is relatively easy and only involves a few lines of code, but a quick test with the offending application supports my claim above that there's no benefit to this path.  Only a tiny fraction of the extant objectmnitors are sequestered in the per-thread caches.  Adding the switch is a fool's errand and provides no relief from the root problem, which has existed since day one.