Current ByteBuffer.allocateDirect() code uses sun.misc.Cleaner to deallocate the native memory. Cleaner is being invoked by the ReferenceHandler thread, which is problematic for several reasons. First, there is no feedback between allocation and deallocation of DirectByteBuffers. That means that with plenty allocators we overrun the cleanup threads, and catch fire (OOME). Second, ReferenceHandler is the single thread who's job is to enqueue the references on their relevant queues. If there are many references currently in flight in the VM, Cleaners are getting penalized in their cleanups.
In the end, that means the simple loop:
while(true) ByteBuffer.allocateDirect(100);
...can put all of the direct buffer users down on the knees, and severely handicap JRE performance.
We should be more aggressive in DIrectByteBuffer cleanups.