United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-4855795 : Spurious OutOfMemoryError exceptions

Details
Type:
Bug
Submit Date:
2003-04-29
Status:
Resolved
Updated Date:
2008-11-05
Project Name:
JDK
Resolved Date:
2006-01-31
Component:
hotspot
OS:
windows_2000
Sub-Component:
gc
CPU:
x86
Priority:
P4
Resolution:
Fixed
Affected Versions:
1.4.2
Fixed Versions:

Related Reports
Backport:
Backport:
Relates:
Relates:

Sub Tasks

Description
Name: rmT116609			Date: 04/29/2003


FULL PRODUCT VERSION :
java version "1.4.1_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1_01-b01)
Java HotSpot(TM) Client VM (build 1.4.1_01-b01, mixed mode)

java version "1.4.2-beta"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2-beta-b19)
Java HotSpot(TM) Client VM (build 1.4.2-beta-b19, mixed mode)

FULL OPERATING SYSTEM VERSION :
Microsoft Windows 2000 [Version 5.00.2195]


A DESCRIPTION OF THE PROBLEM :
The HotSpot VM sometimes throws an OutOfMemoryError even
though there is plenty of reclaimable garbage present. It
appears that an allocation failure on an application
request does trigger a full garbage collection, but an
OutOfMemoryError may be thrown regardless of the results of
that GC.

In the process of debugging an out-of-memory exit in a
complex application, I wrote a simple test program that
allocates a series of 4KB blocks (byte arrays). When the
amount of free space on the heap gets small, say <1MB free,
the program begins dropping its references to some of the
blocks & continues to allocate new blocks. Depending on
various heap option settings, an OutOfMemory exception may
be thrown. An example of the output in one of these cases
follows:

C:\heaptest>java -server -verbose:gc -XX:NewRatio=4 -x100m -
Xms100m HeapTest

requested allocation size = 4096  actual size = 4112 bytes
[Full GC 446K->330K(100352K), 0.0104416 secs]
--- starting allocations -------
[GC 16714K->16714K(100352K), 0.0470250 secs]
[GC 33097K->33097K(100352K), 0.0497818 secs]
[GC 49481K->49481K(100352K), 0.0480004 secs]
[GC 65865K->65865K(100352K), 0.0489293 secs]
[Full GC 82249K->82249K(100352K), 0.0753784 secs]
24908: may throw OutOfMemory. free heap is 4632
[Full GC 100348K->98307K(100352K), 0.0306476 secs]
[Full GC 98307K->98307K(100352K), 0.1124120 secs]

24908: *** Out of Memory Exception caught *** (expected
2032K garbage)
 free heap is  2092448 bytes
 total blocks allocated = 24399

The line beginning 24908: is logged indicating that the
free heap space (as returned by Runtime.freeMemory()) is
low enough that the next 4KB allocation attempt may fail.
However, at this point there are roughly 2MB of previously
allocated but now unreferenced blocks in the heap. The next
line shows a full GC, presumably caused by the allocation
failure.
  [Full GC 100348K->98307K(100352K), 0.0306476 secs]
Note that this full GC reclaimed around 2MB, yet the
OutOfMemoryError exception is still thrown. Furthermore,
when the exception is caught, freeMemory() shows that 2MB
are indeed available. Also, immediately retrying the same
allocation now succeeds!

This behavior is so unexpected that I do not see how it
could be considered anything but a bug. I realize that the
VM spec is vague on this subject, but the relevant sentence
is:

"If a computation requires more heap than can be made
available by the automatic storage management system, the
Java virtual machine throws an OutOfMemoryError."

Clearly in the example above, the system COULD have made
sufficient heap available had it tried to do so
synchronously.

Lest you think this is a contrived example, the original
problem causing our application to crash exhibited similar
symptoms. We were able to prevent most of the application
errors by periodically forcing a full GC via System.gc(),
however this is not recommended & should not be necessary.




STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. compile the attached HeapTest.java class
2.java -server -verbose:gc -XX:NewRatio=3 -Xmx100m -Xms100m HeapTest
(different options may or may not reproduce the problem)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
class HeapTest
{
    static final private int KB = 1024;
    static final private int MB = 1024*1024;

    public static void main( String[] args )
    {
        Runtime rt = Runtime.getRuntime();

        java.util.Vector v = new java.util.Vector(50000);


        int MAXITER = 2000000;
        int BLOCKSIZE = 4*KB;
        int actBlockSize = BLOCKSIZE;

        // determine actual space allocated per block including overhead
        long preFree = rt.freeMemory();
        byte[] ba = new byte[BLOCKSIZE];
        long curFree = rt.freeMemory();
        ba = null;
        actBlockSize = (int)(preFree-curFree);
        System.out.println( "requested allocation size = " + BLOCKSIZE
                                + "  actual size = " + actBlockSize + "
bytes" );

        System.gc();
        System.out.println( "--- starting allocations -------" );

        int  gCount = 0;

        for( int i=1; i < MAXITER; i++ ) {
            preFree = rt.freeMemory();

            if( preFree < 2*BLOCKSIZE ) {
                System.out.println( i + ": may cause allocation failure. free
heap is " + preFree );
             //   System.out.println( i + ": forcing GC" );
             //   System.gc();
            }

            try {
                ba = new byte[BLOCKSIZE];
                v.add( ba );
                ba = null;
            }
            catch( java.lang.OutOfMemoryError x ) {
                System.out.println( "" );
                System.out.println( i + ": *** Out of Memory Exception caught
*** (expected " + (gCount*BLOCKSIZE/KB) + "K garbage)" );
                System.out.println( " free heap is now " + rt.freeMemory() + "
bytes" );
                System.out.println( " total blocks allocated = " + v.size() );
                System.out.println( "" );
            }

            curFree = rt.freeMemory();

            long freed = (curFree - preFree) + actBlockSize;
            if( freed > 0 ) {
                System.out.println( i + ": GC freed " + freed/KB + "K"
                    + " (" + (gCount*actBlockSize/KB) + "K expected)" );
                gCount = 0;
            }

            // drop references to the most recently allocated 2 blocks if free
memory is tight
            if( rt.freeMemory() < 1*MB && v.size() > 2 ) {
                v.removeElementAt( v.size()-1 );
                v.removeElementAt( v.size()-1 );
                gCount += 2;
            }
        }
    }

}
---------- END SOURCE ----------
(Review ID: 181286) 
======================================================================

                                    

Comments
EVALUATION

This looks like it might be a corner case in our handling of the 
nearly out of memory case.  Usually, 
TwoGenerationCollectorPolicy::mem_allocate_work will make two 
attempts to allocate in the young generation: one without the 
heap lock, which can't allocate in from-space, and one with the 
heap lock, which then can allocate in from-space.  In contrast, 
I think GenCollectedHeap::do_collection just calls gen->allocate, 
but the requesting thread owns the heap lock, not the VM thread, 
so we don't attempt to allocate in the from-space.

That said, users really shouldn't be depending on Runtime.freeMemory() 
to know when memory is free.  For instance, because of fragmentation, 
just because Runtime.freeMemory() says there are 4KB free doesn't mean 
that the user can get all 4KB for one object.  That and the fact that 
Runtime.freeMemory() is notoriously broken, inaccurate, and untimely.

###@###.### 2003-05-01
----------------------------------------------------------------
                                     
2003-05-01
SUGGESTED FIX

It might be enough for DefNewGeneration::allocate_from_space to 
allow allocations from from-space if we are the VM thread.  The 
problem will be convincing ourselves that that doesn't cause a 
race in which two threads think they can allocate from from-space: 
one because they hold the heap lock, and one because they are the 
VM thread.  That would hand out the same address as two different 
objects, and be hard to diagnose.

###@###.### 2003-05-01
----------------------------------------------------------------
                                     
2003-05-01
SUGGESTED FIX

------- defNewGeneration.cpp -------
418c418,420
<     if (Heap_lock->owned_by_self()) {
---
>     if (Heap_lock->owned_by_self() ||
>         (SafepointSynchronize::is_at_safepoint() &&
>          Thread::current()->is_VM_thread())) {
                                     
2006-01-24
SUGGESTED FIX

Event:            putback-to
Parent workspace: /net/jano.sfbay/export/disk05/hotspot/ws/main/gc_baseline
                  (jano.sfbay:/export/disk05/hotspot/ws/main/gc_baseline)
Child workspace:  /net/prt-web.sfbay/prt-workspaces/20060126152955.ysr.policy/workspace
                  (prt-web:/net/prt-web.sfbay/prt-workspaces/20060126152955.ysr.policy/workspace)
User:             ysr

Comment:

---------------------------------------------------------

Job ID:                 20060126152955.ysr.policy
Original workspace:     neeraja:/net/spot/scratch/ysr/policy
Submitter:              ysr
Archived data:          /net/prt-archiver.sfbay/data/archived_workspaces/main/gc_baseline/2006/20060126152955.ysr.policy/
Webrev:                 http://analemma.sfbay.sun.com/net/prt-archiver.sfbay/data/archived_workspaces/main/gc_baseline/2006/20060126152955.ysr.policy/workspace/webrevs/webrev-2006.01.26/index.html

Fixed 6369448: Unnecessary FullGC seems to run in 6.0b65
Fixed 4855795: Spurious OutOfMemoryError exceptions

  Webrev: http://analemma.sfbay/net/spot/scratch/ysr/policy/webrev

The problem in 6369448 was that the policy for deciding
whether it was possible to allocate an object in the
young gen was based on the maximum potential size of
young gen (not even Eden) while there was nothing in the
resizing policy that was affected by the failure to accomodate
such requests. The correct thing, under the circumstances, is
to recognize that Eden cannot allocate objects larger than
its _current_ size, and allow an older generation to
satisfy such an allocation request if possible.
This is a day-one bug in the policy object.

The problem in 4855795 was identified by Peter,
as described in the bug report.
We now allow the VM thread to allocate out of from
space when space is tight. This prevents the
"unexpected" out of memory error described.

Piggyback: recent testing with -XX:+CMSMarkStackOverflowALot
revealed some obsolete non-product code had been left around.
The lock in question had been removed sometime ago, but
not all of its uses had been removed. Some (related)
documentation comments were also fixed.

Reviewed by: Jon Masamitsu, John Coomes
Approved (low risk) by: Dave Cox

Fix Verified: yes

Verification Testing: 
   6369448 test in bug report
   4855795 test in bug report
 
Other testing: specjvm, PRT
   refworkload (performance) with CMS, Serial.

Files:
update: src/share/vm/memory/collectorPolicy.cpp
update: src/share/vm/memory/concurrentMarkSweepGeneration.cpp
update: src/share/vm/memory/defNewGeneration.cpp

Examined files: 3768

Contents Summary:
       3   update
    3765   no action (unchanged)
                                     
2006-01-27



Hardware and Software, Engineered to Work Together