FULL PRODUCT VERSION :
Java(TM) SE Runtime Environment (build 1.6.0_01-b06)
Java HotSpot(TM) Server VM (build 1.6.0_01-b06, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
Linux cdscil1 2.6.9-42.ELsmp #1 SMP Wed Jul 12 23:32:02 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux


A DESCRIPTION OF THE PROBLEM :
At 7:30 AM this morning, a Java program ran out of memory at this spot:

2007-05-04 07:30:22,532 [pool-1-thread-4] INFO  opcalc.CalculationsExecutioner - Running: 297563 from 2007-05-04 01:00:00.0 to 2007-05-05 08:00:18.0 (perform)
Exception in thread "LocationThread0" java.lang.OutOfMemoryError: GC overhead limit exceeded
	at java.util.concurrent.ConcurrentHashMap$Segment.put(ConcurrentHashMap.java:435)
	at java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:894)
	at java.util.ResourceBundle.putBundleInCache(ResourceBundle.java:1642)
	at java.util.ResourceBundle.findBundle(ResourceBundle.java:1371)
	at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1236)
	at java.util.ResourceBundle.getBundle(ResourceBundle.java:834)
	at sun.util.resources.LocaleData$1.run(LocaleData.java:127)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.util.resources.LocaleData.getBundle(LocaleData.java:125)
	at sun.util.resources.LocaleData.getTimeZoneNames(LocaleData.java:97)
	at sun.util.TimeZoneNameUtility.getBundle(TimeZoneNameUtility.java:115)
	at sun.util.TimeZoneNameUtility.retrieveDisplayNames(TimeZoneNameUtility.java:80)
	at java.util.TimeZone.getDisplayNames(TimeZone.java:399)
	at java.util.TimeZone.getDisplayName(TimeZone.java:350)
	at java.util.Date.toString(Date.java:1025)
	at java.lang.String.valueOf(String.java:2827)
	at java.lang.StringBuffer.append(StringBuffer.java:219)
	at com.ge.geps.mdweb.Odas.RMD_ODAS_OPDATAS.RMD_ODAS_REQSTRATEGY.AddTagDataToDataStore(RMD_ODAS_REQSTRATEGY.java:1034)
	at com.ge.geps.mdweb.Odas.RMD_ODAS_OPDATAS.RMD_ODAS_REQSTRATEGY.INTERNAL_QueryDataLocations(RMD_ODAS_REQSTRATEGY.java:821)
	at com.ge.geps.mdweb.Odas.RMD_ODAS_OPDATAS.RMD_ODAS_REQSTRATEGY.GetDataLocationForDataStore(RMD_ODAS_REQSTRATEGY.java:580)
	at com.ge.geps.mdweb.Odas.RMD_ODAS_OPDATAS.RMD_ODAS_LOCATIONSTHREAD.run(RMD_ODAS_LOCATIONSTHREAD.java:64)

Ponder these facts:

1. The OutOfMemoryError was thrown from ResourceBundle at line 1371 (after ResourceBundle.beginLoading was called but before ResourceBundle.endLoading was called).

2. ResourceBundle stored the current Thread (LocationThread0, a com.ge.geps.mdweb.Odas.RMD_ODAS_OPDATAS.RMD_ODAS_LOCATIONSTHREAD object) to use as a lock object (see ResourceBundle.beginLoading at line 1455).

3. ResourceBundle was supposed to call ResourceBundle.endLoading at line 1372 but never did because the OutOfMemoryError interrupted execution at line 1371 and the method had no try/finally block to ensure that endLoading would be called.  This is BUG NUMBER ONE.

4. ResourceBundle.beginLoading catches and ignores InterruptedException at line 1474 within a loop which means there is no way to cancel the worker.wait() call at line 1473.  According to _Java Concurrency in Practice_ by Brian Goetz et al, an InterruptedException should never be ignored like this.  This is BUG NUMBER TWO.

5. The Java program had other threads which continued to run.  Some of these threads tried to print Date objects as Strings, which caused these threads to re-enter the ResourceBundle.beginLoading method and hang indefinitely waiting for the LocationThread0 lock to be released.

As a result, the Java program was unable to terminate on its own and I had to kill it once I noticed that it was hanging.  I sent the Java program a QUIT signal to get a thread stack dump, which allowed me to find the above bugs.  I will conclude this bug report by attaching the thread dump further below.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
You would have to make the Java program run out of memory at just the right time.  I think the thread dump contains enough evidence to show that the bugs in ResourceBundle caused the program to hang indefinitely.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Other threads in the Java program should have been able to continue execution even after calling into ResourceBundle.
ACTUAL -
Other threads in the Java program hung indefinitely waiting for a lock object in ResourceBundle to be released.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
see err.log in attachment


REPRODUCIBILITY :
This bug can be reproduced rarely.