JDK-4899343 : CMS collector: prompter finalization?
  • Type: Enhancement
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: 1.4.2
  • Priority: P4
  • Status: Closed
  • Resolution: Duplicate
  • OS: windows_2000
  • CPU: x86
  • Submitted: 2003-07-31
  • Updated: 2005-06-10
  • Resolved: 2005-06-10
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7
7Resolved
Related Reports
Duplicate :  
JDK-4797189 - Finalizers not called promptly enough
Relates :  
JDK-4871440 - memory growth of over 100m with 1.4.1_02 vs. 1.3.1
Relates :  
JDK-4916142 - CompilerThread0: OutOfMemoryError Chunk::new Out of swap space?
Description
Name: tb29552			Date: 07/31/2003


FULL PRODUCT VERSION :
java.vm.info=mixed mode
java.vm.name=Java HotSpot(TM) Server VM
java.vm.specification.name=Java Virtual Machine Specification
java.vm.specification.vendor=Sun Microsystems Inc.
java.vm.specification.version=1.0
java.vm.vendor=Sun Microsystems Inc.
java.vm.version=1.4.2-b28

FULL OS VERSION :
OS Name	Microsoft Windows 2000 Server
Version	5.0.2195 Service Pack 2 Build 2195
System Manufacturer	IBM
System Type	X86-based PC
Processor	x86 Family 6 Model 10 Stepping 1 GenuineIntel ~700 Mhz
Processor	x86 Family 6 Model 10 Stepping 1 GenuineIntel ~700 Mhz
Locale	United States
  Total Physical Memory	2.096.648 KB
  Total Virtual Memory	7.180.764 KB
Page File Space	5.084.116 KB

A DESCRIPTION OF THE PROBLEM :
The HotSpot server VM ignores the maximum heap size setting, resulting in an out of memory error.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Our VM settings are:
-server
-Xms900m
-Xmx900m
-XX:NewSize=32m
-XX:MaxNewSize=32m
-XX:SurvivorRatio=128
-XX:MaxTenuringThreshold=0
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=60
-verbosegc
-XX:+PrintGCApplicationStoppedTime
-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails
-Xloggc:F:\LOG\DT02\SYSTEM\gc.log



EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
We expect the CMS collector to run when the vm reaches the heap size of 900MB.
ACTUAL -
When we run our home grown application server, we see the memory claimed by the VM raising to up to 1,317,138,432 bytes (W2K Performance Monitor/working set). But the last line of the gc-log states that only a few hundred MBs are used by the VM :
[GC [ParNew: 32639K->0K(32704K)] 312471K->287751K(921536K)]
and the OS has 138,289,152 bytes (physical memory) free.
Then the VM exits with the provided message.

I can provide the full gc-log and "W2K Performance Monitor" log if needed.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Exception in thread "CompilerThread0" java.lang.OutOfMemoryError: requested 111372 bytes for Chunk::new. Out of swap space?

At that time W2K reports 138,289,152 bytes free physical memory

REPRODUCIBILITY :
This bug can be reproduced always.

CUSTOMER SUBMITTED WORKAROUND :
When set the maximum heap size to a smaller value, so the old generation fills up to the "CMSInitiatingOccupancyFraction" of 60%, then a normal collection is done by the CMS collector.
(Review ID: 193262) 
======================================================================
###@###.### 10/9/04 00:12 GMT
Comments
CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: dragon
14-06-2004
EVALUATION ###@###.### 2003-08-01: Does the application load a lot of classes? Note that the CMS collector does not unload classes by default. This can cause dead classes to pile up in the perm gen (space that is not reported in the gc logs normally), and moreover for the nmethods in those classes to stay around forever, thus filling up the C heap. The error message indicates that we have run out of space in the C-heap, which could be because of the above. But this is just a conjecture. Use -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled to get the CMS collector to unload classes. Also, run with -XX:+PrintHeapAtGC for observing the state of the perm gen as well (although that in itself will not settle the question). Please send us the results of these experiments, including the verbose gc logs (generated from -XX:+PrintGCDetails which you are already using). ###@###.### 2003-08-01: New logs from customer do not indicate lack of class GC to be implicated. Note that the error indicates exhaustion of the C-heap. CMS does respect the Java heap bound restriction as articulated in the -Xmx directive. Have asked the customer to do a set of small experiments to narrow down the C-heap exhaustion issue. . CMS allocates some extra side data-structures (and external marking bit map etc.) which take up some fixed extra space in the C-heap. This space overhead is proportional to the max space available to the old generation and the perm generation. This overhead may be pushing the C-heap over the edge. Running with a smaller Java heap (since the application requires much less than 900 mb of Java heap, as indicated by the log files) may work. Otherwise get more memory. . It may also turn out that this is orthogonal to the specific collector and may be related in some way to bugs 4871440 which indicates increase C-heap consumption by the JVM runtime. This space will be updated with what we learn from the requested experiments. ###@###.### 2003-08-19: It doesn't look like the fixed overheads of the CMS collector are responsible since the application runs for a while before exhausting the C-heap. It is possible that he program may be holding on to C-heap memory because of relying on finalization or other reference object processing to free up non-heap memory. Investigating that angle. ###@###.### 2003-09-09: Customer found that the problem went away as soon as Inflater/Deflater streams were explicitly closed, indicating that promptness (of lack thereof) or finalization of these objects may be implicated. It has been generally recommended that users explicitly close/end Inflater/Deflater/ZipFile objects after last use rather than rely on "prompt" finalization to reclaim native resources. While it is easy to think of adversarial scenarios where finalization can be delayed in the case of CMS, we are trying to determine the precise cause for the delayed finalization in the case of CMS and try and improve it if possible. I am changing this to an RFE for that purpose.
11-06-2004