United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-6299405 : ImageInputStreamImpl still uses a finalize() which causes java.lang.OutOfMemoryError

Details
Type:
Bug
Submit Date:
2005-07-20
Status:
Resolved
Updated Date:
2010-04-26
Project Name:
JDK
Resolved Date:
2005-12-05
Component:
client-libs
OS:
windows_2000
Sub-Component:
javax.imageio
CPU:
x86
Priority:
P3
Resolution:
Fixed
Affected Versions:
6
Fixed Versions:
6 (b63)

Related Reports

Sub Tasks

Description
J2SE Version (please include all output from java -version flag):
java version "1.6.0-ea"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.6.0-ea-b38)
Java HotSpot(TM) Client VM (build 1.6.0-ea-b38, mixed mode, sharing)

Does this problem occur on J2SE 1.4.x or 5.0.x ?  Yes / No (pick one)
According to Bug Reports 4827358 and 4867874 it was fixed in 1.5(tiger), 
however I can't seem to find a version with it working.

Operating System Configuration Information (be specific):
Microsoft Windows 2000 [Version 5.00.2195]

Hardware Configuration Information (be specific):
Desktop

Bug Description:
Regression: Memory allocation doesn't wait for finalization to catch up.

See bug Report 4827358 for more details. 
Sun decided not to fix the bug properly, but instead decided to remove 
finalize() from everywhere (basically stating that finalize is a useless 
feature that should not be used).  Bug Reports 4827358 and 4867874 state 
that Sun was meant to have removed finalize() and replaced it with the 
Java 2D Disposer mechanism.  Sure enough even in 1.6.0-ea 
javax.imageio.stream.ImageInputStreamImpl still has the finalize() 
method and is still causing OutOfMemoryErrors.
Basically each instance of ImageInputStreamImpl is holding a lot of data 
that is not getting used while it waits for it's turn to be finalized, 
and hence the OutOfMemoryErrors are thrown.
 
Steps to Reproduce (be specific):
run the code: (at some point in time you will see the error 
java.lang.OutOfMemoryError: Java heap space)

The lines: 
  FileImageInputStream[] streams = new FileImageInputStream[numberOfStreams];
  for (int i = 0; i < numberOfStreams; i++) {
    streams[i] = new FileImageInputStream(file); 
  }
  streams = null; 
are not required and are there simply to prove the max heap space is
big enough to hold numberOfStreams streams.

Expected: program to complete normally.
Actual: Error: java.lang.OutOfMemoryError: Java heap space

Also note that the error report is stating that there is a problem with
java's finalize implemention not just these specific examples.  In
many other reports Sun has said for people to remove their finalize
methods (basically finalize should be deprecated and is a useless
feature).  Sun is still using finalize in other places as well such as
Fonts and ShellFolders, which is also a problem but not as noticable
due to these objects being a lot smaller hence would take a very long
time to cause an OutOfMemoryError for a system with a normal heap
space.  ColorIndexModel is probably a larger object which also uses
finalize, however so far this has not being causing us
OutOfMemoryErrors.


###@###.### 2005-07-20 16:20:57 GMT

Comments
EVALUATION

The submitter also mentions the fact that other classes in the JDK still use
finalizers.  There is already an existing bug for Font (6247526), and I
recently filed one for IndexColorModel (6353493).
2005-11-23 
EVALUATION

As mentioned in the evaluation above, we can't simply empty the finalize()
method at the ImageInputStreamImpl level, since third-party IISI subclasses
may rely on the finalize() method calling their close() method as per the spec.
But for ImageInputStreamImpl subclasses under our control, we can empty their
finalize() methods and use the Java2D Disposer mechanism instead.  A few of
them are easy (from the javax.imageio.stream package):
    FileImageInputStream
    FileImageOutputStream
    FileCacheImageInputStream
    MemoryCacheImageInputStream

Unfortunately, we cannot easily apply this technique to the two remaining
public stream classes that ship with the JDK:
    FileCacheImageOutputStream
    MemoryCacheImageOutputStream

In these two cases, there is some complex logic in the close() method that
ensures that any remaining cached data is written to the underlying output
stream before the ImageOutputStream is disposed itself.  With some major
reworking we could probably find a way to get the Disposer mechanism to
handle this work, but it will be messy, so for now I suggest we just forget
about having an empty finalize() method in these two subclasses.

Interestingly, there are other subclasses of Image{Input,Output}StreamImpl
in our implementation that I had not previously considered:
    com.sun.imageio.plugins.common.SubImageInputStream
    com.sun.imageio.plugins.png.PNGImageWriter (ChunkStream and IDATOutputStream)

These classes are completely under our control and do not require any
explicit disposal, so we can simply add an empty finalize() method to these
classes.

There are some pretty incredible performance gains associated with these
changes, especially for small images where creating and disposing
ImageInputStreams accounts for much of the overhead.  Most of the benefit
comes from more prompt disposal of native resources (e.g. RandomAccessFile
handles) via the Disposer mechanism, instead of waiting around for finalizers
to be run.  This means smaller, less-frequent GCs are required overall.

For most of my J2DBench test runs, I've used large heap settings to help minimize
the cost of GC pauses in the results, but as you can see, even with a large
heap there will still be lots of pauses from GC due to the cost of
finalization (prior to my changes).  The results I report here will look even
better with smaller/default heap sizes (for example, if you see a performance
improvement of 40% with a large heap size, you might see a performance
improvement of 3x with a smaller heap size with these changes since less
full GCs are required now).

For example, here are some J2DBench results for constructing and closing
ImageInputStreams in a tight loop.  These results were taken with
solaris-sparc on SB2000, 900 MHz USIII, 1GB RAM, -client, -Xms256M -Xmx512M,
"base" is the 2005-11-19.mustang build, "test" contains the changes outlined
above:

Options common across all tests:
  testname=imageio.input.stream.tests.construct
  imageio.opts.size=1
  imageio.opts.content=blank
  imageio.input.opts.imageio.useCache=false
imageio.input.opts.general.source=byteArray:
base: 10.26186612 (var=1.96%) (100.0%)
test: 13.51616839 (var=3.39%) (131.71%)
imageio.input.opts.general.source=file:
base: 11.54184630 (var=2.51%) (100.0%)
test: 16.06037735 (var=2.72%) (139.15%)

Here are some more J2DBench results, this time measuring performance of
ImageReader.read() in a tight loop.  In this case, we construct and close a
new ImageInputStream for each iteration.  These results were taken with
solaris-i586 on W2100z, 2.0 GHz Opteron, 2GB RAM, -client, -Xms256M -Xmx512M,
"base" is the 2005-11-19.mustang build, "test" contains the changes outlined
above:

Options common across all tests:
  testname=imageio.input.image.imageio.reader.tests.read
  imageio.input.image.imageio.reader.opts.ignoreMetadata=true
  imageio.input.image.imageio.reader.opts.installListener=false
  imageio.input.image.imageio.reader.opts.seekForwardOnly=true
  imageio.opts.content=blank
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=false,imageio.opts.size=1:
base: 9.386375088 (var=1.38%) (100.0%)
test: 11.34964176 (var=0.62%) (120.92%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=false,imageio.opts.size=20:
base: 2369.359445 (var=0.75%) (100.0%)
test: 2788.373506 (var=0.8%) (117.68%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=true,imageio.opts.size=1:
base: 3.650076962 (var=0.42%) (100.0%)
test: 4.027091058 (var=0.4%) (110.33%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=true,imageio.opts.size=20:
base: 1205.155702 (var=0.43%) (100.0%)
test: 1306.425452 (var=0.8%) (108.4%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=false,imageio.opts.size=1:
base: 9.402652627 (var=0.34%) (100.0%)
test: 11.49206186 (var=0.87%) (122.22%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=false,imageio.opts.size=20:
base: 2365.254495 (var=0.34%) (100.0%)
test: 2761.758152 (var=0.37%) (116.76%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=true,imageio.opts.size=1:
base: 9.327962944 (var=0.55%) (100.0%)
test: 11.36734489 (var=0.92%) (121.86%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=true,imageio.opts.size=20:
base: 2391.495451 (var=0.82%) (100.0%)
test: 2749.798224 (var=0.12%) (114.98%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=false,imageio.opts.size=1:
base: 7.070189840 (var=0.53%) (100.0%)
test: 8.279342487 (var=1.22%) (117.1%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=false,imageio.opts.size=20:
base: 2471.124744 (var=0.71%) (100.0%)
test: 2852.417970 (var=0.56%) (115.43%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=true,imageio.opts.size=1:
base: 2.924363636 (var=0.33%) (100.0%)
test: 3.168499688 (var=0.28%) (108.35%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=true,imageio.opts.size=20:
base: 1101.486641 (var=0.83%) (100.0%)
test: 1185.383022 (var=0.54%) (107.62%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=false,imageio.opts.size=1:
base: 6.617219917 (var=0.44%) (100.0%)
test: 7.626905644 (var=0.88%) (115.26%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=false,imageio.opts.size=20:
base: 2307.011070 (var=0.72%) (100.0%)
test: 2639.233097 (var=1.09%) (114.4%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=true,imageio.opts.size=1:
base: 6.542915185 (var=0.69%) (100.0%)
test: 7.696662270 (var=1.27%) (117.63%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=true,imageio.opts.size=20:
base: 2296.733617 (var=0.96%) (100.0%)
test: 2658.816412 (var=1.23%) (115.77%)

So this is showing a 10-20% performance improvement in reading smallish (1x1
and 20x20) images.  As described earlier, if we use default heap sizes, the
performance gain becomes much more apparent.  This would also be true in a
server side situation, for example, where full GCs can bring the server to
a crawl, so the changes suggested in this bug report should help reduce
expensive full GCs in heavily loaded server environments.  Here are some
J2DBench results for windows-i586 showing this effect, taken on a 2x 2.8 GHz P4,
1GB RAM, -client, default heap settings:

Options common across all tests:
  testname=imageio.input.image.imageio.reader.tests.read
  imageio.input.image.imageio.reader.opts.ignoreMetadata=true
  imageio.input.image.imageio.reader.opts.installListener=false
  imageio.input.image.imageio.reader.opts.seekForwardOnly=true
  imageio.opts.content=blank
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=false,imageio.opts.size=1:
base: 2.843131123 (var=0.01%) (100.0%)
test: 9.026369168 (var=0.31%) (317.48%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=false,imageio.opts.size=20:
base: 1048.280175 (var=0.16%) (100.0%)
test: 2378.619701 (var=0.0%) (226.91%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=true,imageio.opts.size=1:
base: 0.514964610 (var=6.45%) (100.0%)
test: 0.627415777 (var=5.53%) (121.84%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=true,imageio.opts.size=20:
base: 203.4689116 (var=6.33%) (100.0%)
test: 239.4026666 (var=3.37%) (117.66%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=false,imageio.opts.size=1:
base: 2.113619422 (var=0.01%) (100.0%)
test: 5.392725355 (var=0.33%) (255.14%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=false,imageio.opts.size=20:
base: 747.0786044 (var=0.66%) (100.0%)
test: 1592.436363 (var=0.0%) (213.16%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=true,imageio.opts.size=1:
base: 2.113211532 (var=0.5%) (100.0%)
test: 5.340571428 (var=0.65%) (252.72%)
imageio.input.image.imageio.opts.format=core-jpg,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=true,imageio.opts.size=20:
base: 746.4361029 (var=0.5%) (100.0%)
test: 1598.984635 (var=27.19%) (214.22%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=false,imageio.opts.size=1:
base: 1.273921200 (var=0.0%) (100.0%)
test: 5.714314374 (var=0.0%) (448.56%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=false,imageio.opts.size=20:
base: 537.3260227 (var=0.01%) (100.0%)
test: 1916.055469 (var=0.97%) (356.59%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=true,imageio.opts.size=1:
base: 0.448228595 (var=2.31%) (100.0%)
test: 0.593630835 (var=3.27%) (132.44%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=byteArray,imageio.input.opts.imageio.useCache=true,imageio.opts.size=20:
base: 180.4028535 (var=1.97%) (100.0%)
test: 225.3961804 (var=3.76%) (124.94%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=false,imageio.opts.size=1:
base: 1.077513513 (var=0.17%) (100.0%)
test: 3.444194142 (var=1.29%) (319.64%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=false,imageio.opts.size=20:
base: 439.9187947 (var=6.58%) (100.0%)
test: 1324.614065 (var=0.51%) (301.1%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=true,imageio.opts.size=1:
base: 1.103473684 (var=0.5%) (100.0%)
test: 3.428396447 (var=0.48%) (310.69%)
imageio.input.image.imageio.opts.format=core-png,imageio.input.opts.general.source=file,imageio.input.opts.imageio.useCache=true,imageio.opts.size=20:
base: 467.3706666 (var=0.33%) (100.0%)
test: 1295.015384 (var=2.59%) (277.09%)

It is interesting to note that the PNGImageReader actually
creates one or more SubImageInputStreams in addition to the provided
ImageInputStream as part of each decoding process.  There will be one
SubImageInputStream for every IDAT chunk within the image stream, so for
example, small PNG images may use only one SubImageInputStream, but larger
PNG images with little compression may use lots of SubImageInputStreams.
Now that we've added an empty finalize() method to SubImageInputStream, we
will see performance improvements for all PNG images, both large and small.
More results taken with the solaris-i586 configuration described above:

Options common across all tests:
  testname=imageio.input.image.imageio.reader.tests.read
  imageio.input.image.imageio.reader.opts.ignoreMetadata=true
  imageio.input.opts.general.source=file
  imageio.input.image.imageio.opts.format=core-png
  imageio.input.image.imageio.reader.opts.installListener=false
  imageio.input.image.imageio.reader.opts.seekForwardOnly=true
  imageio.input.opts.imageio.useCache=false
  imageio.opts.content=random
imageio.opts.size=1000:
base: 25511.84478 (var=0.06%) (100.0%)
test: 26318.99597 (var=0.2%) (103.16%)
imageio.opts.size=20:
base: 2266.548403 (var=0.53%) (100.0%)
test: 2833.257023 (var=0.29%) (125.0%)
imageio.opts.size=250:
base: 25526.77439 (var=0.04%) (100.0%)
test: 25802.99502 (var=0.67%) (101.08%)
2005-11-22 
EVALUATION

Apps that intensively create ImageInputStreams should already be closing the
streams explicitly and the ImageI/O convenience methods already do this.
But the non-empty finalize() will mean less prompt GC().
The finalize() method needs to be emptied and its functionality (closing the
stream) replaced with a more prompt mechanism.

But once native resources are freed these aren't large objects if they
null out the stream in which case perhaps only programs that don't
explicitly close() streams really run into any problems.

But it looks like FileImageInputStream.close() and FileCacheImageInputStream.close()
don't null out their RandomAccessFile references.
The fix for this perhaps should do that too - although its possible that this
is done so that calling functions such as read() after the stream is closed
get an IOException rather than an NPE. If that's the case then that part of the
work would need to include specific checks for null to preserve that behaviour.
Perhaps if the finalize() method is emptied it will make this much less of an issue
anyway.

On second thoughts I don't think what Disposer will work here at the ImageInputStreamImpl
level as we can't refer to it in the Disposer and I don't see anything
else we can use that's in that shared superclass. It may be back to plan A to
just implement Disposer and override finalize() to be empty in the subclasses.
2005-09-26


Hardware and Software, Engineered to Work Together