Bug ID: JDK-6268083 Core J2SE JPEG ImageWriter consumes much memory when writing

Type: Bug
Component: client-libs
Sub-Component: javax.imageio
Affected Version: 1.0_01,1.1.2

Priority: P3
Status: Closed
Resolution: Won't Fix
OS: generic,windows_xp
CPU: generic,x86

Submitted: 2005-05-10
Updated: 2016-01-12
Resolved: 2016-01-12

Running the attached test as either

$ java JPEGTest imageio

or

$ java JPEGTest imageio rgb

produces the same result:

27000832
20406272
47407104

which indicates that the JPEG writer is consuming an additional 20 Mb when writing a 27Mb image. Similar behavior is observed when using the old core J2SE JPEG API to write BGR images but this old API consumes no additional memory when writing RGB images.
###@###.### 2005-05-10 18:05:46 GMT
###@###.### 2005-05-10 18:06:37 GMT
The original test case must have been changed from 3000x3000 before attaching to the report. I think the VM used was version 1.4.2. I changed the size to 3000x3000 and ran the test again; a transcript follows.

--- BEGIN ---

$ java -mx512m JPEGTest imageio rgb
Using RGB source image
Testing Image I/O
27000832
20389888
47390720
bpb@chatnoir:tmp{562}$ ~/test/solaris-sparc/jdk1.5.0_01/bin/java -mx512m JPEGTes
t imageio rgb
Using RGB source image
Testing Image I/O
29360128
0
29360128
bpb@chatnoir:tmp{563}$ java -mx512m JPEGTest imageio bgr
Using BGR source image
Testing Image I/O
27000832
20389888
47390720
bpb@chatnoir:tmp{564}$ ~/test/solaris-sparc/jdk1.5.0_01/bin/java -mx512m JPEGTes
t imageio bgr
Using BGR source image
Testing Image I/O
29360128
16777216
46137344
bpb@chatnoir:tmp{565}$ java -version
java version "1.4.2_04"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_04-b05)
Java HotSpot(TM) Client VM (build 1.4.2_04-b05, mixed mode)
bpb@chatnoir:tmp{566}$ ~/test/solaris-sparc/jdk1.5.0_01/bin/java -version
java version "1.5.0_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_01-b08)
Java HotSpot(TM) Server VM (build 1.5.0_01-b08, mixed mode)

--- END ---

So it looks as if the problem is fixed for RGB in 1.5.0_01 versus 1.4.2_04 but a problem remains for BGR.

EVALUATION See also 5086246:The JPEG encoder reserves a lot of memory to convert large BGR images which also talks about the memory needed to encode JPEG images. I think that's the issue referred to in the description >Similar behavior is observed when using the old core J2SE JPEG API to write BGR images I haven't closed that as a dup as I have not verified if both the com.sun.image.code.jpeg and the image i/o plugin need to be fixed separately. Certainly I think both 5086246 and 6268083 need to be considered together.
06-12-2005
EVALUATION I can reproduce this problem with latest jdk builds if server VM is used. We are allocating bunch of buffers to do conversion and server VM is releasing this memory too slowly (client VM does this much faster and this is why problem is not observable). After fix for 6266748 we are doing much better for RGB images but problem still exist for other types of images. For instance: === BEGIN === bash-2.03$ /java/re/jdk/6.0/promoted/all/b54/binaries/solaris-sparc/bin/java -server -verbose:gc -Xmx512m JPEGTest imageio rgb Using RGB source image Testing Image I/O 108003328 0 108003328 bash-2.03$ /java/re/jdk/6.0/promoted/all/b54/binaries/solaris-sparc/bin/java -server -verbose:gc -Xmx512m JPEGTest imageio bgr Using BGR source image Testing Image I/O [GC 108530K->105978K(110464K), 0.0243160 secs] [Full GC 105978K->105720K(138432K), 0.1943596 secs] [GC 108791K->105752K(138432K), 0.0178995 secs] [GC 108812K->105768K(138432K), 0.0237343 secs] [GC 108840K->105752K(139392K), 0.0216070 secs] [GC 109772K->105848K(139392K), 0.0235697 secs] [GC 109875K->105848K(140736K), 0.0241414 secs] [GC 111273K->105784K(140736K), 0.0106827 secs] [GC 111208K->105768K(142336K), 0.0175313 secs] [GC 112871K->105768K(142400K), 0.0173744 secs] [GC 112857K->105880K(143616K), 0.0169620 secs] [GC 114259K->105768K(143616K), 0.0190720 secs] [GC 114145K->105768K(145088K), 0.0000000 secs] [GC 115616K->105768K(145088K), 0.0219148 secs] [GC 115614K->105784K(146304K), 0.0184878 secs] [GC 117096K->105800K(146496K), 0.0201939 secs] [GC 117112K->105784K(147776K), 0.0221436 secs] 108003328 38207488 146210816 === END === There are few things we can do to improve this: - first of all we may avoid allocation of temporary buffers for BGR images at all We can lower our requirements for rasters that can be converted on the fly (i.e. what are required to consider source and destination similar enough) and avoid usage of generic procedure that allocated these buffers. - we might consider reusage of buffers we are allocating in the generic codepath This might decrease number of allocations even in complex cases. (this seems to be rfe 5086246).
03-10-2005
EVALUATION I'm a little confused. The attached testcase uses W/H values of 6000 pixels, so the BufferedImage uses about 108 MB (600060003 bytes/pixel), not the 27 MB (maybe the submitter changed the testcase a bit). In order to not get an OutOfMemoryError I had to increase the max heap size using -Xmx512M. When I run the testcase on JDK 1.4.2 I see: bytes for BI: 108003328 extra bytes: 90701824 total bytes: 198705152 On JDK 5 or JDK 6: bytes for BI: 108003328 extra bytes: 0 total bytes: 108003328 I am able to reproduce this on JDK 1.4.2, but not on JDK 5 or JDK 6, so I suspect that the problem may have been resolved in JDK 5. Could the submitter tell us what version of the JDK this was reproducible on, and whether he can reproduce this on a recent JDK? Marking incomplete.
29-09-2005
EVALUATION Could this be related to the recently filed 6266748? ###@###.### 2005-05-10 18:38:07 GMT Could be. The attached test could be modified optionally to use a JAI image source which unlike a BufferedImage does not copy data in getData() for untiled images when the requested region is within the single tile. So fixing the other problem might fix this one too. ###@###.### 2005-05-10 18:47:31 GMT
10-05-2005

Relates :	JDK-5086246 - The JPEG encoder reserves a lot of memory to convert large BGR images
Relates :	JDK-6266748 - JPEGImageWriter.write() makes a copy of the raster (inefficient)