United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: JDK-6409506 On b78 it takes 10 times longer to read a compressed binary file
JDK-6409506 : On b78 it takes 10 times longer to read a compressed binary file

Details
Type:
Bug
Submit Date:
2006-04-06
Status:
Resolved
Updated Date:
2011-02-16
Project Name:
JDK
Resolved Date:
2006-04-29
Component:
core-libs
OS:
windows_2003,linux_redhat_4.0,linux,solaris_10,windows_xp
Sub-Component:
java.io
CPU:
x86,sparc
Priority:
P3
Resolution:
Fixed
Affected Versions:
6
Fixed Versions:

Related Reports
Duplicate:
Duplicate:
Relates:

Sub Tasks

Description
FULL PRODUCT VERSION :
[gat@MyLaptop ~]$ /home/gat/JAVA/JDK16/jdk1.6.0_b78/bin/java -version
java version "1.6.0-beta2"
Java(TM) SE Runtime Environment (build 1.6.0-beta2-b78)
Java HotSpot(TM) Client VM (build 1.6.0-beta2-b78, mixed mode, sharing)
[gat@MyLaptop ~]$


ADDITIONAL OS VERSION INFORMATION :
[gat@MyLaptop ~]$ uname -a
Linux MyLaptop.gatworks.com 2.6.11-1.1369_FC4 #1 Thu Jun 2 22:55:56 EDT 2005 i686 i686 i386 GNU/Linux
[gat@MyLaptop ~]$


A DESCRIPTION OF THE PROBLEM :
Tried the next beta snapshot release. Program startup time took a bit longer than usual, that I taught it was spinning.
After 200 seconds the process idled. With a few ^\, i was able to see that the program was reading the compressed data file. That routine used to take some 17-19 seconds. Now it takes nearly 200 seconds.

REGRESSION.  Last worked in version mustang

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the attached source code against a compressed binary file.


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
/*
 * Main.java
 *
 * Created on April 4, 2006, 9:33 AM
 *
 * To change this template, choose Tools | Options and locate the
template under
 * the Source Creation and Management node. Right-click the template
and choose
 * Open. You can then make changes to the template in the Source Editor.
 */
/* Copyright (c) 2006 U. George, Gatworks.com. All rights reserved. */

//package slowread;

/**
 *
 * @author gat
 */
import java.io.*;
import  java.util.*;
import  java.util.zip.*;

public class Main {
    
    /** Creates a new instance of Main */
    public Main() {
        rOpen();
    }
    
    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        // TODO code application logic here
        new Main();
    }
    
    DataInputStream     in = null;
    RandomAccessFile    ra = null, rb = null;
    
    void
            rOpen() {
        long startTime = Calendar.getInstance().getTimeInMillis();
        
        try {
            String filename  = "GpsDataUSB17.gz";
            String directory = "";
            directory = ( directory.equals("") ) ? "" : directory +
                    File.separatorChar;
            File f = new File( directory + filename );
            System.out.println("GpsData_FilePortBinary.open(): Opening Binary Data file: " + f.getAbsolutePath() + ". "
                    + ( f.exists() ? " (File Found)" : " (File Not Found)" ));
            if (! f.exists() )
                return ;
            FileInputStream fr = new FileInputStream( f );
            BufferedInputStream isr = null;
            try {
                GZIPInputStream gz = new GZIPInputStream( fr );
                isr = new BufferedInputStream( gz );
            } catch ( Exception e ) {
                if ( e instanceof java.io.IOException
                        ||   e instanceof java.io.EOFException ) {
                    isr = new BufferedInputStream( fr );
                } else
                    e.printStackTrace();
            }
            in = new DataInputStream( isr );
        } catch( Exception e ){ e.printStackTrace(); return; }
        
        File f;
        try {
            f = File.createTempFile( "jGPSf", ".tmp");
            ra = new RandomAccessFile( f, "rw" );
            f.deleteOnExit();
            f = File.createTempFile( "jGPSg", ".tmp");
            rb = new RandomAccessFile( f, "rw" );
            //f.deleteOnExit();
            f.delete();
        } catch( Exception e ) { e.printStackTrace(); ra = rb = null;
        return; }
        int pos = 0;
        int gpsInputPos = 0;
        byte[] gpsBuf = new byte[ 8000 ];
        byte[] buf = new byte[80];
        byte[]  b = null;
        while ( ( b = readBinaryPacket()) != null ) {
            //System.out.println("Writing line "+new String(b));
            try {
                rb.writeInt((int) ra.getFilePointer() );
                ra.write( b );
            } catch ( Exception e ) { e.printStackTrace(); }
        }
        long endTime = Calendar.getInstance().getTimeInMillis();
        long diffTime = endTime - startTime;
        System.out.println("FilePort_Binary.rOpen(); time to read="
                +(diffTime/1000) + "." + (diffTime%1000) + " seconds" );
        try {
            in.close();
        } catch ( Exception e ) {}
        return;
    }
    byte[]
            readBinaryPacket() {
        
        try {
            int majic = in.readInt();
            if ( majic != 0xdeadbeef ) {
                System.out.println("WrongMajic. received "
                        + Integer.toHexString( majic )
                        + " should be "
                        + Integer.toHexString(0xdeadbeef )+" at file position " );
                
                return null;
            }
            long time  = in.readLong();
            int len   = in.readInt();
            byte[] buf = new byte[ len ];
            in.readFully(buf);
            int majic2 = in.readInt();
            if ( majic2 != 0xbeefdead ) {
                System.out.println("BadMajic2 ");
                System.out.println("Majic2="+Integer.toHexString(
                        majic2 ));
                Date d = new Date( time );
                System.out.println("len="+len);
                return null;
            }
            return buf;
        } catch ( java.io.EOFException e ) {
            if ( e.getMessage().equals( "Unexpected end of ZLIB input stream" ) ) {
                System.out.println("FilePort_Binary.readBinaryPacket();"+
                        " Unexpected end of ZLIB input stream" + " Continueing");
            }
        } catch( Exception e ) { e.printStackTrace(); }
        
        System.out.println("Returning null");
        return null;
    }
}
---------- END SOURCE ----------

                                    

Comments
WORK AROUND

Do not buffer the GZipInputStream.
                                     
2006-04-07
EVALUATION

This is as a result of the fix for 6192696.
                                     
2006-04-07
EVALUATION

The fix for 6192696 changes BufferedInputStream from trying to read enough data from its wrapped InputStream to fill the internal buffer, to only filling the amount the wrapped InputStream has available. That way avoiding the possibility of blocking. 

This has a negative performace effect on InputStreams that cannot implement a good available method, for example GZipInputStream returns 1 for data available and 0 for none. There is no easy way to determine the amount of available data from a dynamically uncompressed stream. This causes Buffered GZipInputStream to be read 1 byte at a time, and hence poor performance. 

There may also be other InputStreams that are implemented this way, either by force (as they cannot determine the amount of available data), or simply bad implementation where by the implementer simply returns 1 to indicate "data available". As BufferedInputStream has had this "block" behavior forever and in the interest of compatibility the best solution here is to backout the fix for 6192696, and revert BufferedInputStream back to its previous behavior.
                                     
2006-04-19



Hardware and Software, Engineered to Work Together