United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: JDK-6550798 Using InputStream.skip with ResponseCache will cause partial data to be cached
JDK-6550798 : Using InputStream.skip with ResponseCache will cause partial data to be cached

Details
Type:
Bug
Submit Date:
2007-04-26
Status:
Resolved
Updated Date:
2010-05-10
Project Name:
JDK
Resolved Date:
2007-05-18
Component:
core-libs
OS:
generic,windows_xp
Sub-Component:
java.net
CPU:
x86,generic
Priority:
P2
Resolution:
Fixed
Affected Versions:
6,6u1
Fixed Versions:
6u2 (b03)

Related Reports
Backport:
Backport:
Duplicate:

Sub Tasks

Description
While investigating:

bug 6549146 : Java resource cache is corrupted

I noticed that if we use ResponseCache and InputStream.skip (is.skip) together, the skipped bytes won't be write out to the cache file.

Is this expected ?  Or is this a bug ?

Problem is sun.net.www.protocol.http.HttpURLConnection, inner class HttpInputStream, does not override the skip method.  It will only write to the cache file on read, but skip is ignored.

Attached simple testcase will show the problem.   Just modify the URL in test.java to point to any resource will do.

We can detect the difference in size of the cached file and the http content-length header, and then choose whether to write out the file to the cache.  But is this the correct way to do it ?   Should the networking code handle is.skip ?  Seems like it is not very useful to write out partial file to the cache.

The JMF code uses the is.skip method during loading of video, and with Java 6 plugin fully implement response cache, we run into this problem.



import java.net.*;
import java.io.*;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.BufferedInputStream;
import java.io.ByteArrayInputStream;
import java.io.PrintStream;
import java.io.InputStream;
import java.io.File;
import java.net.CacheRequest;
import java.net.CacheResponse;
import java.net.ResponseCache;
import java.net.URI;
import java.net.URL;
import java.net.URLConnection;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;
import java.util.List;
import java.util.Iterator;
import java.util.StringTokenizer;
import java.util.jar.JarInputStream;
import java.util.jar.JarFile;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.security.PrivilegedExceptionAction;
import java.security.PrivilegedActionException;
import java.security.Principal;
import java.security.cert.Certificate;
import javax.net.ssl.SSLPeerUnverifiedException;
/**
 * A callback mechanism to be set up with URLConnection to enable
 * access to caching management.
 *
 * A CacheHandler can be registered with URLConnection by doing a
 * URLConnection.setDefaultCacheHandler(CacheHandler). The currently
 * registered CacheHandler is stored in a protected field in
 * URLConnection.
 *
 * @since 1.5
 */
public class DeployCacheHandler extends java.net.ResponseCache {
    private boolean inCacheHandler = false;
    private boolean _downloading = false;
    
    public static void reset() {
        // Set system wide cache handler
	System.out.println("install deploy cache handler");
        ResponseCache.setDefault(new DeployCacheHandler());
    }
    
    /**
     * Retrieve the cached response based on the requesting uri,
     * request method and request headers. Typically this method is
     * called by the protocol handler before it sends out the request
     * to get the network resource. If a cached response is returned,
     * that resource is used instead.
     *
     * @param uri a <code>URI</code> used to reference the requested
     *            network resource
     * @param rqstMethod a <code>String</code> representing the request
     *            method
     * @param rqstHeaders - a Map from request header
     *            field names to lists of field values representing
     *            the current request headers
     * @returns a <code>CacheResponse</code> instance if available
     *          from cache, or null otherwise
     * @throws	IOException if an I/O error occurs
     * @throws  IllegalArgumentException if any one of the arguments is null
     *
     * @see     java.net.URLConnection#setUseCaches(boolean)
     * @see     java.net.URLConnection#getUseCaches()
     * @see     java.net.URLConnection#setDefaultUseCaches(boolean)
     * @see java.net.URLConnection#getDefaultUseCaches()
     */
    public synchronized CacheResponse get(final URI uri, String rqstMethod,
            Map requestHeaders) throws IOException {
	System.out.println("get: " + uri);
	Thread.currentThread().dumpStack();
	return null;
    }
    
    /**
     * The protocol handler calls this method after a resource has
     * been retrieved, and the ResponseCache must decide whether or
     * not to store the resource in its cache. If the resource is to
     * be cached, then put() must return a CacheRequest object which
     * contains a WriteableByteChannel that the protocol handler will
     * use to write the resource into the cache. If the resource is
     * not to be cached, then put must return null.
     *
     * @param uri a <code>URI</code> used to reference the requested
     *            network resource
     * @param conn - a URLConnection instance that is used to fetch
     *            the response to be cached
     * @returns a <code>CacheRequest</code> for recording the
     *            response to be cached. Null return indicates that
     *            the caller does not intend to cache the response.
     * @throws IOException if an I/O error occurs
     * @throws IllegalArgumentException if any one of the arguments is
     *            null
     */
    public synchronized CacheRequest put(URI uri, URLConnection conn)
    throws IOException {
	System.out.println("put: " + uri);
	Thread.currentThread().dumpStack();
        URL url = uri.toURL();
        return new DeployCacheRequest(url, conn);
        
    }
}

class DeployByteArrayOutputStream extends java.io.ByteArrayOutputStream {
    
    private URL _url;
    private URLConnection _conn;
    
    DeployByteArrayOutputStream(URL url, URLConnection conn) {
        _url = url;
        _conn = conn;
    }
    
 
    public void close() throws IOException {
      
	System.out.println("contentLength: " + _conn.getContentLength());
        System.out.println("byte array size: " + size());
	if ( _conn.getContentLength() == size()) {
	    System.out.println("TEST PASSED");
	} else {
	    System.out.println("TEST FAILED");
	}
        super.close();
    }
}

class DeployCacheRequest extends java.net.CacheRequest {
    
    private URL _url;
    private URLConnection _conn;
    private boolean _downloading = false;
    
    DeployCacheRequest(URL url, URLConnection conn) {
	System.out.println("DeployCacheRequest ctor for: " + url);
        _url = url;
        _conn = conn;
    }
    
    public void abort() {
	System.out.println("abort called");
    }
    
    public OutputStream getBody() throws IOException {
	System.out.println("getBody called");
        return new DeployByteArrayOutputStream(_url, _conn);
    }
}



import java.net.*;
import java.io.*;

class test {

    public static void main(String[] args) {

	
	DeployCacheHandler.reset();
	
	try {
	    System.out.println("http request with cache hander");
	    URL u = new URL("http://localhost/6549146/SASB.mov");
	    URLConnection conn = u.openConnection();
	   
	    InputStream is = null;
                    try {
                        // this calls into DeployCacheHandler.get
                        byte[] buf = new byte[8192];
                        is = new BufferedInputStream(conn.getInputStream());

			is.skip(1000);

                        while (is.read(buf) != -1) {
                        }
                        
                    } finally {
                        if (is != null) {
                            // this calls into DeployCacheHandler.put
                            // DeployCacheHandler.put will check if the resource
                            // should be cached
                            is.close();
                        }
                    }
	  
	} catch (Exception e) {
	    e.printStackTrace();
	}


    }
}

                                    

Comments
SUGGESTED FIX

I found a fix in the networking code to fix the issue.

Basically the problem is sun.net.www.http.HttpURLConnection.HttpInputStream does not override skip method to handle the case when there is a cache request.  In this case, the InputStream is actually a KeepAliveStream (which also does not override skip), and delegates back to the skip method of MeteredStream.  As you can see in the MeteredStream implementation, it does not handle any cache request.  Therefore the skipped data will be not cached.

The fix I have is to instead implement skip in sun.net.www.http.HttpURLConnection.HttpInputStream.  Basically I just used the implementation of InputStream.skip.

   public long skip(long n)
       throws IOException {
                                     long remaining = n;
           int nr;
                        byte[] localSkipBuffer = new byte[2048];
                     if (n <= 0) {
               return 0;
           }
                     while (remaining > 0) {
               nr = read(localSkipBuffer, 0,
                       (int) Math.min(2048, remaining));
               if (nr < 0) {
                   break;
               }
               remaining -= nr;
           }
                        return n - remaining;
           // return ret;
       }


This will fix the problem, and the cache request will write out the complete file, including the skipped bytes.   But then of course this might not be the best/efficient fix, since we are not really skipping/seeking, we are just reading the bytes in.
                                     
2007-04-26
EVALUATION

skip() must read the data and consume it
                                     
2007-04-30



Hardware and Software, Engineered to Work Together