JDK-4455047 : Caching in URLConnection is not working properly
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.net
  • Affected Version: 1.3.0
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: windows_2000
  • CPU: x86
  • Submitted: 2001-05-06
  • Updated: 2003-04-12
  • Resolved: 2002-08-30
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
1.4.2 mantisFixed
Description

Name: boT120536			Date: 05/06/2001


java version "1.3.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0-C)
Java HotSpot(TM) Client VM (build 1.3.0-C, mixed mode)

One URL is:
http://sports.yahoo.com/mlbstats/stats_n_regular_season.txt

Although I am getting two pages within the same program, the situation
you mentioned in that bug report does not apply here, because even after I
exited the application and rerun it, I still get the older version, until 24
hours has passed since I got the page. Looks like it is caching for 24 hours
without my control.


import java.net.*;
import java.io.*;
import java.util.*;

public class GetStats
{

    private String html;

    public GetStats(String url)
    {
        Object o;
    	int size;
    	Properties prop = System.getProperties();
    	URL u;
        HttpURLConnection c;
	byte[] arr = new byte[512];
        try
		{
		String propStr=prop.getProperty("proxySet");
		if (propStr!=null && propStr.equals("true"))

        		u = new URL("http", prop.getProperty("proxyhost"), url);
        	else
        		u = new URL(url);
		Object o1= u.openConnection();
		c = (HttpURLConnection) o1;
		c.setDefaultUseCaches(false);
		c.setUseCaches(false);
        	o = c.getContent();
        }
        catch (MalformedURLException e)
        {
        	System.out.println("MException");
        	return;
        }
        catch (IOException e)
        {
        	System.out.println("IException:" + e.toString());
        	return;
        }
        try
        {
               BufferedInputStream stream=new BufferedInputStream ((InputStream)
o);
		size = stream.available();
		StringBuffer htmlBuf = new StringBuffer();

		int r;
		{
			int i;
			int sum=0;
			for (i=0; true; i++)
			{
				Thread.sleep(100);
				r=stream.read(arr, 0, 512);
				sum += r;
				if (r==-1)
					break;
				htmlBuf.append(new String(arr, 0, r));
				if (i==0)
					System.out.println(new String(arr,0,r));
			}
		}
		c.disconnect();
		stream.close();
		html = htmlBuf.toString();  // String html stores the page
		}
		catch (Exception e)
		{
			System.out.println("exception: " + e.toString());
			e.printStackTrace();
		}
   }

    public String getHtml()
    {
    	return(html);
    }

    public static void main(String[] arg) throws Exception
    {
	GetStats n;
        n = new
GetStats("http://sports.yahoo.com/mlbstats/stats_n_regular_season.txt");
        System.out.println("First Page: " + n.getHtml());
        n = new
GetStats("http://sports.yahoo.com/mlbstats/stats_p_regular_season.txt");
        System.out.println("Second Page: " + n.getHtml());
     }
}

Below is additional info I posted in my earlier postings:
===============================================================
Hi Allen Castaban,

You forgot some stuff.
>- Exact steps to reproduce the problem.
>- Complete Java source code that demonstrates the problem.

I still need an example URL (use Lycos/Yahoo/Sun etc..)
and complete and concise source code that shows the issue.
If it is more than 100 lines you are doing somthing wrong.
The complete source allows us to easily verify that you have an error.
And, allows us to test against different versions of the JDK.

---------------- Original Bug Report-------------------

OK, I did some more testing on this problem. It definitely exists in Win2000, I
think it is working OK in Win98 (I did not do as much testing). As you see from
my code below, I have everything mentioned in that forum Brett mentioned (which
I did have to start with anyway), it still does not work. Although I have
caching turned off, it still remembers (and returns) the previous page, exactly
for 24 hours since I got the first page, then I started getting the new page.
Where is this cache located anyway, and how can I flush it?

category : java
subcategory : classes_net
release : 1.3
type : bug
synopsis : Caching in URLConnection is not working properly
description : java version "1.3.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0-C)
Java HotSpot(TM) Client VM (build 1.3.0-C, mixed mode)

I am trying to get a page from the net, using HttpURLConenction class. The page
updates once daily. After I run the program first time, I got the file OK.
Afterwards when I run the program (say the next day) I do not get the updated
file, I got the old file. Looks like it is cached somewhere and the cache
mechanism does not work properly as the file has been updated in the server. I
can get an updated version from Netscape. I tried to turn off caching (call
setUseCaches(false) method), no effect. I am running behind a firewall, but no
proxy caching there, as the browser can get the updated page.  I also found
other people are having the same problems. Here is a link:
http://forum.java.sun.com/read/16789759/q_UPnoGAQUZ8AAbNm#LR

I also like to know if VM is acching HTTP connections, where the cache files
are, and/or how to flush out the cache (Rebooting does not work). There is a
possibility that this is working in Win98. Also I think it works after a while
(a while after the page has been updated in the server), say 24 hours maybe
(Review ID: 123498) 
======================================================================

Comments
CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: mantis mantis-b02 FIXED IN: mantis mantis-b02 INTEGRATED IN: mantis mantis-b02
14-06-2004

WORK AROUND Name: boT120536 Date: 05/06/2001 None ======================================================================
11-06-2004

PUBLIC COMMENTS The value returned by URLConnection.getUseCaches() is ignored in the SDK default http handler.
10-06-2004

EVALUATION If getUseCaches() returns false we should be sending the request with the "no-cache" request header values for Cache-Control and Pragma. In the meantime, a user of the URLConnection api can do this themselves by calling setRequestProperty("Cache-Control", "no-cache") and setRequestProperty("Pragma", "no-cache") on the URLConnection object. jeff.nisewanger@Eng 2001-05-07
07-05-2001