United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-4455047 : Caching in URLConnection is not working properly

Details
Type:
Bug
Submit Date:
2001-05-06
Status:
Resolved
Updated Date:
2003-04-12
Project Name:
JDK
Resolved Date:
2002-08-30
Component:
core-libs
OS:
windows_2000
Sub-Component:
java.net
CPU:
x86
Priority:
P4
Resolution:
Fixed
Affected Versions:
1.3.0
Fixed Versions:
1.4.2 (mantis)

Related Reports

Sub Tasks

Description

Name: boT120536			Date: 05/06/2001


java version "1.3.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0-C)
Java HotSpot(TM) Client VM (build 1.3.0-C, mixed mode)

One URL is:
http://sports.yahoo.com/mlbstats/stats_n_regular_season.txt

Although I am getting two pages within the same program, the situation
you mentioned in that bug report does not apply here, because even after I
exited the application and rerun it, I still get the older version, until 24
hours has passed since I got the page. Looks like it is caching for 24 hours
without my control.


import java.net.*;
import java.io.*;
import java.util.*;

public class GetStats
{

    private String html;

    public GetStats(String url)
    {
        Object o;
    	int size;
    	Properties prop = System.getProperties();
    	URL u;
        HttpURLConnection c;
	byte[] arr = new byte[512];
        try
		{
		String propStr=prop.getProperty("proxySet");
		if (propStr!=null && propStr.equals("true"))

        		u = new URL("http", prop.getProperty("proxyhost"), url);
        	else
        		u = new URL(url);
		Object o1= u.openConnection();
		c = (HttpURLConnection) o1;
		c.setDefaultUseCaches(false);
		c.setUseCaches(false);
        	o = c.getContent();
        }
        catch (MalformedURLException e)
        {
        	System.out.println("MException");
        	return;
        }
        catch (IOException e)
        {
        	System.out.println("IException:" + e.toString());
        	return;
        }
        try
        {
               BufferedInputStream stream=new BufferedInputStream ((InputStream)
o);
		size = stream.available();
		StringBuffer htmlBuf = new StringBuffer();

		int r;
		{
			int i;
			int sum=0;
			for (i=0; true; i++)
			{
				Thread.sleep(100);
				r=stream.read(arr, 0, 512);
				sum += r;
				if (r==-1)
					break;
				htmlBuf.append(new String(arr, 0, r));
				if (i==0)
					System.out.println(new String(arr,0,r));
			}
		}
		c.disconnect();
		stream.close();
		html = htmlBuf.toString();  // String html stores the page
		}
		catch (Exception e)
		{
			System.out.println("exception: " + e.toString());
			e.printStackTrace();
		}
   }

    public String getHtml()
    {
    	return(html);
    }

    public static void main(String[] arg) throws Exception
    {
	GetStats n;
        n = new
GetStats("http://sports.yahoo.com/mlbstats/stats_n_regular_season.txt");
        System.out.println("First Page: " + n.getHtml());
        n = new
GetStats("http://sports.yahoo.com/mlbstats/stats_p_regular_season.txt");
        System.out.println("Second Page: " + n.getHtml());
     }
}

Below is additional info I posted in my earlier postings:
===============================================================
Hi Allen Castaban,

You forgot some stuff.
>- Exact steps to reproduce the problem.
>- Complete Java source code that demonstrates the problem.

I still need an example URL (use Lycos/Yahoo/Sun etc..)
and complete and concise source code that shows the issue.
If it is more than 100 lines you are doing somthing wrong.
The complete source allows us to easily verify that you have an error.
And, allows us to test against different versions of the JDK.

---------------- Original Bug Report-------------------

OK, I did some more testing on this problem. It definitely exists in Win2000, I
think it is working OK in Win98 (I did not do as much testing). As you see from
my code below, I have everything mentioned in that forum Brett mentioned (which
I did have to start with anyway), it still does not work. Although I have
caching turned off, it still remembers (and returns) the previous page, exactly
for 24 hours since I got the first page, then I started getting the new page.
Where is this cache located anyway, and how can I flush it?

category : java
subcategory : classes_net
release : 1.3
type : bug
synopsis : Caching in URLConnection is not working properly
description : java version "1.3.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0-C)
Java HotSpot(TM) Client VM (build 1.3.0-C, mixed mode)

I am trying to get a page from the net, using HttpURLConenction class. The page
updates once daily. After I run the program first time, I got the file OK.
Afterwards when I run the program (say the next day) I do not get the updated
file, I got the old file. Looks like it is cached somewhere and the cache
mechanism does not work properly as the file has been updated in the server. I
can get an updated version from Netscape. I tried to turn off caching (call
setUseCaches(false) method), no effect. I am running behind a firewall, but no
proxy caching there, as the browser can get the updated page.  I also found
other people are having the same problems. Here is a link:
http://forum.java.sun.com/read/16789759/q_UPnoGAQUZ8AAbNm#LR

I also like to know if VM is acching HTTP connections, where the cache files
are, and/or how to flush out the cache (Rebooting does not work). There is a
possibility that this is working in Win98. Also I think it works after a while
(a while after the page has been updated in the server), say 24 hours maybe
(Review ID: 123498) 
======================================================================

                                    

Comments
EVALUATION

	If getUseCaches() returns false we should be sending the request
with the "no-cache" request header values for Cache-Control and Pragma.

	In the meantime, a user of the URLConnection api can do
this themselves by calling setRequestProperty("Cache-Control", "no-cache")
and setRequestProperty("Pragma", "no-cache") on the URLConnection object.


jeff.nisewanger@Eng 2001-05-07
                                     
2001-05-07
PUBLIC COMMENTS

	The value returned by URLConnection.getUseCaches() is ignored
in the SDK default http handler.
                                     
2004-06-10
WORK AROUND



Name: boT120536			Date: 05/06/2001


None
======================================================================
                                     
2004-06-11
CONVERTED DATA

BugTraq+ Release Management Values

COMMIT TO FIX:
mantis
mantis-b02

FIXED IN:
mantis
mantis-b02

INTEGRATED IN:
mantis
mantis-b02


                                     
2004-06-14



Hardware and Software, Engineered to Work Together