JDK-6745052 : SSLServerSocket file descriptor leak
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.net.ssl
  • Affected Version: 5.0,6
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux
  • CPU: x86
  • Submitted: 2008-09-05
  • Updated: 2023-11-29
  • Resolved: 2008-11-22
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other Other JDK 6 JDK 7 Other
5.0u26-rev,OpenJDK6Fixed 5.0u27Fixed 6u12Fixed 7 b41Fixed OpenJDK6Fixed
Related Reports
Duplicate :  
Description
FULL PRODUCT VERSION :
java version "1.6.0_02"
Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
Java HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing)

java version "1.6.0_04"
Java(TM) SE Runtime Environment (build 1.6.0_04-b12)
Java HotSpot(TM) 64-Bit Server VM (build 10.0-b19, mixed mode)

java version "1.5.0_14"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_14-b03)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_14-b03, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Linux myserver1 2.6.22 #2 SMP Fri Jan 4 18:21:24 JST 2008 i686 i686 i386 GNU/Linux
Linux myserver2 2.6.22 #11 SMP Thu Feb 7 04:31:44 JST 2008 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
File descriptors of TCP sockets are not released properly when using SSLServerSocket class ( especially with many instances ) on Linux systems.

If a server application ( like Jakarta Tomcat ) runs very long time, this problem will cause a 'too many open files' error and a denial of the service.

Please note that we need to use 'lsof' command instead of 'netstat' command to see whether the file descriptor leak is happening or not.
Because the leaked sockets are not binded to any TCP addresses, we can not see the sockets using 'netstat' command.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Save the source code as 'test1.java'.

2. Compile with 'javac test1.java'.

3. Run with 'java -Djavax.net.ssl.keyStore=.keystore -Djavax.net.ssl.keyStorePassword=changeit test1'
(You need your own .keystore file and password ).

4. In another Linux shell, execute 'lsof' command.
'lsof -n -p PID_OF_JAVA_TEST1'
(PID_OF_JAVA_TEST1 is the process ID of out test case )

5. We will see many lines like below.
java    1919 root  884u  sock    0,5            10398 can't identify protocol
java    1919 root  885u  sock    0,5            10399 can't identify protocol
java    1919 root  886u  sock    0,5            10418 can't identify protocol
java    1919 root  887u  sock    0,5            10420 can't identify protocol
java    1919 root  888u  sock    0,5            10422 can't identify protocol
java    1919 root  890u  sock    0,5            10443 can't identify protocol
java    1919 root  891u  sock    0,5            10444 can't identify protocol
java    1919 root  892u  sock    0,5            10445 can't identify protocol
java    1919 root  893u  sock    0,5            10446 can't identify protocol
java    1919 root  894u  sock    0,5            10447 can't identify protocol
java    1919 root  895u  sock    0,5            10448 can't identify protocol

These are the leaked file descriptors.


In addition,

6. We can also see the leaked file descriptors in '/proc/PID_OF_JAVA_TEST1/fd'

7. And in /proc/net/sockstat, these leaked file descriptors are counted as allocated TCP sockets.
For example: 'TCP: inuse 3 orphan 0 tw 0 alloc 663 mem 2'
When this java process ended, the number 'alloc 663' will be decreased.

8. If we repeat 'foo()' function more , we can see the 'too many open files 'error message.
Please change the line
for( int i = 0; i < 1000; ++i )
to
for( int i = 0; i < 2000; ++i )
and test again to see the error message.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
'lsof' command does not show too many lines of 'can't identify protocol' sockets.
ACTUAL -
Please see the 'Steps to Reproduce' field.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Exception in thread "main" java.net.SocketException: Too many open files
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:259)
        at test1.foo(test1.java:25)
        at test1.main(test1.java:15)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.net.*;
import javax.net.*;
import javax.net.ssl.*;

public class test1
{
private static ServerSocketFactory ssf;
//------------------------------------------------
public static void main( String[] args )
throws Exception
{
ssf = SSLServerSocketFactory.getDefault();
for( int i = 0; i < 1000; ++i )
        {
        foo();
        }
Thread.sleep( 1000000 );
}
//------------------------------------------------
private static void foo()
throws Exception
{
ServerSocket sSocket = ssf.createServerSocket( 0, 1 );
Socket socket1  = new Socket( "127.0.0.1", sSocket.getLocalPort() );
Socket socket2 = sSocket.accept();
sSocket.close();
socket1.close();
socket2.close();
}
//------------------------------------------------
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
I don't have.

Comments
WORK AROUND The workaround worked fine for one customer. Will mention this to the others.
13-11-2008

EVALUATION It's indirectly caused by the checkEnabledSuite, but that's not the real problem. The problem is actually in the finalizer in BaseSSLSocketImpl. The finalizer is trying to close its socket, which is throwing a NullPointerException due to the sockInput being null. As written, the finalizer catch block only grabs IOExceptions, so this is skipping the close() on super, which is leaking the fd's. super.finalize doesn't do anything with fd's. I think the fix is to change the catch to Exception, but wanted to look into it more. try { close(); } catch (IOException e1) { try { if (self == this) { super.close(); } } catch (IOException e2) { // ignore } } finally { super.finalize(); } } That's the bug, still want to look a bit closer at how everything interacts, especially whether removing the SSLSocketImpl test in the accept is a good idea or not.
16-09-2008

WORK AROUND I had an idea last night which I believe should work. In JSSE, you can layer a SSLSocket over an existing Socket. That will avoid SSLServerSocket creating the dummy Socket that is eventually leaking the file descriptors. SSLSocketFacory.createSocket(Socket, String, int, boolean); So do something like this: // Create and initialize a SSLContext, from which you obtain a // SSLSocketFactory, sslssf ServerSocket ss = new ServerSocket(port); Socket s = ss.accept(); sslssf.createSocket(s, s.getInetAddress().getHostName(), s.getPort(), false); I'm going to add this to the bug report now, so if you think that would work, I'll push that info to the customers that have reported this problem. Brad
16-09-2008