Bug ID: JDK-4630104 REGRESSION: JVM crash on unexpected EOF/SIGPIPE

JDK-4630104 : REGRESSION: JVM crash on unexpected EOF/SIGPIPE

Type: Bug
Component: hotspot
Sub-Component: runtime
Affected Version: 1.4.0

Priority: P4
Status: Closed
Resolution: Won't Fix
OS: linux
CPU: x86

Submitted: 2002-01-29
Updated: 2005-04-01
Resolved: 2003-01-17

Related Reports

Relates :

JDK-5101391 - JVM abort on unexpected signal

Description


Name: rmT116609			Date: 01/29/2002


java version "1.4.0-rc"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.0-rc-b91)
Java HotSpot(TM) Client VM (build 1.4.0-rc-b91, mixed mode)

alany@localhost:~$ uname -a
Linux localhost.localdomain 2.4.2-2 #1 Sun Apr 8 20:41:30
EDT 2001 i686 unknown
alany@localhost:~$ cat /etc/*release
Red Hat Linux release 7.1 (Seawolf)
alany@localhost:~$ rpm -query glibc
glibc-2.2.2-10



A DESCRIPTION OF THE PROBLEM :
The 1.3.x and 1.4.0-beta3 JVMs crash with a signal 11 in the
JVM_handle_linux_signal function when a write is attempted
on a remotely closed socket.  The 1.2.2 version works as
expected throwing an IOException.

The issue was first discovered in an Application server
who's JVM would die whenever a web client failed to
completely drain the servlet output buffer.

REGRESSION.  Last worked in version 1.2.2

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. compile attached class
2. execute it
3. telnet to port 1234 on localhost
4. (chars will be received)
5. close connection (^]q)
6. JVM will crash

EXPECTED VERSUS ACTUAL BEHAVIOR :
I expect the write method to throw an IOException.
Instead the JVM crashes with a signal 11 (segfault) in the
JVM_handle_linux_signal function.

ERROR MESSAGES/STACK TRACES THAT OCCUR :

Crash output:

Unexpected Signal : 11 occurred at PC=0x40343023
Function=JVM_handle_linux_signal+0x93
Library=/usr/java/j2sdk1.4.0/jre/lib/i386/client/libjvm.so

Current Java thread:
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
        at
java.net.SocketOutputStream.write(SocketOutputStream.java:105)
        at TestServ.main(TestServ.java:9)

Dynamic libraries:
08048000-0804d000 r-xp 00000000 03:03 356855
/usr/java/j2sdk1.4.0/bin/java
0804d000-0804e000 rw-p 00004000 03:03 356855
/usr/java/j2sdk1.4.0/bin/java
40000000-40016000 r-xp 00000000 03:03 48453      /lib/ld-2.2.2.so
40016000-40017000 rw-p 00015000 03:03 48453      /lib/ld-2.2.2.so
40018000-40021000 r-xp 00000000 03:03 131547
/usr/java/j2sdk1.4.0/jre/lib/i386/native_threads/libhpi.so
40021000-40022000 rw-p 00008000 03:03 131547
/usr/java/j2sdk1.4.0/jre/lib/i386/native_threads/libhpi.so
40022000-40023000 r--p 00000000 03:03 16118
/usr/lib/locale/en_US/LC_IDENTIFICATION
40023000-40030000 r-xp 00000000 03:03 80638
/lib/i686/libpthread-0.9.so
40030000-40038000 rw-p 0000c000 03:03 80638
/lib/i686/libpthread-0.9.so
40038000-4003b000 r-xp 00000000 03:03 48466      /lib/libdl-2.2.2.so
4003b000-4003c000 rw-p 00002000 03:03 48466      /lib/libdl-2.2.2.so
4003c000-40162000 r-xp 00000000 03:03 80634      /lib/i686/libc-2.2.2.so
40162000-40168000 rw-p 00125000 03:03 80634      /lib/i686/libc-2.2.2.so
4016c000-4042f000 r-xp 00000000 03:03 276158
/usr/java/j2sdk1.4.0/jre/lib/i386/client/libjvm.so
4042f000-40577000 rw-p 002c2000 03:03 276158
/usr/java/j2sdk1.4.0/jre/lib/i386/client/libjvm.so
4058b000-4059e000 r-xp 00000000 03:03 48471      /lib/libnsl-2.2.2.so
4059e000-405a0000 rw-p 00012000 03:03 48471      /lib/libnsl-2.2.2.so
405a2000-405d6000 r-xp 00000000 03:03 257620
/usr/lib/libstdc++-2-libc6.1-1-2.9.0.so
405d6000-405e2000 rw-p 00033000 03:03 257620
/usr/lib/libstdc++-2-libc6.1-1-2.9.0.so
405e4000-40607000 r-xp 00000000 03:03 80636      /lib/i686/libm-2.2.2.so
40607000-40608000 rw-p 00022000 03:03 80636      /lib/i686/libm-2.2.2.so
40608000-40619000 r-xp 00000000 03:03 276181
/usr/java/j2sdk1.4.0/jre/lib/i386/libverify.so
40619000-4061b000 rw-p 00010000 03:03 276181
/usr/java/j2sdk1.4.0/jre/lib/i386/libverify.so
4061b000-4063c000 r-xp 00000000 03:03 276169
/usr/java/j2sdk1.4.0/jre/lib/i386/libjava.so
4063c000-4063e000 rw-p 00020000 03:03 276169
/usr/java/j2sdk1.4.0/jre/lib/i386/libjava.so
4063f000-40653000 r-xp 00000000 03:03 276182
/usr/java/j2sdk1.4.0/jre/lib/i386/libzip.so
40653000-40656000 rw-p 00013000 03:03 276182
/usr/java/j2sdk1.4.0/jre/lib/i386/libzip.so
40656000-41ccd000 r--s 00000000 03:03 67144
/usr/java/j2sdk1.4.0/jre/lib/rt.jar
41d0f000-41d26000 r--s 00000000 03:03 67145
/usr/java/j2sdk1.4.0/jre/lib/sunrsasign.jar
41d26000-41d94000 r--s 00000000 03:03 67138
/usr/java/j2sdk1.4.0/jre/lib/jsse.jar
41d94000-41da7000 r--s 00000000 03:03 67137
/usr/java/j2sdk1.4.0/jre/lib/jce.jar
41da7000-42028000 r--s 00000000 03:03 67126
/usr/java/j2sdk1.4.0/jre/lib/charsets.jar
4c2d3000-4c2ee000 r--p 00000000 03:03 466841
/usr/lib/locale/en_US/LC_CTYPE
4c2ee000-4c2ef000 r--p 00000000 03:03 16119
/usr/lib/locale/en_US/LC_MEASUREMENT
4c2ef000-4c2f0000 r--p 00000000 03:03 16122
/usr/lib/locale/en_US/LC_TELEPHONE
4c2f0000-4c2f1000 r--p 00000000 03:03 16117
/usr/lib/locale/en_US/LC_ADDRESS
4c2f1000-4c2f2000 r--p 00000000 03:03 16120
/usr/lib/locale/en_US/LC_NAME
4c2f2000-4c2f3000 r--p 00000000 03:03 16121
/usr/lib/locale/en_US/LC_PAPER
4c2f3000-4c2f4000 r--p 00000000 03:03 193208
/usr/lib/locale/en_US/LC_MESSAGES/SYS_LC_MESSAGES
4c2f4000-4c2f5000 r--p 00000000 03:03 321932
/usr/lib/locale/en_US/LC_MONETARY
4c2f5000-4c2fb000 r--p 00000000 03:03 273684
/usr/lib/locale/en_US/LC_COLLATE
4c2fb000-4c2fc000 r--p 00000000 03:03 16123
/usr/lib/locale/en_US/LC_TIME
4c2fc000-4c2fd000 r--p 00000000 03:03 193198
/usr/lib/locale/en_US/LC_NUMERIC
4c2fd000-4c2ff000 r-xp 00000000 03:03 452038
/usr/lib/sasl/libanonymous.so.1.0.15
4c2ff000-4c300000 rw-p 00001000 03:03 452038
/usr/lib/sasl/libanonymous.so.1.0.15
4c300000-4c303000 r-xp 00000000 03:03 452041
/usr/lib/sasl/libcrammd5.so.1.0.15
4c303000-4c304000 rw-p 00002000 03:03 452041
/usr/lib/sasl/libcrammd5.so.1.0.15
4c304000-4c306000 r-xp 00000000 03:03 452047
/usr/lib/sasl/liblogin.so.0.0.5
4c306000-4c307000 rw-p 00001000 03:03 452047
/usr/lib/sasl/liblogin.so.0.0.5
4c308000-4c312000 r-xp 00000000 03:03 48487
/lib/libnss_files-2.2.2.so
4c312000-4c313000 rw-p 00009000 03:03 48487
/lib/libnss_files-2.2.2.so
4c313000-4c31d000 r-xp 00000000 03:03 48495
/lib/libnss_nisplus-2.2.2.so
4c31d000-4c31f000 rw-p 00009000 03:03 48495
/lib/libnss_nisplus-2.2.2.so
4c31f000-4c3f3000 r-xp 00000000 03:03 49298
/lib/libnss_ldap-2.2.2.so
4c3f3000-4c400000 rw-p 000d3000 03:03 49298
/lib/libnss_ldap-2.2.2.so
4c408000-4c47f000 r-xp 00000000 03:03 48510      /lib/libdb-3.1.so
4c47f000-4c481000 rw-p 00076000 03:03 48510      /lib/libdb-3.1.so
4c481000-4c490000 r-xp 00000000 03:03 48500      /lib/libresolv-2.2.2.so
4c490000-4c491000 rw-p 0000e000 03:03 48500      /lib/libresolv-2.2.2.so
4c494000-4c49b000 r-xp 00000000 03:03 452044
/usr/lib/sasl/libdigestmd5.so.0.0.17
4c49b000-4c49c000 rw-p 00006000 03:03 452044
/usr/lib/sasl/libdigestmd5.so.0.0.17
4c49c000-4c49e000 r-xp 00000000 03:03 452050
/usr/lib/sasl/libplain.so.1.0.14
4c49e000-4c49f000 rw-p 00001000 03:03 452050
/usr/lib/sasl/libplain.so.1.0.14
4c49f000-4c4a2000 r-xp 00000000 03:03 48484
/lib/libnss_dns-2.2.2.so
4c4a2000-4c4a4000 rw-p 00002000 03:03 48484
/lib/libnss_dns-2.2.2.so
4c4a4000-4c4a6000 r--s 00000000 03:03 2776
/usr/java/j2sdk1.4.0/jre/lib/ext/dnsns.jar
4c4a7000-4c555000 r-xp 00000000 03:03 257862
/usr/lib/libcrypto.so.0.9.6
4c555000-4c560000 rw-p 000ad000 03:03 257862
/usr/lib/libcrypto.so.0.9.6
4c563000-4c569000 r-xp 00000000 03:03 48464      /lib/libcrypt-2.2.2.so
4c569000-4c56a000 rw-p 00005000 03:03 48464      /lib/libcrypt-2.2.2.so
4c591000-4c598000 r-xp 00000000 03:03 49162      /lib/libpam.so.0.74
4c598000-4c599000 rw-p 00006000 03:03 49162      /lib/libpam.so.0.74
4c7c8000-4c7d6000 r--s 00000000 03:03 2777
/usr/java/j2sdk1.4.0/jre/lib/ext/ldapsec.jar
4c7d6000-4c879000 r--s 00000000 03:03 2778
/usr/java/j2sdk1.4.0/jre/lib/ext/localedata.jar
4c879000-4c896000 r--s 00000000 03:03 2779
/usr/java/j2sdk1.4.0/jre/lib/ext/sunjce_provider.jar
4c896000-4c8a8000 r-xp 00000000 03:03 276178
/usr/java/j2sdk1.4.0/jre/lib/i386/libnet.so
4c8a8000-4c8a9000 rw-p 00011000 03:03 276178
/usr/java/j2sdk1.4.0/jre/lib/i386/libnet.so

Local Time = Thu Jan 24 17:04:51 2002
Elapsed Time = 9
#
# HotSpot Virtual Machine Error : 11
# Error ID : 4F530E43505002D3
# Please report this error at
# http://java.sun.com/cgi-bin/bugreport.cgi
#
# Java VM: Java HotSpot(TM) Client VM (1.4.0-rc-b91 mixed mode)
#
# An error report file has been saved as hs_err_pid11192.log.
# Please refer to the file for further information.
#
Aborted (core dumped)

Expected behaviour of the test case:

An exception and not a JVM crash.

Here is the output from 1.2.2:

java version "1.2.2"
Classic VM (build Linux_JDK_1.2.2_FCS, native threads, sunwjit)

Exception in thread "main" java.io.IOException: Broken pipe
        at java.net.SocketOutputStream.socketWrite(Native Method)
        at java.net.SocketOutputStream.write(SocketOutputStream.java, Compiled Code)
        at TestServ.main(TestServ.java, Compiled Code)


This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.util.*;
import java.net.*;
public class TestServ {
        public static void main(String args[]) throws Exception {
                ServerSocket listen = new ServerSocket(1234);
                Socket s = listen.accept();
                byte b = (byte)'a';
                while (true) {
                        s.getOutputStream().write(b);
                        if (b == (byte)'z') {
                                b = (byte)'a';
                        } else {
                                b++;
                        }
                }
        }
}

---------- END SOURCE ----------

Release Regression From : 1.2.2
The above release value was the last known release where this 
bug was knwon to work. Since then there has been a regression.

(Review ID: 138637) 
======================================================================

Comments

WORK AROUND Preload libjsig.so before running java code. libjsig.so is in <jre-path>/lib/i386. For example, with bash shell, I can do: export LD_PRELOAD=/export/jdk1.4.1/jre/lib/i386/libjsig.so ###@###.### 2003-01-16

16-01-2003

EVALUATION For the last several months, a couple of people offered help on this bug. But it turned out their problems were related to system setup (one person had a bad memory chip, another person's problem was due to signal conflicts between JVM and JNI libraries). Now I believe the original problem was also due to system configurations (though the orignal bug submitter never replied our inquires). I am inclined to close this bug as will-not-fix. If you are still seeing the crash, please contact me at hui dot huang at sun dot com. Or you could file a new bug with detailed steps on how to reproduce the problem. ###@###.### 2002-10-27 not-reproducible is probably a more appropriate category. Anyway, contact me if you are still seeing the problem. ###@###.### 2002-10-28 ------------------------------------------------- With help from ###@###.###, I can now reproduce the crash by modifying /etc/nsswitch.conf and adding "ldap" before "files" for passwd. reopen. ###@###.### 2003-01-16 ------------------------------------------------- The problem is in libnss_ldap.so. _nss_ldap_enter tries to block SIGPIPE by changing the SIGPIPE handler to SIG_IGN, and in _nss_ldap_leave, the original signal handler (in this case, the JVM signalHandler) is restored by using sigset() or signal(). We set up our SIGPIPE handler first by sigaction() and with the SA_SIGINFO flag set, so we get both siginfo and ucontext in the signal handler; but sigset() and/or signal() used by nss_ldap can only restore signal handlers without siginfo/ucontext. As the result, when a SIGPIPE is delivered after nss_ldap has changed and "restored" the SIGPIPE handler, our signal handler is invoked with only one argument (i.e. signal number), but we expect three arguments (signal number, siginfo and ucontext). We will crash when we try to figure out the PC from ucontext (JVM_handle_linux_signal+0x93 is "pc = (address) uc->uc_mcontext.gregs[REG_EIP];"). nss_ldap should use sigaction() to change and restore signal handlers. libjsig.so is designed to work around this type of problems. libjsig.so works by interposing signal(), sigset() and sigaction() functions to make sure JVM can still function correctly when any of the functions is used to change a signal that is also being used by JVM. Note from the ChangeLog, nss_ldap from version 200 started to use sigprocmask() to block SIGPIPE. This should fix the problem. If you don't have the latest nss_ldap (Redhat 8.1 beta has nss_ldap-202, therefore does not have this problem), you can preload libjsig.so to work around the problem, please see the Workaround section. We could improve our fatal error handler by printing a message about loading libjsig.so if this type of crash is detected (if it is possible), that should be tracked as 4515367. Close this bug as will-not-fix. ###@###.### 2003-01-16 ------------------------------------ Please see also 5101391. We did make some changes in VM to make it more tolerant of such errors. ###@###.### 2005-04-01 18:28:45 GMT

16-01-2003