Bug ID: JDK-4800337 java 1.4.1's java_vm dumps core under mozilla 1.2 and libumem

Type: Bug
Component: deploy
Sub-Component: plugin
Affected Version: 1.4.1_01

Priority: P2
Status: Resolved
Resolution: Fixed
OS: solaris_10
CPU: sparc

Submitted: 2003-01-08
Updated: 2003-01-17
Resolved: 2003-01-17

Other
1.4.2 b14Fixed

[dep, 08Jan2003]

  In fact, it dumps core on its own:

    $ /usr/j2se/jre/bin/java_vm
    Segmentation Fault(coredump)
    139 $ pstack core
    core 'core' of 125339:  /usr/j2se/jre/bin/java_vm
     ff2b0fb0 strlen   (12598, 0, ffbfba98, 0, 0, 0) + 80
     ff30527c snprintf (24160, c8, 12598, 0, 81010100, ff00) + 68
     000119bc SetClassPath (0, 0, ff270000, ff3c2b14, 0, ff3d7820) + 34
     000111c8 main     (1, ffbff98c, ffbff994, 22800, 0, 0) + 104
     00010d80 _start   (0, 0, 0, 0, 0, 0) + 108

  When running under mozilla 1.2, the stack trace is:

    core '/var/cores/core.java_vm.124767@1042058483' of 124767:     java_vm
     ff2f0814 memset   (31303431, 36343437, 39340a2d, 686f7374, ...) + 34

  (memset is a leaf function called from main)
  
  Interestingly, it seems to work fine under mozilla 1.1.

CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: mantis-beta FIXED IN: mantis-beta INTEGRATED IN: mantis-b14 mantis-beta

14-06-2004

EVALUATION Which platform are you running with? Do you mean Mozilla fails to launch? ###@###.### 2003-01-08 [dep, 08Jan2003] I'm running on OS bits roughly equivalent to those found in s10_25. The SUNWj3rt package version is "1.4.1,REV=2002.09.30.05.41". Mozilla will launch, but it dies when I log into sunreserve with the following error: INTERNAL ERROR on Browser End: Could not read ack from browser System error?:: Resource temporarily unavailable which is accompanied by java_vm crashing. Upon further investigation, there seem to be two problems: 1) Running /usr/j2se/jre/bin/java_vm on its own, while perhaps a silly thing to do, crashes. 2) Perhaps related to 1, preloading libumem (which the /home/kg/bin/mozilla1.2 wrapper does) causes mozilla 1.2's invocation of java_vm to fail. From either one of these we can conclude that there is some sort of bug in the java_vm binary, probably related to its memory management. ###@###.### 2003-01-08 Are you building Mozilla yourself? In other word, how do you get your mozilla binary? [jwadams 2003-01-08] The crash under mozilla is quite interesting. There are a sequence of 10 calls to AddOption before the crash. The first 9 have the arguments: 0x22b9c: -DtrustProxy=true 0x22bb0: -Xverify:remote 0x4deb0: -Djava.class.path=/home/kg-arch/j2se-1.4.1_01-b01/jre/classes 0x22a68: -Djava.protocol.handler.pkgs=sun.plugin.net.protocol 0x55f08: -Xbootclasspath/a:/home/kg-arch/j2se-1.4.1_01-b01/jre/lib/javapl 0x59ea8: -Djavaplugin.lib=/home/kg-arch/j2se-1.4.1_01-b01/jre/lib/sparc/l 0x22bd4: -Dmozilla.workaround=true 0xffbfd860: -Djavaplugin.nodotversion=141_01 0xffbfd760: -Djavaplugin.version=1.4.1_01 And the last is passed garbage, which starts with a part of the contents of the mount table, and ends with a -Djavaplugin.vm.options= string: 0xffbfb7c0: 59 -hosts /net autofs indirect,nosuid,ignore,nobrowse,dev=4100012 10416508 59 coil:vold(pid100360) /vol nfs ignore,dev=4140002 1041650868 jurassic.eng:/export/rahi/dep /home/dep nfs intr,nosuid,noquota,xatt r,dev=4140001 1041650870 ... udmpk16c,udmpk16a,udmpk16e:/export/dist /usr/dist nfs ro,hard,noquota, intr,nosuid,xattr,dev=41400a2 1042095602 kg.eng:/aux0/kg /home/kg nfs intr,nosuid,noquota,xattr,dev=41400a3 1042095719 -Djavaplugin.vm.options=-DtrustProxy=true -Xverify:remote -Djava.class.path=/hom e/kg-arch/j2se-1.4.1_01-b01/jre/classes -Djava.protocol.handler.pkgs=sun.plugin. net.protocol -Xbootclasspath/a:/home/kg-arch/j2se-1.4.1_01-b01/jre/lib/javaplugi n.jar:/home/kg-arch/j2se-1.4.1_01-b01/jre/lib/javaplugin_l10n.jar -Djavaplugin.l ib=/home/kg-arch/j2se-1.4.1_01-b01/jre/lib/sparc/libjavaplugin_jni.so -Dmozilla. workaround=true -Djavaplugin.nodotversion=141_01 -Djavaplugin.version=1.4.1_01 In main, there is a line of code that looks something like: (from the disassembly) main() { char buf[size]; ... strcat(buf, "-Djavaplugin.vm.options="); for (something) { ... strcat(buf, a); strcat(buf, b); ... } AddOption(buf); memset(&var, 0, 16); ... } It appears to me that the first call to strcat(3C) (strcat(buf, "-Djavaplugin.vm.options=") is expecting the buffer to start with a null byte, but it doesn't. Either: 1. Something (most probably the code that reads the mount table) is overflowing the stack, negating an earlier initialization of the buffer, or 2. The code is incorrect, and that strcat() should really be a strcpy(). In any case, this extremely long string causes a stack smash down in AddOption. when the function returns, it gets its frame pointer from the bogus data, and tries to memset a 16-byte local variable to 0. The bad frame pointer means that the address computed is off in la-la land, and the program coredumps on the bad address. ###@###.### 2003-01-09 Please file another bug for java_vm problem. We have to evaluate it and see if we need to fix it. For the problem you filed here and for the code you put here: main() { char buf[size]; ... strcat(buf, "-Djavaplugin.vm.options="); for (something) { ... strcat(buf, a); strcat(buf, b); ... } AddOption(buf); memset(&var, 0, 16); ... Probably we need to check whether the buf is NULL or not before we use it just for safe. The size here for our program is 8096. ###@###.### 2003-01-09 Okay, I think the reason causing Solaris 10 crash is due to not initializing the buffer. The proposed fix is to initialize the buffer before using it.

09-01-2003

Relates :	JDK-4802619 - plugin exits mozilla on slightest error
Relates :	JDK-4801172 - java_vm crashes when invoked directly