JDK-8028216 : Repeated crashes in java.util.zip.deflater.deflatebytes
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util
  • Affected Version: 6,7u25
  • Priority: P2
  • Status: Resolved
  • Resolution: Cannot Reproduce
  • OS: generic
  • CPU: generic
  • Submitted: 2013-11-12
  • Updated: 2017-03-13
  • Resolved: 2015-04-02
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7
7-poolResolved
Related Reports
Duplicate :  
Relates :  
Relates :  
Description
Customer had this issue a month ago and unfortunately at that
time there was no coredump available and only a hotspot.

The application crashed again and this time we have collected
a dataset which includes a hotspot and coredump (pkgapp output)

The customer is requesting an RCA to be completed

The dataset is located on CORES2
/cores_data/pool-0/data4/3-6908991941

The stack is as follows
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002aaaaaf1a712, pid=28152, tid=1156618560
#
# JRE version: 6.0_31-b23
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.6-b02 mixed mode
linux-amd64 compressed oops)
# Problematic frame:
# C [libzip.so+0x5712] double+0x42
/Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
C=native code)/
C [libzip.so+0x5712] double+0x42

<snip>
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J java.util.zip.Deflater.deflateBytes(J[BII)I
J java.util.zip.GZIPOutputStream.finish()V
J org.apache.coyote.http11.filters.FlushableGZIPOutputStream.finish()V
J org.apache.coyote.http11.filters.GzipOutputFilter.end()J
J org.apache.coyote.http11.InternalOutputBuffer.endRequest()V
J
org.apache.coyote.http11.AbstractHttp11Processor.action(Lorg/apache/coyote/Act
ionCode;Ljava/lang/Object;)V
J org.apache.catalina.connector.OutputBuffer.close()V
J
org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request
;Lorg/apache/coyote/Response;)V
J
org.apache.coyote.http11.AbstractHttp11Processor.process(Lorg/apache/tomcat/ut
il/net/SocketWrapper;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$Soc
ketState;
J
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Lorg/apac
he/tomcat/util/net/SocketWrapper;Lorg/apache/tomcat/util/net/SocketStatus;)Lor
g/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;
J org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run()V
J java.util.concurrent.ThreadPoolExecutor$Worker.run()V
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub


Customer is requesting a RCA to be completed.

They have not been able to reliably reproduce this issue yet
but are continuing attempts to reprocuce.

One of our senior engineers has done some analysis on the corefile / pkgapp
dataset which follows

 $ ./saenv.gen.sh
jsdb> load("/home/javateam/tools/sa.js");
jsdb> lookAtJavaThread2(8780);
Look at the 8780.stack generated.

hsdb> inspect 0x00000007032a87d8
instance of Oop for org/apache/coyote/Request @ 0x00000007032a87d8 @
0x00000007032a87d8 (size = 160)
unparsedURIMB: Oop for org/apache/tomcat/util/buf/MessageBytes @
0x00000007033105c0 Oop for org/apache/tomcat/util/buf/MessageBytes @
0x00000007033105c0
uriMB: Oop for org/apache/tomcat/util/buf/MessageBytes @ 0x00000007033105f0
Oop for org/apache/tomcat/util/buf/MessageBytes @ 0x00000007033105f0
decodedUriMB: Oop for org/apache/tomcat/util/buf/MessageBytes @
0x0000000703310620 Oop for org/apache/tomcat/util/buf/MessageBytes @
0x0000000703310620

hsdb> inspect 0x00000007033105c0
instance of Oop for org/apache/tomcat/util/buf/MessageBytes @
0x00000007033105c0 @ 0x00000007033105c0 (size = 48)
byteC: Oop for org/apache/tomcat/util/buf/ByteChunk @ 0x0000000703307900 Oop
for org/apache/tomcat/util/buf/ByteChunk @ 0x0000000703307900
charC: Oop for org/apache/tomcat/util/buf/CharChunk @ 0x00000007032fe0f0 Oop
for org/apache/tomcat/util/buf/CharChunk @ 0x00000007032fe0f0

hsdb> inspect 0x0000000703307900
instance of Oop for org/apache/tomcat/util/buf/ByteChunk @ 0x0000000703307900
0x0000000703307900 (size = 48)
_mark: 5
buff: [B @ 0x000000070349d748 Oop for [B @ 0x000000070349d748
start: 4
end: 271


(gdb) x/4gx 0x000000070349d748
0x70349d748:    0x0000000000000001      0x00010000f80001c6
0x70349d758:    0x6572432f20544547      0x74726f5074654e77

(gdb) print (char*) 0x70349d758
$3 = 0x70349d758 "GET
/CrewNetPortal/CrewAdvSearch?searchType=AdvSearch&lncriteria=begins+with&lastn
ame=&fncriteria=begins+with&firstname=david&id=&uid=&extension=&station=&mailp
ort=&supervisorpoid=&supervisorname=&deptID=&deptIDx=&deptNamex=&building=&ski
lls=&projects=&groups=&otherinfo= HTTP/1.1\r\n

HTTP requests ends with HTTP/1.1\r\n.

So the key is, reproducing it reliably.  And then get a fix so that we know
it is fixed.
jdk7 has a newer libzip than 6.
Comments
Closing this one out. I could never reproduce this one internally nor could I get a root cause analysis from the supplied data. Should be re-opened if we can get a reproducer.
02-04-2015

@ I see that the org.apache.coyote.http11.filters.FlushableGZIPOutputStream @ stream in use here is a subclass of java.util.zip.GZIPOutputStream and as a @ result gets access to the protected Deflater Object. If Apache code is @ modifying this Object while we're performing some native zip operations @ (hopefully the native operations are delegated to our classes), then there's @ a possibility that the buffers on the native zlib level are accessed/modified @ when the zip structure is undergoing change. The SEGV seen by Cu (in @ deflate_slow) would certainly suggest that a bad address has been written to @ a zip structure. The Apache class has a custom flush() method which changes @ the compression level twice on a Deflater object in a bid for it to flush its @ data. We'd have to investigate further with Apache if that's a good design. @ . @ Their javadoc for that class is interesting also. They wrote it to overcome @ two java bugs : @ http://tomcat.apache.org/tomcat-6.0-doc/api/org/apache/coyote/http11/filters/FlushableGZIPOutputStream.html @ > Extension of GZIPOutputStream to workaround for a couple of long standing @ JDK bugs (Bug 4255743 and Bug 4813885) so the GZIP'd output can be flushed. @ https://bugs.openjdk.java.net/browse/JDK-4255743 -- Fixed in JDK 7 b77 @ https://bugs.openjdk.java.net/browse/JDK-4813885 -- Fixed in JDK 7 b97 @ . @ we might have to look into backporting these via a system property but I'm @ not sure if we want to expose GZIPOutputStream to a Z_SYNC_FLUSH type flush @ on a global (VM) level..
18-12-2013

Also reported on 7u45
09-12-2013

This issue also reported on JDK 7 : JDK-8029807
09-12-2013