Bug ID: JDK-8219658 Deadlock in sun.security.ssl.SSLSocketImpl

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.

JDK 12	JDK 13
12-poolResolved	13 b11Fixed

ADDITIONAL SYSTEM INFORMATION :
OS: Linux 64-bit
Linux Kernel: 4.20.10-200.fc29.x86_64
---
openjdk version "12" 2019-03-19
OpenJDK Runtime Environment (build 12+33)
OpenJDK 64-Bit Server VM (build 12+33, mixed mode, sharing)

A DESCRIPTION OF THE PROBLEM :
I was analyzing last remaining WildFly failures against JDK 12 RC and I identified one SSL deadlock we are observing in our test suite.
For some reason thread dump on JDK12 didn't work for me because of some weird maven surefire bug.
In order to identify the root cause of JVM freeze quickly I decided to debug it manually and use debug logs from custom OpenJDK local build.
The issue I identified was introduced in sun.security.ssl.SSLSocketImpl via this commit:

https://github.com/openjdk/jdk/commit/db04b02ec3831f97f2b093c80cde363fd3314af9

To identify the root cause of failure we are observing I used this custom branch built on top of JDK 12 build 33:

https://github.com/ropalka/jdk/commits/ssl-deadlock-analysis

This custom branch helped me to produce debug log (I'm attaching - openjdk.debug) where I discovered this SSL issue.
In the end of that log you can see that in some usecases SSLSocketImpl$AppInputStream.read()
method may hold inner class intrinsic lock when SSLSocketImpl$AppInputStream.deplete() method is being called
and thread invoking that deplete() method will never succeed with inner class intrinsic lock acquisition.

When I remove the deplete() method synchronization everything works for me.
Here's my proposal fix of this issue addressing JDK13 upstream:

https://github.com/ropalka/jdk/commit/d1bbfd126e9bde9d3db0b8b73ee1c60ea93b1dc9

It would be great if that issue could be backported into JDK12 RC too.

REGRESSION : Last worked in version 12

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
git clone https://github.com/wildfly/wildfly-core
cd wildfly-core
mvn clean install -DallTests -DskipTests
export JAVA_HOME=$ORACLE_JDK12
export PATH=$JAVA_HOME/bin:$PATH
cd testsuite/manualmode
mvn clean test -Dtest=**/AuditLogBootingSyslogTest

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Test passes
ACTUAL -
Test always deadlocks

FREQUENCY : always

This push was backouted by JDK-8219990, so it is unnecessary to verify it.
29-06-2019
The AuditLogBootingSyslogTest.java test was patched to ignore JDK 12+. Please make sure the following patch rollbacked before running the test. https://github.com/wildfly/wildfly-core/commit/0a16a1519b2c13c1379801322123712caa79f0d7
02-05-2019
The update was backout within JDK-8219990.
01-03-2019
Fix request: A regression from the JDK-8209333 fix. Best to eliminate this deadlock and port to 12 Updates.
01-03-2019
Fix Request: There could be a deadlock if different threads are used for socket closing and reading. The fix is simple and the risk is minimal. The bug reported had verified the update.
26-02-2019
No new regression test. The fix is similar to the contributed patch and the patch had been verified by the bug reporter.
25-02-2019
Issue is reproducible in linux platform. To reproduce, follow the steps mentioned in the bug description. Reproduciblity with different builds: JDK 12 build 33 - Failed JDK 12 build 25 - Failed JDK 12 build 24 - Passed
25-02-2019

Relates :	JDK-8219990 - Backout JDK-8219658
Relates :	JDK-8219991 - New fix of the deadlock in sun.security.ssl.SSLSocketImpl
Relates :	JDK-8209333 - Socket reset issue for TLS 1.3 socket close
Relates :	JDK-8219955 - Intermittent failure of some HTTP Client tests due to 8219658