JDK-8044794 : ECC ciphers still fail intermittently with "bad_record_mac" on Solaris 11.1
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.net.ssl
  • Affected Version: 7,8,9
  • Priority: P4
  • Status: Resolved
  • Resolution: Won't Fix
  • OS: solaris_11
  • Submitted: 2014-06-04
  • Updated: 2021-10-12
  • Resolved: 2021-10-12
Related Reports
Relates :  
Description
HTTPS connection fails intermittently on Solaris with the following output:

*** CertificateVerify
[write] MD5 and SHA1 hashes:  len = 71
0000: 0F 00 00 43 00 41 30 3F   02 1D 24 D4 80 FE 9D B9  ...C.A0?..$.....
0010: 5C E3 4A 3A D6 3E 53 CE   C2 74 F6 CB 16 3E F4 78  \.J:.>S..t...>.x
0020: 7E A5 CF 21 A5 0C 65 02   1E 00 9D 1A 79 B1 DF 99  ...!..e.....y...
0030: CE 6D 2A CE 8A 3A FD F0   1A A6 68 5C AB 15 21 72  .m*..:....h\..!r
0040: BB D1 22 59 8A 40 82                               .."Y.@.
main, WRITE: TLSv1 Handshake, length = 71
[Raw write]: length = 76
0000: 16 03 01 00 47 0F 00 00   43 00 41 30 3F 02 1D 24  ....G...C.A0?..$
0010: D4 80 FE 9D B9 5C E3 4A   3A D6 3E 53 CE C2 74 F6  .....\.J:.>S..t.
0020: CB 16 3E F4 78 7E A5 CF   21 A5 0C 65 02 1E 00 9D  ..>.x...!..e....
0030: 1A 79 B1 DF 99 CE 6D 2A   CE 8A 3A FD F0 1A A6 68  .y....m*..:....h
0040: 5C AB 15 21 72 BB D1 22   59 8A 40 82              \..!r.."Y.@.
main, WRITE: TLSv1 Change Cipher Spec, length = 1
[Raw write]: length = 6
0000: 14 03 01 00 01 01                                  ......
*** Finished
verify_data:  { 118, 226, 166, 15, 9, 187, 195, 123, 27, 21, 167, 232 }
***
[write] MD5 and SHA1 hashes:  len = 16
0000: 14 00 00 0C 76 E2 A6 0F   09 BB C3 7B 1B 15 A7 E8  ....v...........
Padded plaintext before ENCRYPTION:  len = 40
0000: 14 00 00 0C 76 E2 A6 0F   09 BB C3 7B 1B 15 A7 E8  ....v...........
0010: 23 71 99 9C EC 97 6D 65   BC 38 09 28 EA 96 ED F8  #q....me.8.(....
0020: 6F DE 0A 57 03 03 03 03                            o..W....
main, WRITE: TLSv1 Handshake, length = 40
[Raw write]: length = 45
0000: 16 03 01 00 28 58 BD 46   D3 D2 45 70 1E FD 5D AC  ....(X.F..Ep..].
0010: FF 51 E8 2B 32 FB CD D7   78 1D 65 D4 63 70 20 5D  .Q.+2...x.e.cp ]
0020: 5A D9 3F 68 9D 32 7F 66   50 26 15 A7 D7           Z.?h.2.fP&...
[Raw read]: length = 5
0000: 15 03 01 00 02                                     .....
[Raw read]: length = 2
0000: 02 14                                              ..
main, READ: TLSv1 Alert, length = 2
main, RECV TLSv1 ALERT:  fatal, bad_record_mac
%% Invalidated:  [Session-1, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA]
main, called closeSocket()
main, handling exception: javax.net.ssl.SSLException: Received fatal alert: bad_record_mac
Cleared system properties
javax.net.ssl.SSLException: Received fatal alert: bad_record_mac
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
        at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1991)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1104)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1343)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1371)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1355)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:563)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
        at TestHttpsECC.readWithParticularCipher(TestHttpsECC.java:140)
        at TestHttpsECC.testCipherWithURL(TestHttpsECC.java:197)
        at TestHttpsECC.main(TestHttpsECC.java:229)

I was not able to reproduce the failure with -Dsun.security.pkcs11.enable-solaris=false option, so it looks like only SunPKCS11-Solaris provider is affected. Maybe it is caused by an issue in Solaris crypto libs.


Comments
The last test failure was 2015, so this may have been a Solaris problem that was addressed or something else that fixed the problem. Additionally future releases no long support Solaris. At this time there is no reason to keep this bug open. If there are errors on older jdk releases that support Solaris, then it can be reopened and addressed in that release.
12-10-2021

There is another issue with similar errors. If the ECDH premaster secret has a leading zero, the PKCS11 provider on Solaris sometimes strips it off before generating the master secret. SESSION KEYGEN: PreMaster Secret: 0000: 00 CE A7 E4 7D 9A E8 08 6F 3D A4 C9 9E 58 DA 21 ........o=...X.! 0010: 48 3C 54 4F 93 FC 47 01 C4 93 27 7E 30 61 8A 2E H<TO..G...'.0a.. 0020: 2D 43 48 21 67 16 00 C4 A6 4B 2C 8D 5A CB 43 D0 -CH!g....K,.Z.C. 0030: A0 4A 0E DF C1 27 38 FA F8 DC 5C 85 E5 5F A4 DB .J...'8...\.._.. 0040: 52 A5 18 27 66 BB 17 AC R..'f... JDK-8037591: Still seeing Invalid Padding length SSL errors after the fix for JDK-8014618 18290301: Still seeing Invalid Padding length SSL errors after the fix for JDK-8014618
30-06-2014

The test fails on JDK 8 b132, so it is not a regression in JDK 8u. Also I was able to reproduce in on OpenJDK 9 build.
04-06-2014