JDK-8226719 : Kerberos login to Windows 2000 failed with "Inappropriate type of checksum in message"
  • Type: Bug
  • Component: security-libs
  • Sub-Component: org.ietf.jgss
  • Affected Version: 11,13
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2019-06-25
  • Updated: 2020-10-12
  • Resolved: 2019-07-03
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 13 JDK 14 JDK 8 Other
11.0.6Fixed 13 b28Fixed 14Fixed 8u270Fixed openjdk8u242Fixed
Related Reports
Relates :  
JDK-8227381 - GSS login fails with PREAUTH_FAILED
Relates :  
JDK-8215032 - Support Kerberos cross-realm referrals (RFC 6806)
Description
When authenticate into a Windows 2000 server using a DES encrytion type, the below error is observed,
KrbException: Inappropriate type of checksum in message (50)
	at java.security.jgss/sun.security.krb5.Checksum.verifyKeyedChecksum(Checksum.java:192)
	at java.security.jgss/sun.security.krb5.KrbKdcRep.check(KrbKdcRep.java:162)
	at java.security.jgss/sun.security.krb5.KrbAsRep.decrypt(KrbAsRep.java:158)
	at java.security.jgss/sun.security.krb5.KrbAsRep.decryptUsingPassword(KrbAsRep.java:139)
	at java.security.jgss/sun.security.krb5.KrbAsReqBuilder.resolve(KrbAsReqBuilder.java:308)
	at java.security.jgss/sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:443)
	at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:743)
	at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:592)
	at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:726)
	at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:665)
	at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:663)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:690)
	at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:663)
	at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:574)
        ...
Comments
Fix request (11u) 11u is affected by this bug after the backport of 8215032. I'd like to request an approval. Patch applies cleanly and this risk is minimal.
04-11-2019
[~rhalade] I'll ping again.
03-07-2019
[~weijun] what is the status of this fix? We have about ~70 test failures due to this putting ATR results in red. I see that you have a review out, let's hope that we can get this fix in for JDK 13+28.
03-07-2019
It looks like the KDC is using the RSA-MD5 checksum in its PA_REQ_ENC_PA_REP in AS-REP. RSA-MD5 is unkeyed and we treat it unsafe in Java. Maybe we should reconsider this because here the checksum is inside an encrypted blob and there is no need for it to be keyed. That said, the checksum type is only used along with the DES etype. Modern etypes use other checksum types and they are still supported.
02-07-2019