JDK-7171703 : JNI DefineClass crashes client VM when first parameter is NULL
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 7
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: windows_7
  • CPU: x86
  • Submitted: 2012-05-25
  • Updated: 2014-02-05
  • Resolved: 2012-06-16
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7 JDK 8 Other
7u40Fixed 8Fixed hs23.2Fixed
Related Reports
Relates :  
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.7.0_04"

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows [Version 6.1.7601]

A DESCRIPTION OF THE PROBLEM :
Before Java 7, you could pass NULL to the first parameter on the JNI DefineClass function, meaning the VM should figure out the name of the class by itself.

On Java 7, this causes the VM to crash, taking down the entire process and creates a minidump

REGRESSION.  Last worked in version 6u31

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
- Load the VM into a process using LoadLibrary
- Lookup the JNI_CreateJavaVM entry point
- Create the VM
- Call env->DefineClass with the first parameter being NULL
- BOOM!

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
- Same as Java 6: the class loads and the VM figures out the name by looking at the bytecode
ACTUAL -
BOOM! minidump

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
#define CLASS_FILE_NAME _T("blob.bytes")
#define JAVA_BINARY_CLASS_NAME "com/nanshusoft/defineclass/SimpleClass"


HANDLE class_file = CreateFile ( CLASS_FILE_NAME, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL );
	if ( class_file == INVALID_HANDLE_VALUE )
	{
		exit ( GetLastError() );
	}
	
	LARGE_INTEGER lpFileSize;
	if ( !GetFileSizeEx ( class_file, &lpFileSize ) )
	{
		exit ( GetLastError() );
	}

	LPBYTE buffer = (LPBYTE)malloc( lpFileSize.LowPart );
	ZeroMemory ( buffer, lpFileSize.LowPart );
	DWORD bytesRead = 0;
	if ( !ReadFile ( class_file, buffer, lpFileSize.LowPart, &bytesRead, NULL ) )
	{
		exit ( GetLastError() );
	}
	CloseHandle ( class_file );
	jclass classObj = NULL;

	if ( safe_method )
	{
		/* This works in both Java 6 and Java 7 */
		classObj = env->DefineClass ( JAVA_BINARY_CLASS_NAME, NULL , (const jbyte*)buffer, bytesRead );
	}
	else
	{
		/* This works in Java 6 and crashes the VM in Java 7 */
		classObj = env->DefineClass ( NULL, NULL , (const jbyte*)buffer, bytesRead );
	}

	free ( buffer );
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Pass the name of the class in the first parameter.

The big problem is that it breaks existing applications by crashing them instead of returning a Java exception.  Any function callable by the user should at least check the parameters before acting on them.

If Java 7 is no longer supporting a NULL first parameter (from Java code, it still works) in DefineClass, please update the documentation AND do a NULL check in the native code and throw a Java exception

Comments
During the investigation of the following bug: JDK-7034137 HS Crash; Unable to launch Java to play a Java game (Axis and Allies) on Win7 SP1 a test was developed to reproduce this failure mode. Attaching that test as DefineClassTest.tgz.
05-02-2014

EVALUATION http://hg.openjdk.java.net/lambda/lambda/hotspot/rev/dcfcdd01af4b
29-06-2012

EVALUATION http://hg.openjdk.java.net/hsx/hotspot-main/hotspot/rev/dcfcdd01af4b
14-06-2012

EVALUATION <moved to hs23.2>
07-06-2012

EVALUATION http://hg.openjdk.java.net/hsx/hotspot-rt/hotspot/rev/dcfcdd01af4b
05-06-2012