JDK-6548428 : JavaCompiler API and Webstart bug
  • Type: Bug
  • Component: tools
  • Sub-Component: javac
  • Affected Version: 6,6u18
  • Priority: P4
  • Status: Resolved
  • Resolution: Won't Fix
  • OS: linux,windows_xp
  • CPU: x86
  • Submitted: 2007-04-20
  • Updated: 2022-05-17
  • Resolved: 2022-05-17
Related Reports
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.6.0_01"
Java(TM) SE Runtime Environment (build 1.6.0_01-b06)
Java HotSpot(TM) Server VM (build 1.6.0_01-b06, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
Linux mogul 2.6.17-10-generic #2 SMP Tue Dec 5 22:28:26 UTC 2006 i686 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
(after posting here http://forum.java.sun.com/thread.jspa?messageID=9622115 I was suggested to file a bug report)

I am using the brand new javax.tools.JavaCompiler feature to compile on the fly some source code. However using the library under web start becomes very tricky.
1) The compiler tries to read the java.endorsed.dirs system property and it fails. The only solution I have found so far is to grant all permission on the client javaws.policy file (which is the more than bad solution. I have posted a thread here too: http://forum.java.sun.com/thread.jspa?messageID=9620391 )

2) If I solve problem 1, then the compilation fails again because the new class I am creating implements an Interface which is normally found in the class path. However that is no more true on web start since application jars are loaded by a custom class loader, and the JavaCompiler (even though called from within the classloader who knows the required interface) is not able to find the interface. This can be solved however by providing the -cp option to the compiler, but again in case of webstart is not easy to find out where the downloaded jars are.

Now the question is: is there a reason why JavaCompiler uses a class loader whose parent is not taken from the instantiating method but rather the system class loader is used directly? Or is there something I am missing?


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
You can find the source code I use for compiling here http://www.iocreo.com/OnTheFlyCompiler.java
When running this I get
error: Could not create class loader for annotation processors: access denied (java.util.PropertyPermission java.endorsed.dirs read)

Problem ii) is solved by creating the String ops[]. However this solution only works out with NetX jnlp implementation, and not with sun's one which returns original urls still in the form http://something

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
It should work as out of web start
ACTUAL -
error: Could not create class loader for annotation processors: access denied (java.util.PropertyPermission java.endorsed.dirs read)

ERROR MESSAGES/STACK TRACES THAT OCCUR :
error: Could not create class loader for annotation processors: access denied (java.util.PropertyPermission java.endorsed.dirs read)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
package ch.olsen.products.util.compile;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.charset.Charset;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;

import javax.tools.Diagnostic;
import javax.tools.DiagnosticCollector;
import javax.tools.FileObject;
import javax.tools.ForwardingJavaFileManager;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileManager;
import javax.tools.JavaFileObject;
import javax.tools.SimpleJavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;
import javax.tools.JavaCompiler.CompilationTask;
import javax.tools.JavaFileObject.Kind;

public class OnTheFlyCompiler {

	private static JavaFileObject generateJavaSource(final String fileName, final String source) {

		return new SimpleJavaFileObject(toURI(fileName), Kind.SOURCE) {

			@Override
			public CharSequence getCharContent(boolean
					ignoreEncodingErrors)
			throws IOException, IllegalStateException,
			UnsupportedOperationException {
				return source;
			}

		};
	}


	static class RAMJavaFileObject extends SimpleJavaFileObject {

		RAMJavaFileObject(String name, Kind kind) {
			super(toURI(name), kind);
		}

		ByteArrayOutputStream baos;

		@Override
		public CharSequence getCharContent(boolean ignoreEncodingErrors)
		throws IOException, IllegalStateException,
		UnsupportedOperationException {
			throw new UnsupportedOperationException();
		}

		@Override
		public InputStream openInputStream() throws IOException,
		IllegalStateException, UnsupportedOperationException {
			return new ByteArrayInputStream(baos.toByteArray());
		}

		@Override
		public OutputStream openOutputStream() throws IOException,
		IllegalStateException, UnsupportedOperationException {
			return baos = new ByteArrayOutputStream();
		}

	}

	public static interface FileSpec {
		String getFileName();
		String getSource();
	}

	public static class FileSpecImpl implements FileSpec {
		String fileName;
		String source;
		public FileSpecImpl(String fileName, String source) {
			this.fileName = fileName;
			this.source = source;
		}
		public final String getFileName() {
			return fileName;
		}
		public final String getSource() {
			return source;
		}
	}

	public static ClassLoader onTheFlyCompile(final FileSpec fileSpec) throws Exception {
		return onTheFlyCompile(Arrays.asList(fileSpec));
	}
	@SuppressWarnings("unchecked")
	public static ClassLoader onTheFlyCompile(final List<FileSpec> fileSpecs) throws Exception {

		if ( System.getSecurityManager() != null ) {
			java.util.PropertyPermission perm = new java.util.PropertyPermission("java.endorsed.dirs", "read");
			System.getSecurityManager().checkPermission(perm);
		}

		final Map<String, JavaFileObject> output =
			new HashMap<String, JavaFileObject>();

		final JavaCompiler compiler =
			ToolProvider.getSystemJavaCompiler();

		final DiagnosticCollector<JavaFileObject> diagnostics =
			new DiagnosticCollector<JavaFileObject>();

		final JavaFileManager jfm = new
		ForwardingJavaFileManager<StandardJavaFileManager> (
				compiler.getStandardFileManager(diagnostics, Locale.getDefault(), Charset.defaultCharset())) {

			@Override
			public JavaFileObject getJavaFileForOutput(Location location,
					String name,
					Kind kind,
					FileObject sibling) throws IOException {
				JavaFileObject jfo = new RAMJavaFileObject(name, kind);
				output.put(name, jfo);
				return jfo;
			}

		};

		final List<JavaFileObject> files = new ArrayList<JavaFileObject>();
		for ( FileSpec fs : fileSpecs )
			files.add(generateJavaSource(fs.getFileName(), fs.getSource()));

		String ops[] = buildClassPath(OnTheFlyCompiler.class.getClassLoader());
		System.out.println("On the fly compiler: calling with opstions: "+ ops[0]+ " " + ops[1]);

		//DO NOT PASS PARAMETER ops TO VIEW PROBLEM ii)

		CompilationTask task = compiler.getTask(
				null, jfm, diagnostics, Arrays.asList(ops), null,
				files);
		if (! task.call()) {
			for(Diagnostic dm : diagnostics.getDiagnostics())
				System.err.println(dm);
			throw new RuntimeException("Could not compile");
		}

		System.out.println("On the fly compiler: generated classes: "+ output.keySet());

		ClassLoader cl = new OnTheFlyClassLoader(output);
		
		String cp[] = buildClassPath(cl);
		System.out.println("On the fly compiler: generated class loader with classpath: "+cp[1]);

		return cl;
		/*Class<?> c = Class.forName("just.generated.Hello", false, cl);
		c.getMethod("main", String[].class)
		.invoke(null, new Object[] {args});*/

	}
	
	static class OnTheFlyClassLoader extends ClassLoader {
		final Map<String, JavaFileObject> output;
		public OnTheFlyClassLoader(final Map<String, JavaFileObject> output) {
			super(OnTheFlyCompiler.class.getClassLoader());
			this.output = output;
		}
		@Override
		protected Class<?> findClass(String name) throws
		ClassNotFoundException {
			JavaFileObject jfo = output.get(name);
			if (jfo != null) {
				byte[] bytes = ((RAMJavaFileObject)
						jfo).baos.toByteArray();
				return defineClass(name, bytes, 0, bytes.length);
			}
			//return super.findClass(name);
            throw new ClassNotFoundException();
		}
	}

	private static String[] buildClassPath(ClassLoader cl) {
		List<URL> classpath = new ArrayList<URL>();
		if ( cl instanceof URLClassLoader ) {
			URLClassLoader cl2 = (URLClassLoader)cl;
			for (URL jar : cl2.getURLs()) {
				classpath.add(jar);
			}
		}
		String classPath = System.getProperty("java.class.path");
		for (String path : classPath.split(File.pathSeparator)) {
			try {
				classpath.add(new URL("file:"+path));
			} catch (MalformedURLException e) {
				System.err.println("Wrong url: "+e.getMessage());
				e.printStackTrace();
			}
		}

		StringBuffer sb = new StringBuffer();
		for (URL jar : classpath) {
			sb.append(jar.getPath());
			sb.append(File.pathSeparatorChar);
		}
		String ops[] = new String[] {"-cp" , sb.toString() };
		return ops;
	}
	private static URI toURI(String name) {
		try {
			return new URI(name);
		} catch (URISyntaxException e) {
			throw new RuntimeException(e);
		}
	}

	@Deprecated
	/**
	 * just as an example. do not use it
	 */
	static JavaFileObject generateJavaSourceCode() {

		final String source =
			"package just.generated;\n" +
			"public class Hello {\n" +
			"public static void main(String... args) {\n" +
			"System.out.println(new Object() {\n" +
			"public String toString() {\n" +
			"return \"just hello!\";\n" +
			"}\n" +
			"});\n" +
			"}\n" +
			"}";
		return generateJavaSource("Hello.java", source);
	}
	

}



---------------
package ch.olsen.products.util.compile.test;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

import junit.framework.TestCase;
import ch.olsen.products.util.compile.OnTheFlyCompiler;
import ch.olsen.products.util.compile.OnTheFlyCompiler.FileSpec;

public class TestCompiler extends TestCase {

	public static void test1() throws Exception {
		ClassLoader cl = OnTheFlyCompiler.onTheFlyCompile(
				new OnTheFlyCompiler.FileSpecImpl("Test1.java",
						"package ch.olsen.test; " +
						"public class Test1 implements ch.olsen.test.TestCompiler.Interface1 { " +
						"  public String say() {" +
						"    return \"hello compiled on the fly\"; " +
						"  }" +
						"}"
				));
		Class clazz = cl.loadClass("ch.olsen.test.Test1");
		Interface1 i = (Interface1)clazz.newInstance();
		System.out.println(i.say());
	}
	
	public interface Interface1 {
		String say();
	}
	
	public static void test2() throws Exception {
		List<FileSpec> files = new ArrayList<FileSpec>();
		files.add(
				new OnTheFlyCompiler.FileSpecImpl("Test1.java",
						"package ch.olsen.test; " +
						"public class Test1 implements ch.olsen.test.TestCompiler.Interface1 { " +
						"  public String say() {" +
						"    return \"hello compiled on the fly\"; " +
						"  }" +
						"}"
				));
		files.add(
				new OnTheFlyCompiler.FileSpecImpl("Test2.java",
						"package ch.olsen.test; " +
						"public class Test2 implements ch.olsen.test.TestCompiler.Interface1 { " +
						"  public String say() {" +
						"    return \"hello2 compiled on the fly\"; " +
						"  }" +
						"}"
				));
		ClassLoader cl = OnTheFlyCompiler.onTheFlyCompile(files);
		Class clazz = cl.loadClass("ch.olsen.test.Test1");
		Interface1 i = (Interface1)clazz.newInstance();
		System.out.println(i.say());
		clazz = cl.loadClass("ch.olsen.test.Test2");
		i = (Interface1)clazz.newInstance();
		System.out.println(i.say());
		
	}


	public static void test3() throws Exception {
		ClassLoader cl = OnTheFlyCompiler.onTheFlyCompile(Arrays.asList(new FileSpec[] {
				new OnTheFlyCompiler.FileSpecImpl("Test1.java",
						"package ch.olsen.test; " +
						"public class Test1 implements ch.olsen.test.TestCompiler.Interface1 { " +
						"  public String say() {" +
						"    return \"hello compiled on the fly\"; " +
						"  }" +
						"}"
				),
				new OnTheFlyCompiler.FileSpecImpl("Test2.java",
						"package ch.olsen.test; " +
						"public class Test2 implements ch.olsen.test.TestCompiler.Interface1 { " +
						"  public String say() {" +
						"    return \"hello2 compiled on the fly\"; " +
						"  }" +
						"}"
				) }));
		Class clazz = cl.loadClass("ch.olsen.test.Test1");
		Interface1 i = (Interface1)clazz.newInstance();
		System.out.println(i.say());
		clazz = cl.loadClass("ch.olsen.test.Test2");
		i = (Interface1)clazz.newInstance();
		System.out.println(i.say());
		
	}
}

---------- END SOURCE ----------

Comments
EVALUATION There are two issues here: permissions and classloaders Permisssions: javac obviously makes use of various system APIs in order to do its job, and one should reasonably expect to give it all necessary permissions, such as the ability to read properties and to read and write files. ClassLoaders and classpath: javac actually uses *two* classpaths, although most people do not need to be aware of one of them. The first is the classpath used by the JVM used to locate the classes that make up javac. The second is the classpath you see on the javac command line, which is how javac locates the classes to be used in the compilation. This is the classpath that most people are aware of, and is the one you set with javac's -classpath option. When you add classes into the classloader used to run javac, you're only putting them onto the first of the classpaths -- the classpath used to run javac; you need to put your classes on the other classpath: the one javac uses to access the classes for the compilation. This is why you still need to use -classpath for your classes.
23-02-2010