Bug ID: JDK-8156568 Update javac to support compiling against a modular JAR that is a multi-release JAR

JDK 9
9 b138Fixed

One plausible explanation for why I saw accidental correct behavior on and off: This could be due to the walk order. If a file is seen first which has a sibling directory, then both would get visited as the code stands. If the walk order is reversed due to layout where a directory was seen first, then its sibling file won't get visited as the code stands.
18-08-2016
Of the 3 attachments, only repromr.jar is required. Jira does not allow me to delete/hide/obsolete others.
18-08-2016
This looks like a bug in src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java /** * walk the IndexNode tree processing all leaf nodes / private void walk(IndexNode inode, Consumer<IndexNode> process) { if (inode == null) return; if (inode.isDir()) { walk(inode.child, process); } else { process.accept(inode); walk(inode.sibling, process); } } Basically, when we see a directory, we descend into it, but don't pay attention to its siblings. This patch aligns the compile time module boundary enforcement with that of runtime: diff -r cc9b31691df2 src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java Wed Aug 17 16:03:52 2016 -0700 +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/JarFileSystem.java Thu Aug 18 15:40:21 2016 +0530 @@ -165,8 +165,8 @@ walk(inode.child, process); } else { process.accept(inode); - walk(inode.sibling, process); } + walk(inode.sibling, process); } /* (I can see why/how my earlier experiments appeared to work properly - basically I also had a module-info.class inside META-INF/versions/9/service apart from META-INF/versions/9 Edit: That does not seem to be all that is needed to reproduce the accidental right behavior I saw earlier) ) Looking at the layout specified here at https://bugs.openjdk.java.net/browse/JDK-8156497, I conclude the right place is for module-info is META-INF/versions/9 and not META-INF/versions/9/service.
18-08-2016
At the very least there is some inconsistent/non-deterministic behavior here. I'll continue to investigate.
18-08-2016
There is something fishy here ... In my earlier experiments I had used jar from JDK8 to create a MR jar that has patent differences between top level module-info and the one in versioned section. This simply introduces a needless extraneous variable into the picture and so I changed track and built a version of openjdk with a jar tool that would ignore export differences between module-info classes. (This was done by: $ hg diff src/jdk.jartool/share/classes/sun/tools/jar/Main.java diff -r cc9b31691df2 src/jdk.jartool/share/classes/sun/tools/jar/Main.java --- a/src/jdk.jartool/share/classes/sun/tools/jar/Main.java Wed Aug 17 16:03:52 2016 -0700 +++ b/src/jdk.jartool/share/classes/sun/tools/jar/Main.java Thu Aug 18 14:03:31 2016 +0530 @@ -2021,10 +2021,12 @@ } } } + /* if (!rd.exports().equals(vd.exports())) { fatalError(getMsg("error.versioned.info.exports.notequal")); return false; } + */ if (!rd.provides().equals(vd.provides())) { fatalError(getMsg("error.versioned.info.provides.notequal")); return false; ) Using such a modifed jar tool, I built repromr.jar (attached) such that the following is observable: $ ls -l mods total 4 -rw-rw-r-- 1 srikanth srikanth 2051 Aug 18 13:25 repromr.jar $ jar -tvf mods/repromr.jar 0 Thu Aug 18 13:25:12 IST 2016 META-INF/ 91 Thu Aug 18 13:25:12 IST 2016 META-INF/MANIFEST.MF 170 Thu Aug 18 13:25:12 IST 2016 module-info.class 156 Thu Aug 18 13:25:12 IST 2016 META-INF/versions/9/module-info.class 0 Thu Aug 18 11:26:48 IST 2016 META-INF/versions/ 0 Thu Aug 18 11:45:42 IST 2016 META-INF/versions/9/ 0 Thu Aug 18 13:24:04 IST 2016 META-INF/versions/9/service/ 252 Thu Aug 18 11:26:50 IST 2016 META-INF/versions/9/service/Service.class 0 Thu Aug 18 11:24:12 IST 2016 service/ 252 Thu Aug 18 11:24:12 IST 2016 service/Service.class $ unzip -p mods/repromr.jar META-INF/MANIFEST.MF Manifest-Version: 1.0 Multi-Release: true Created-By: 9-internal (Oracle Corporation) $ ~/openjdk/dev/build/linux-x86_64-normal-server-release/images/jdk/bin/javap -classpath mods/repromr.jar /module-info Warning: Binary file /module-info contains service.module-info Compiled from "module-info.java" module service { requires java.base; exports service; } $ ~/openjdk/dev/build/linux-x86_64-normal-server-release/images/jdk/bin/javap -classpath mods/repromr.jar META-INF/versions/9/module-info Warning: Binary file META-INF/versions/9/module-info contains service.module-info Compiled from "module-info.java" module service { requires java.base; } $ cat src/client/module-info.java module client { requires service; } $ cat src/client/client/Client.java package client; import service.Service; public class Client { public static void main(String [] args) { System.out.println("Service version = " + new Service().version()); } } $ ~/openjdk/dev/build/linux-x86_64-normal-server-release/images/jdk/bin/javac --module-path mods -d mods/client src/client/module-info.java src/client/client/Client.java $ ~/openjdk/dev/build/linux-x86_64-normal-server-release/images/jdk/bin/java --module-path mods -m client/client.Client Exception in thread "main" java.lang.NoClassDefFoundError: service/Service at client.Client.main(client/Client.java:6) Caused by: java.lang.ClassNotFoundException: service.Service at jdk.internal.loader.BuiltinClassLoader.loadClass(java.base@9-internal/BuiltinClassLoader.java:366) at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(java.base@9-internal/ClassLoaders.java:185) at java.lang.ClassLoader.loadClass(java.base@9-internal/ClassLoader.java:419) ... 1 more So it wouldl appear that javac ends up using the top level module-info at compilation time - At least that is what I consistently observe using the repromr.jar file. I must say that while the behavior is consistent(ly wrong) now, in my interim experiments with other artifacts, I did see javac correctly using the versioned files. Unfortunately, I didn't preserve them :(
18-08-2016
As Jon notes, we created this issue to look into how multi-release modular JARs are handled by javac. The initial concern was that javac appeared to examine the module-info.class in the top-level directory and also the module-info.class in the versioned section. With a well formed multi-release JAR then this shouldn't matter as the differences should be limited to non-public `requires` of java.* or jdk.* module, or different `uses` clauses. Such differences don't impact the module's "API". On other hand, there may be corner cases with malformed multi-release JARs (say where someone creates a multi-release modular JARs with a completely difference module declaration in the versioned section). Also with automatic modules in the picture then javac needs the transitive closure and this might be different set of modules. So I agree with the suggestions that this issue should focus on creating tests, painful as it might be.
18-08-2016
The comment about Locations using ZipFileSystem instead of JarFileSystem is noted, and worth following up. At a minimum it should use JarFileSystem, and ideally interacting well with the file managers ability to cache open jar file systems.
18-08-2016
The issue was filed proactively, because it was not clear that the two features would work well together. A satisfactory resolution of this issue would be a test that effectively demonstrates these two features working together as expected. For example, have a modular jar containing some baseline classes, and some additional classes, including module-info.class, for the JDK 9 variant. Run javac to compile some code, with the modular jar on the classpath, and -release 8, to verify compilation against the baseline files. Run javac to compile some different code, with the modular jar on the classpath to verify compilation against the version 9 files, excluding module-info.class Run javac to compile some different code, with the modular jar on the modulepath, to verify compilation against the version 9 files, including module-info.class
18-08-2016
Do we have any conclusion yet on whether javac needs an update?
10-06-2016
If the JAR file is opened twice, once to get the module name, and the second time with MR configured then it should be okay as it would be a malformed MR JAR if the module name were different in the versioned section.
09-05-2016
Can do, but right now, I don't see where/why the system is failing. In Locations, we only get the module name; ModuleFinder always goes through the normal file manager code paths, so should be fully aware of multi-version-ness. So right now, I'm not understanding whether there is really a bug here and whether we need to do anything. (Setting aside the perf reasons for only opening file systems once.) More to the point, is this a problem within jar-fs itself, not looking in the versioned section for module-info.class.
09-05-2016
Jon, I think this relates to opening the FileSystems only once I am looking at: we will open the jars with the proper multiversion settings, and use from there on. Feel free to reassign to me.
09-05-2016
javac will currently read the top level module-info early on, to get the module name. All significant content (i.e. class file attributes, like Module etc) should come from the versioned file, assuming that jar-fs will read the versioned file. In other words, javac has very very little knowledge of MR-jars, except to pass down the desired version value when opening the jar file system.
09-05-2016
This is not the intent, and needs investigation.
09-05-2016