JDK-8039262 : Java compiler performance degradation jdk1.7 vs. jdk1.6 should be amended
  • Type: Enhancement
  • Component: tools
  • Sub-Component: javac
  • Affected Version: 7u45
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: linux
  • CPU: x86_64
  • Submitted: 2014-04-01
  • Updated: 2015-09-29
  • Resolved: 2015-06-04
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 8 JDK 9
8u60Fixed 9 b68Fixed
Description
A DESCRIPTION OF THE REQUEST :
When compiling a large project of generated java code (translated from ASN.1 and TTCN-3 input), the performance of javax.tools.JavaCompiler$CompilationTask.call is much worse in jdk1.7 as opposed to jdk1.6 on 64 bit architecture (did not check other architectures as there the input is too large). 

Where the 1.6 version needs 50 seconds, the 1.7 version needs over 130 seconds.

The 1.7 version also needs much more memory and uses more CPU resources while doing so.

The code to be compiled contains java 1.4  features only (no generics, no enums) and is compiled with target 1.5. 

It is distributed over 187 java source files which comprise 164 MB. The classes all reside in the same package and mostly use inner non-static classes and anonymous classes (sometimes several levels deep). The result is 48000 class files. 

The code is compiled against a runtime library of maybe 200 smaller classes and rt.jar.

The classes are generated in memory in a special JavaFileManager (and kept via soft reference) and cached on file system in case of garbage collection. Profiling does not suggest any additional overhead there (observed behavior is similar in 1.6 and 1.7)

Also, after compilation, there seems to be a memory-leak of com.sun.tools.javac.util.SharedNameTable$NameImpl (300,000 objects totalling 12 MB) and of com.sun.tools.javac.file.ZipFileIndex$Entry (35,000) even though JavaFileManager.close() has been called explicitly (and full gc has run several times). I don't know if that is related to the other issue.


JUSTIFICATION :
Since the performance of 1.7 in this respect is so much worse, we have to stick with using 1.6 and thus maybe cannot profit from advantages that 1.7 might have to offer (which I'm pretty sure it does).

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The performance should be at least as good as with 1.6. Actually, I suppose it should be better as I'm sure that java 7 jvms and jit-compilers have also improved. 

I assume that the issue is caused by either additional checks that need to be performed or by other optimizations which in our case are pessimizations. However, if that is the case, at least some sort of backward compatibility options should be provided.

CUSTOMER SUBMITTED WORKAROUND :
Using java 1.6.
Comments
The underlying cause of the slowdown is a memory leak caused by Scope listeners. Every time we need to call Types.membersClosure/Types.implementation, a type hierarchy is scanned (from subtypes to supertypes) and all the members are stored in a so called CompoundScope. A CompoundScope, as the name indicates, it's just an aggregate of several 'leaf' scopes (corresponding to real class symbol scopes). So, given a hierarchy like: class A { } class B extends A { } class C extends B { } the CompoundScope for C would be something like: CompoundScope(C) = [Scope(C), CompoundScope(B)] Now, as those CompoundScopes are cached (to speed up performances) it is crucial for javac to be able to detect when any of the underlying scopes a CS depend on might change. If that's the case, the cached CS line must be thrown away - a failure to do so would result in erroneous membership computations. CompounsScope changes are detected through a listener mechanism - when the CS for C is created, it register itself as a listener for changes in all of its subscopes (Scope(C) and CS(B), respectively). Whenever a change is detected, a 'mark' (long field) is updated. This will tell the caching system that the compound scope is stale; so any information that is cached based on a given CS mark, will have to be thrown away (most notably, the cached info in Types.implementation). Now, a CS for a given class can be created more than one time - in fact the CS cache lines are thrown away in two cases: * the client requests a different set of members for C [i.e. it wants only class members but not interfaces members, while the cached CS has _all_ members] * the system is low on memory - as cache is a WeakHashMap, so entries could be thrown away from time to time When the above happens, the CS cache line is thrown - but the CS could still be referenced in the listener list of some other scope. This will (1) keep the CS alive and (2) generate increasingly big listener list, as the old CS will be replaced by a new CS which will, again, register itself as listener onto its subscopes (which will be the same as before). So, in the above example, any time the CS for C is re-created, Scope(C) gets a new listener (namely, CompoundScope(C)). If there are plently of cache misses and if the sources to be compiler refer either directly or indirectly to big set of classes, the memory footprint will start to increase. In the submitted example I've counted at least (*) 131 scopes having more than 100 listeners - as javac uses linked lists for listeners - this equates to 131 * 100 ~= 13K list instances only to keep track of all the listeners - all of which (but one) are stale! (*) - I only counted CS coming from symbols completed from source files - but in this example there are many more coming from classfiles (as the submitted example uses few jar files which contain toplevel declaration used from the classes being compiled).
30-04-2015
Attached are few patches which attempt to address the memory leak; one gets there by unregistering scope listeners when they are not used. The remaining try to avoid cache misses.
30-04-2015
You could avoid the leak of ZipFileIndexEntry by avoiding javac's internal zip file library by using the platforma standard zip library. For that, you'd need to set the system property useJavaUtilZip=true to the runtime running javac. If you were using the standard javac launcher, that would be javac -J-DuseJavaUtilZip=true javac-args but you're going through the CompilerAPI with a custom file manager, so you'll have to set the system property yourself in the runtime before you invoke javac through the compiler API.
28-04-2015
Raising a priority, as it is a regression and it has become a showstopper for upgrading to jdk7.
06-05-2014