JDK-6988220 : java.lang.ObjectName use of String.intern() causes major performance issues at scale
  • Type: Bug
  • Component: core-svc
  • Sub-Component: javax.management
  • Priority: P2
  • Status: Closed
  • Resolution: Won't Fix
  • OS: generic
  • CPU: generic
  • Submitted: 2010-09-29
  • Updated: 2019-08-16
  • Resolved: 2012-04-03
Related Reports
Relates :  
JDK-6962930 - make the string table size configurable
Description
BUI interactive performance analysis of the ops-center product at scale has shown that
there are significant slow-downs in the BUI interactions due to the time spent in ObjectName creation.

This in turn turns out to be due to the call to String.intern() in the ObjectName for the canonical string.

String.intern() is designed for constant strings, it's implemented in C/C++ in the hotspot core code, and it has a fixed-size hashmap of around 20K entries and if more than this number of strings are interned, the performance degrades linearly since there are hash collisions and the code has to search down linked lists.

Ops Center, when managing ~500 systems, had approx 70000 ObjectNames and hence the String.intern() calls from ObjectNames are completely overloading the hashmap, making
each call to String.intern() extremely expensive.

By dropping this intern, we were able to speed up certain complex calls which weren't just manipulating ObjectNames by a factor of 2.
Comments
EVALUATION As explained in Nick's comment, we have found a VM option to solve the scalability issue. The option didn't exist yet when the CR has been created, but it is available for all JDK7 builds. Using this VM option, the OpsCenter team has been able to resolve their scalability issue without having to wait for a new release. BTW, different solutions have been discussed to change the ObjectName implementation to solve the issue in a different way, but they could have cause performance regressions for other workload.
03-04-2012
WORK AROUND The scalability issue was due to the default size (1009 buckets) of the hash table used to store intern Strings compared to the number of intern Strings generated by the ObjectName class (70K+). However, with JDK7 and higher, it's possible to configure the size of this hash table using the -XX:StrintTableSize option. Configuring the hash table appropriately can prevent the scalability issue to happen.
08-03-2012
EVALUATION Frederic has identified that the string.intern() hashmap size is now configurable in JDK 7 By configuring this hashtable size from its default of 1009 buckets to 27001 buckets we are able to maintain near O(1) performance for up to 200K ObjectNames over and above the initially interned 1000K strings, with the performance much better than before, starting to degrade measurably but much more slowly as would be expected. cacao-35 88:$ java -XX:StringTableSize=27001 -XX:PermSize=128m -Xms1000M -Xmx1000M ON 10000 Interning 100000 strings to reach initial configuration Creation time: 197 Recreate time=89 Count=10000 Creation time: 102 Recreate time=81 Count=20000 Creation time: 98 Recreate time=82 Count=30000 Creation time: 98 Recreate time=188 Count=40000 Creation time: 34 Recreate time=35 Count=50000 Creation time: 34 Recreate time=35 Count=60000 Creation time: 35 Recreate time=36 Count=70000 Creation time: 35 Recreate time=35 Count=80000 Creation time: 270 Recreate time=45 Count=90000 Creation time: 37 Recreate time=36 Count=100000 Creation time: 37 Recreate time=36 Count=110000 Creation time: 38 Recreate time=36 Count=120000 Creation time: 41 Recreate time=37 Count=130000 Creation time: 313 Recreate time=36 Count=140000 Creation time: 38 Recreate time=40 Count=150000 Creation time: 39 Recreate time=38 Count=160000 Creation time: 39 Recreate time=37 Count=170000 Creation time: 39 Recreate time=249 Count=180000 Creation time: 39 Recreate time=38 Count=190000 Creation time: 39 Recreate time=38 Count=200000 The new ability to make this configuration change means that we do not need a change to the ObjectName implementation. Thanks
08-03-2012
EVALUATION I've added a README with a unit-test program which reproduces this problem.
07-03-2012
EVALUATION Drop the call to .intern() in the ObjectName class. Request a fix in a Java 6 update release so that it can be automatically picked up by the ops-center product when available.
29-09-2010