JDK-8049737 : Contended Locking reorder and cache line bucket
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 9
  • Priority: P4
  • Status: Closed
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2014-07-09
  • Updated: 2022-03-28
  • Resolved: 2014-10-14
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 9
9 b37Fixed
Related Reports
Relates :  
Relates :  
Relates :  
Relates :  
Relates :  
Relates :  
Relates :  
Relates :  
Description
This bug will be used to track work on the Contended Locking reorder and cache line bucket.

Lesson of the day - don't press return when entering the synopsis. You'll get a pretty much empty bug.
Comments
URL: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/5fc21235d0fd User: lana Date: 2014-10-29 19:22:26 +0000
29-10-2014

Here's the micro benchmark for stressing ObjectMonitor cach-lines: $ unzip -l MonitorCacheLineStresser.zip Archive: MonitorCacheLineStresser.zip Length Date Time Name --------- ---------- ----- ---- 0 10-22-2014 10:40 MonitorCacheLineStresser/ 17916 09-22-2014 11:37 MonitorCacheLineStresser/MonitorCacheLineStresser.java 952 09-22-2014 11:37 MonitorCacheLineStresser/run_MonitorCacheLineStresser.ksh 2055 10-15-2014 18:48 MonitorCacheLineStresser/do_MonitorCacheLineStresser_runs.ksh --------- ------- 20923 4 files 1) Compile the test source: $JAVA_HOME/bin/javac MonitorCacheLineStresser.java MonitorCacheLineStresser.java:14: warning: Contended is internal proprietary API and may be removed in a future release import sun.misc.Contended; ^ MonitorCacheLineStresser.java:133: warning: Contended is internal proprietary API and may be removed in a future release @Contended ^ 2 warnings 2) You can run the test standalone, but that doesn't give you stable results: $ $JAVA_HOME/bin/java MonitorCacheLineStresser Hello from MonitorCacheLineStresser! Usage: java MonitorCacheLineStresser n_secs n_threads where: n_secs ::= > 0 n_threads ::= > 1 3) Use the wrapper to run the test: The reason for using the wrapper to run the test is because there are several options that should be used when running this test: -XX:-UseBiasedLocking - turns off biased locking -XX:+UseHeavyMonitors - turns off stack locks The Contended Locking project is not changing biased locking code so we should test with biased locking disabled to avoid jitter. This particular contended locking bucket only changes the ObjectMonitor field layout and padding so stack locking is also disabled to avoid jitter. $ ksh run_MonitorCacheLineStresser.ksh Usage: run_MonitorCacheLineStresser.ksh <java_options> n_secs n_threads where: n_secs ::= > 0 n_threads ::= > 1 Here's an example run for 30 seconds with 4 cache-line stresser threads: $ ksh run_MonitorCacheLineStresser.ksh 30 4 + /java/re/jdk/1.9.0/latest/binaries/solaris-x64/bin/java -showversion -ms256m -mx256m -XX:+PrintCommandLineFlags -XX:-RestrictContended -XX:-UseBiasedLocking -XX:+UseHeavyMonitors -cp . MonitorCacheLineStresser 30 4 -XX:InitialHeapSize=268435456 -XX:MaxHeapSize=268435456 -XX:+PrintCommandLineFlags -XX:ReservedCodeCacheSize=251658240 -XX:-RestrictContended -XX:+SegmentedCodeCache -XX:-UseBiasedLocking -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseHeavyMonitors -XX:+UseParallelGC java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b35) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-ea-b35, mixed mode) Hello from MonitorCacheLineStresser! INFO: n_secs=30 INFO: n_threads=4 MCLS_WorkerThread: 'MCLS_WorkerThread-1' is running. MCLS_WorkerThread: 'MCLS_WorkerThread-0' is running. MCLS_WorkerThread: 'MCLS_WorkerThread-2' is running. MCLS_WorkerThread: 'MCLS_WorkerThread-4' is running. MCLS_WorkerThread: 'MCLS_WorkerThread-3' is running. MCLS_WorkerThread: 'MCLS_WorkerThread-1' is done. MCLS_WorkerThread: 'MCLS_WorkerThread-0' loop_cnt=50053104, last all_hash=1271226699 MCLS_WorkerThread: 'MCLS_WorkerThread-0' is done. MCLS_WorkerThread: 'MCLS_WorkerThread-2' is done. MCLS_WorkerThread: 'MCLS_WorkerThread-4' is done. MCLS_WorkerThread: 'MCLS_WorkerThread-3' is done. MCLS_WorkerThread-1: counter=107390491 MCLS_WorkerThread-1: nanos_per_lock=279 MCLS_WorkerThread-2: counter=61904606 MCLS_WorkerThread-2: nanos_per_lock=484 MCLS_WorkerThread-3: counter=63984538 MCLS_WorkerThread-3: nanos_per_lock=468 MCLS_WorkerThread-4: counter=61734903 MCLS_WorkerThread-4: nanos_per_lock=485 MonitorCacheLineStresser: avg_nanos_per_lock=429 Bye from MonitorCacheLineStresser! 5) Use do_MonitorCacheLineStresser_runs.ksh to make multiple runs and have the results summarized: $ ksh do_MonitorCacheLineStresser_runs.ksh Usage: do_MonitorCacheLineStresser_runs.ksh <java_options> n_runs n_secs n_threads where: n_runs ::= > 0 n_secs ::= > 0 n_threads ::= > 1 Here's an example batch with 10 (+2 extra) runs for 30 seconds with 4 cache-line stresser threads: $ ksh do_MonitorCacheLineStresser_runs.ksh 10 30 4 2>&1 | sed 's/^/ /' INFO: n_runs=10 (+2 extra) INFO: n_secs=30 (per-run) INFO: n_threads=4 (per-run) INFO: skipping low value of 344 INFO: skipping high value of 418 INFO: # samples used = 10 INFO: average = 372 The average is similar to the single run using run_MonitorCacheLineStresser.ksh. The raw output from all the runs is in do_MonitorCacheLineStresser_runs.raw_data
22-10-2014

URL: http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/5fc21235d0fd User: dcubed Date: 2014-10-14 19:38:03 +0000
14-10-2014

Comparison of ObjectMonitor::sanity_checks() output on Windows X86: $ uname -a CYGWIN_NT-6.1-WOW64 DDAUGHER-PC 1.7.16(0.262/5/3) 2012-07-20 22:55 i686 Cygwin $ bin/java -showversion -client -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp `cygpath -m $HOME` Hello INFO: sizeof(ObjectMonitor)=112 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=16 WARNING: the _header and _owner fields are closer than a cache line which permits false sharing. WARNING: ObjectMonitor size is not a multiple of a cache line which permits false sharing. INFO: sizeof(SharedGlobals)=272 INFO: offset(stwRandom)=64 INFO: offset(_hcSequence)=200 INFO: error_cnt=0 INFO: warning_cnt=2 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) Client VM (build 1.9.0-internal-201407221344.pobajaj.hotspot, mixed mode) Hello world! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=65 $ bin/java -showversion -server -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp `cygpath -m $HOME` Hello INFO: sizeof(ObjectMonitor)=112 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=16 WARNING: the _header and _owner fields are closer than a cache line which permits false sharing. WARNING: ObjectMonitor size is not a multiple of a cache line which permits false sharing. INFO: sizeof(SharedGlobals)=272 INFO: offset(stwRandom)=64 INFO: offset(_hcSequence)=200 INFO: error_cnt=0 INFO: warning_cnt=2 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) Server VM (build 1.9.0-internal-201407221344.pobajaj.hotspot, mixed mode) Hello world! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=65 $ bin/java -showversion -client -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp `cygpath -m $HOME` Hello INFO: sizeof(ObjectMonitor)=104 INFO: sizeof(PaddedEnd<ObjectMonitor>)=128 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=32 WARNING: the _header and _owner fields are closer than a cache line which permits false sharing. INFO: sizeof(SharedGlobals)=96 INFO: offset(stwRandom)=32 INFO: offset(_hcSequence)=64 WARNING: the SharedGlobals.stwRandom field is closer to the struct beginning than a cache line which permits false sharing. WARNING: the SharedGlobals.stwRandom and SharedGlobals.hcSequence fields are closer than a cache line which permits false sharing. WARNING: the SharedGlobals.hcSequence field is closer to the struct end than a cache line which permits false sharing. INFO: error_cnt=0 INFO: warning_cnt=4 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) Client VM (build 1.9.0-internal-201407240020.ddaugher.8049737_for_jdk9_h, mixed mode) Hello world! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=65 $ bin/java -showversion -server -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp `cygpath -m $HOME` Hello INFO: sizeof(ObjectMonitor)=136 INFO: sizeof(PaddedEnd<ObjectMonitor>)=192 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=64 INFO: sizeof(SharedGlobals)=192 INFO: offset(stwRandom)=64 INFO: offset(_hcSequence)=128 INFO: error_cnt=0 INFO: warning_cnt=0 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) Server VM (build 1.9.0-internal-201407240020.ddaugher.8049737_for_jdk9_h, mixed mode) Hello world! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=65
25-07-2014

Comparison of ObjectMonitor::sanity_checks() output on Windows X64: $ uname -a CYGWIN_NT-6.1-WOW64 DDAUGHER-PC 1.7.16(0.262/5/3) 2012-07-20 22:55 i686 Cygwin $ bin/java -showversion -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp `cygpath -m $HOME` Hello INFO: sizeof(ObjectMonitor)=176 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=24 WARNING: the _header and _owner fields are closer than a cache line which permits false sharing. WARNING: ObjectMonitor size is not a multiple of a cache line which permits false sharing. INFO: sizeof(SharedGlobals)=272 INFO: offset(stwRandom)=64 INFO: offset(_hcSequence)=200 INFO: error_cnt=0 INFO: warning_cnt=2 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-internal-201407221344.pobajaj.hotspot, mixed mode) Hello world! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=65 $ bin/java -showversion -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp `cygpath -m $HOME` Hello INFO: sizeof(ObjectMonitor)=232 INFO: sizeof(PaddedEnd<ObjectMonitor>)=256 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=128 INFO: sizeof(SharedGlobals)=384 INFO: offset(stwRandom)=128 INFO: offset(_hcSequence)=256 INFO: error_cnt=0 INFO: warning_cnt=0 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-internal-201407240020.ddaugher.8049737_for_jdk9_h, mixed mode) Hello world! Deflate: InCirc=128 InUse=2 Scavenged=2 ForceMonitorScavenge=0 : pop=127 free=64
25-07-2014

Comparison of ObjectMonitor::sanity_checks() output on Linux X64: $ uname -a Linux slc06fdn 2.6.39-400.21.1.el6uek.x86_64 #1 SMP Thu Apr 4 03:49:00 PDT 2013 x86_64 x86_64 x86_64 GNU/Linux $ bin/java -showversion -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp ~ Hello INFO: sizeof(ObjectMonitor)=176 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=24 WARNING: the _header and _owner fields are closer than a cache line which permits false sharing. WARNING: ObjectMonitor size is not a multiple of a cache line which permits false sharing. INFO: sizeof(SharedGlobals)=272 INFO: offset(stwRandom)=64 INFO: offset(_hcSequence)=200 INFO: error_cnt=0 INFO: warning_cnt=2 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-internal-201407221344.pobajaj.hotspot, mixed mode) Hello World! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=34 $ bin/java -showversion -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp ~ Hello INFO: sizeof(ObjectMonitor)=232 INFO: sizeof(PaddedEnd<ObjectMonitor>)=256 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=128 INFO: sizeof(SharedGlobals)=384 INFO: offset(stwRandom)=128 INFO: offset(_hcSequence)=256 INFO: error_cnt=0 INFO: warning_cnt=0 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-internal-201407240020.ddaugher.8049737_for_jdk9_h, mixed mode) Hello World! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=34
25-07-2014

Comparison of ObjectMonitor::sanity_checks() output on Linux X86: $ uname -a Linux slc06fdn 2.6.39-400.21.1.el6uek.x86_64 #1 SMP Thu Apr 4 03:49:00 PDT 2013 x86_64 x86_64 x86_64 GNU/Linux $ bin/java -showversion -client -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp ~ Hello INFO: sizeof(ObjectMonitor)=108 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=16 WARNING: the _header and _owner fields are closer than a cache line which permits false sharing. WARNING: ObjectMonitor size is not a multiple of a cache line which permits false sharing. INFO: sizeof(SharedGlobals)=268 INFO: offset(stwRandom)=64 INFO: offset(_hcSequence)=200 INFO: error_cnt=0 INFO: warning_cnt=2 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) Client VM (build 1.9.0-internal-201407221344.pobajaj.hotspot, mixed mode) Hello World! Deflate: InCirc=128 InUse=2 Scavenged=2 ForceMonitorScavenge=0 : pop=127 free=33 $ bin/java -showversion -server -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp ~ Hello INFO: sizeof(ObjectMonitor)=108 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=16 WARNING: the _header and _owner fields are closer than a cache line which permits false sharing. WARNING: ObjectMonitor size is not a multiple of a cache line which permits false sharing. INFO: sizeof(SharedGlobals)=268 INFO: offset(stwRandom)=64 INFO: offset(_hcSequence)=200 INFO: error_cnt=0 INFO: warning_cnt=2 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) Server VM (build 1.9.0-internal-201407221344.pobajaj.hotspot, mixed mode) Hello World! Deflate: InCirc=128 InUse=2 Scavenged=2 ForceMonitorScavenge=0 : pop=127 free=33 $ bin/java -showversion -client -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp ~ Hello INFO: sizeof(ObjectMonitor)=100 INFO: sizeof(PaddedEnd<ObjectMonitor>)=128 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=32 WARNING: the _header and _owner fields are closer than a cache line which permits false sharing. INFO: sizeof(SharedGlobals)=96 INFO: offset(stwRandom)=32 INFO: offset(_hcSequence)=64 WARNING: the SharedGlobals.stwRandom field is closer to the struct beginning than a cache line which permits false sharing. WARNING: the SharedGlobals.stwRandom and SharedGlobals.hcSequence fields are closer than a cache line which permits false sharing. WARNING: the SharedGlobals.hcSequence field is closer to the struct end than a cache line which permits false sharing. INFO: error_cnt=0 INFO: warning_cnt=4 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) Client VM (build 1.9.0-internal-201407240020.ddaugher.8049737_for_jdk9_h, mixed mode) Hello World! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=34 $ bin/java -showversion -server -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp ~ Hello INFO: sizeof(ObjectMonitor)=132 INFO: sizeof(PaddedEnd<ObjectMonitor>)=192 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=64 INFO: sizeof(SharedGlobals)=192 INFO: offset(stwRandom)=64 INFO: offset(_hcSequence)=128 INFO: error_cnt=0 INFO: warning_cnt=0 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) Server VM (build 1.9.0-internal-201407240020.ddaugher.8049737_for_jdk9_h, mixed mode) Hello World! Deflate: InCirc=128 InUse=2 Scavenged=2 ForceMonitorScavenge=0 : pop=127 free=33
25-07-2014

Comparison of ObjectMonitor::sanity_checks() output on Solaris SPARCv9: $ uname -a SunOS terminus 5.11 11.0 sun4u sparc SUNW,SPARC-Enterprise $ bin/java -showversion -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp ~ Hello INFO: sizeof(ObjectMonitor)=176 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=24 WARNING: the _header and _owner fields are closer than a cache line which permits false sharing. WARNING: ObjectMonitor size is not a multiple of a cache line which permits false sharing. INFO: sizeof(SharedGlobals)=272 INFO: offset(stwRandom)=64 INFO: offset(_hcSequence)=200 INFO: error_cnt=0 INFO: warning_cnt=2 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-internal-201407221344.pobajaj.hotspot, mixed mode) Hello World! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=34 $ bin/java -showversion -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp ~ Hello INFO: sizeof(ObjectMonitor)=232 INFO: sizeof(PaddedEnd<ObjectMonitor>)=256 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=128 INFO: sizeof(SharedGlobals)=384 INFO: offset(stwRandom)=128 INFO: offset(_hcSequence)=256 INFO: error_cnt=0 INFO: warning_cnt=0 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-internal-201407240020.ddaugher.8049737_for_jdk9_h, mixed mode) Hello World! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=34
25-07-2014

Comparison of ObjectMonitor::sanity_checks() output on Solaris X64: $ uname -a SunOS svc6 5.11 11.1 i86pc i386 i86pc bin/java -showversion -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp ~ Hello INFO: sizeof(ObjectMonitor)=176 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=24 WARNING: the _header and _owner fields are closer than a cache line which permits false sharing. WARNING: ObjectMonitor size is not a multiple of a cache line which permits false sharing. INFO: sizeof(SharedGlobals)=272 INFO: offset(stwRandom)=64 INFO: offset(_hcSequence)=200 INFO: error_cnt=0 INFO: warning_cnt=2 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-internal-201407221344.pobajaj.hotspot, mixed mode) Hello World! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=34 bin/java -showversion -XX:+UnlockExperimentalVMOptions -XX:SyncKnobs=Verbose=1 -cp ~ Hello INFO: sizeof(ObjectMonitor)=232 INFO: sizeof(PaddedEnd<ObjectMonitor>)=256 INFO: L1_data_cache_line_size=64 INFO: offset(_header)=0 INFO: offset(_owner)=128 INFO: sizeof(SharedGlobals)=384 INFO: offset(stwRandom)=128 INFO: offset(_hcSequence)=256 INFO: error_cnt=0 INFO: warning_cnt=0 java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b23) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-internal-201407240020.ddaugher.8049737_for_jdk9_h, mixed mode) Hello World! Deflate: InCirc=128 InUse=2 Scavenged=3 ForceMonitorScavenge=0 : pop=127 free=34
25-07-2014

Setting to 'confidential' for now.
09-07-2014