JDK-6445836 : Dynamic enabling/disabling of extended DTrace probes is broken.
  • Type: Bug
  • Component: core-svc
  • Sub-Component: tools
  • Affected Version: 6
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: solaris_10
  • CPU: generic
  • Submitted: 2006-07-01
  • Updated: 2010-08-24
  • Resolved: 2006-07-27
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6
6 b93Fixed
Related Reports
Relates :  
JDK-6417266 - sun/management/jmxremote/bootstrap/LocalManagementTest.sh fails with IOException: Bad file number
Description
Dynamic enabling/disabling of extended DTrace probes is broken.
There are actually two problems.

1. Staring from JDK6b85 turning off ExtendedDTraceProbes flag in runtime does not work.
   Extended probes are continue to be fired.

   Note that jinfo reports that ExtendedDTraceProbes was disabled but
   DTraceMethodProbes, DTraceAllocProbes, DTraceMonitorProbes flags
   are reported as enabled.

   TESTCASE_1_2
   -> disable ExtendedDTraceProbes flag
      /net/vmsqe.sfbay/export/nightly/mustang/JDK/baseline/latest/solaris-sparc/bin/jinfo -flag -ExtendedDTraceProbes 9048
   -> check ExtendedDTraceProbes is disabled
      /net/vmsqe.sfbay/export/nightly/mustang/JDK/baseline/latest/solaris-sparc/bin/jinfo -flag ExtendedDTraceProbes 9048
      PASSED
   -> check DTraceMethodProbes is disabled
      /net/vmsqe.sfbay/export/nightly/mustang/JDK/baseline/latest/solaris-sparc/bin/jinfo -flag DTraceMethodProbes 9048
      FAILED
   -> check DTraceAllocProbes is disabled
      /net/vmsqe.sfbay/export/nightly/mustang/JDK/baseline/latest/solaris-sparc/bin/jinfo -flag DTraceAllocProbes 9048
      FAILED
   -> check DTraceMonitorProbes is disabled
      /net/vmsqe.sfbay/export/nightly/mustang/JDK/baseline/latest/solaris-sparc/bin/jinfo -flag DTraceMonitorProbes 9048
      FAILED


2. Dynamic turning on/off of DTraceMethodProbes, DTraceAllocProbes,
   DTraceMonitorProbes flags doesn't work at all.

   TESTCASE_2_1
   -> enable DTraceMethodProbes flag
   /net/vmsqe.sfbay/export/nightly/mustang/JDK/baseline/latest/solaris-sparc/bin/jinfo -flag +DTraceMethodProbes 9048
   Exception in thread "main" java.io.IOException: Command failed in target VM
        at sun.tools.attach.SolarisVirtualMachine.execute(SolarisVirtualMachine.java:130)
        at sun.tools.attach.HotSpotVirtualMachine.executeCommand(HotSpotVirtualMachine.java:195)
        at sun.tools.attach.HotSpotVirtualMachine.setFlag(HotSpotVirtualMachine.java:172)
        at sun.tools.jinfo.JInfo.flag(JInfo.java:111)
        at sun.tools.jinfo.JInfo.main(JInfo.java:58)
   TEST_PASSED=0

   TESTCASE_2_2
   -> disable DTraceMethodProbes flag
   /net/vmsqe.sfbay/export/nightly/mustang/JDK/baseline/latest/solaris-sparc/bin/jinfo -flag -DTraceMethodProbes 9048
   Exception in thread "main" java.io.IOException: Command failed in target VM
        at sun.tools.attach.SolarisVirtualMachine.execute(SolarisVirtualMachine.java:130)
        at  sun.tools.attach.HotSpotVirtualMachine.executeCommand(HotSpotVirtualMachine.java:195)
        at sun.tools.attach.HotSpotVirtualMachine.setFlag(HotSpotVirtualMachine.java:172)
        at sun.tools.jinfo.JInfo.flag(JInfo.java:115)
        at sun.tools.jinfo.JInfo.main(JInfo.java:58)
   ...

   Actually this functionality seems never worked.
   I tested with build b81 (where this functionality was introduced) and it doesn't work.

   I am not sure if it was designed this way. But it seems
   this functionality is useful. For instance consider following case:
   1. enable only DTraceMethodProbes by starting java with flag -XX:+DTraceMethodProbes
   2. disable it by running 'jinfo -flag -DTraceMethodProbes'
   3. but then it is not possible to turn on only DTraceMethodProbes.
      it is only possible to turn on all extended probes because only
      'jinfo -flag +ExtendedDTraceProbes' works.

Steps to reproduce the failures:
1. cp /home/ep155969/bugs/DynamicProbes_bug1/* YOUR_DIR
2. compile Dummy.java
3. ./check_dynamic_probes.sh JDK_HOME

   ex: ./check_dynamic_probes.sh /net/vmsqe.sfbay/export/nightly/mustang/JDK/baseline/latest/solaris-sparc/


The scripts are also attached.

Note: check_dynamic_probes.sh actually doesn't run any dtrace scripts yet.
      you can manually play with ObjectAllocation.d (and Alloc.java) to check for sure that probes
      are continue to be fired.
Comments
EVALUATION I confirmed that obervation reported in (part 1) description is correct and it is indeed a bug. When I checked all related b85 putbacks, I came across this change in build 85: 6417266 sun/management/jmxremote/bootstrap/LocalManagementTest.sh fails with IOException: Bad file number We essentially removed the logic for "last client detach" detection and call to detach_all operation. But we depend on last-client-detach event in elsewhere. When disabling ExtendedDTraceProbes flag, we delay sync'ing it with fine-grained DTrace probe flags. This is the root cause of the current problem.
11-07-2006
SUGGESTED FIX With fix for 6417266 (sun/management/jmxremote/bootstrap/LocalManagementTest.sh fails with IOException: Bad file number), we essentially removed (the incorrect) logic for "last-attach-on-demand-client-detach" detection and call to detach_all operation. When disabling ExtendedDTraceProbes flag, we delayed syncing it with fine-grained DTrace probe flags. This is the root cause of the current problem. Code in src/share/vm/services/dtraceAttacher.cpp depends on detach all event being called: void DTrace::set_extended_dprobes(bool flag) { // explicit setting of ExtendedDTraceProbes flag set_bool_flag("ExtendedDTraceProbes", flag); // make sure that the fine grained flags reflect the change. if (flag) { enable_dprobes(DTRACE_ALL_PROBES); } /* * disable_dprobes call is delayed till next "detach all "event. * This is done so that concurrent DTrace clients that may * have enabled one or more fine grained dprobes and may be * running still. On "detach all" clients event, we would * sync ExtendedDTraceProbes with fine grained flags which * would take care of disabling fine grained flags. */ } In the above logic, we don't reflect DTrace "overall" flag change into fine-grained flags immediately -- we instead wait for last-client-detach. This is because concurrent DTrace client could have used libjvm_dtrace.so (it does not as of now-- but it will) and turned a fine grained flag. So, we have to wait for all-client-detach and then disable fine grained flag! So, the correctness depends on all-client-detached event. We can probably not worry about DTrace using libjvm_dtrace.so and go with sync'ing "overall" flag with fine grained flags immediately. After fix, the above would look like: void DTrace::set_extended_dprobes(bool flag) { // explicit setting of ExtendedDTraceProbes flag set_bool_flag("ExtendedDTraceProbes", flag); // make sure that the fine grained flags reflect the change. if (flag) { enable_dprobes(DTRACE_ALL_PROBES); } else { // fix explanation here... disable_dprobes(DTRACE_ALL_PROBES); } }
11-07-2006
EVALUATION jinfo -flag is an interim solution to enable DTrace probes dynamically. As Mandy said, this is suppose to go away. We don't want to commit to too many flags being manageable/writable by jinfo. The CCC (http://ccc.sfbay/6402766) clearly stated about ExtendedDTraceProbes and manageable flags (as defined for HotSpotDiagnosticMBean) only. This bug is accepted for investigating the first part of the bug (i.e., Staring from JDK6b85 turning off ExtendedDTraceProbes flag in runtime does not work).
11-07-2006
EVALUATION DTraceMethodProbes, DTraceAllocProbes, and DTraceMonitorProbes flags are internal flags and not designed for users to be dynamically set via jinfo. The jinfo -flag +ExtendedDTraceProbes is an interim solution until DTrace is changed to use the new JVM/DTrace interface which allows to enable one type of extended probes instead of all. Accept this bug to further investigate problem #1.
10-07-2006