JDK-6595055 : orbd consumes 100% cpu
  • Type: Bug
  • Component: other-libs
  • Sub-Component: corba:orb
  • Affected Version: 7
  • Priority: P3
  • Status: Closed
  • Resolution: Incomplete
  • OS: linux
  • CPU: x86
  • Submitted: 2007-08-20
  • Updated: 2014-08-25
  • Resolved: 2013-10-03
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7
7Resolved
Related Reports
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.7.0-ea"
Java(TM) SE Runtime Environment (build 1.7.0-ea-b16)
Java HotSpot(TM) Tiered VM (build 1.7.0-ea-b16, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Linux io 2.6.18.8-0.5-bigsmp #1 SMP Fri Jun 22 12:17:53 UTC 2007 i686 i686 i386 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
Trying to get all bindings from the transient nameserver brings orbd into a state where it consumes 100% CPU. Its interesting to note that the problem only occurs if the client is programmed in c++. I was not able to reproduce the problem with a client programmed in Java.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1) Get omniORB 4.1 from http://omniorb.sourceforge.net

2) Compile omniORB (requires python devel package installed)
cd /tmp
mkdir omni_local
tar xvzf omniORB-4.1.0.tar.gz
cd omniORB-4.1.0
./configure --prefix=/tmp/omni_local
make
make install

3) Compile the test program (binding_browser, source attached to this report)
g++ -I/tmp/omni_local/include -L/tmp/omni_local/lib -lomniORB4 -lomniDynamic4 -lomnithread -lpthread -lrt BindingBrowser.cc -o binding_browser

4) Start orbd
<JAVA_HOME>/bin/orbd -ORBInitialPort 12345

5) Start binding_browser (from another shell)
5a) export LD_LIBRARY_PATH=/tmp/omni_local/lib:$LD_LIBRARY_PATH
5b) ./binding_browser -ORBInitRef NameService=corbaloc::1.2@localhost:12345/TNameService

Repeat step 5b until orbd consumes 100% cpu.
(On my machine mostly 2 attempts are enough)

  To start the Java implementation (BindingBrowser.java) which does not show the problem:
java BindingBrowser -ORBInitialPort 12345


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
orbd does not consume 100% cpu after call to binding_browser has finished.
ACTUAL -
orbd consumes 100% cpu after call to binding_browser has finished.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
---------------------------------------------------------------------------------------------------------------
BindingBrowser.cc
---------------------------------------------------------------------------------------------------------------
#include <iostream>
#include <sstream>
#include <sys/time.h>

#include "omniORB4/CORBA.h"
#include "omniORB4/Naming.hh"

#define NUM_LOOPS 1

static long long currentTimeMillis();
static void addLevel(std::ostringstream& out, int level);
static void printBinding(CosNaming::Binding& binding,
        CosNaming::NamingContext_ptr root, int level,
        std::ostringstream& out);
static void printNamingContext(CosNaming::NamingContext_ptr root, int level,
        std::ostringstream& out);

static long long currentTimeMillis() {
    struct timeval tv;
    gettimeofday(&tv, 0);
    double value = static_cast<double>(tv.tv_sec)*1000.0 +
        static_cast<double>(tv.tv_usec)/1000.0;
    return static_cast<long long>(value + 0.5);
}

static void addLevel(std::ostringstream& out, int level) {
    for (int i = 0; i < level; i++) {
        out << "    ";
    }
}

static void printBinding(CosNaming::Binding& binding,
        CosNaming::NamingContext_ptr root, int level,
        std::ostringstream& out) {
    for (CORBA::ULong i = 0; i < binding.binding_name.length(); i++) {
        addLevel(out, level);
        out << binding.binding_name[i].id;
        if (binding.binding_name[i].kind != 0 &&
                strlen(binding.binding_name[i].kind) > 0) {
            out << "[" << binding.binding_name[i].kind << "]";
        }
        out << "\n";
    }
    if (binding.binding_type == CosNaming::ncontext) {
        try {
            CORBA::Object_var objRef = root->resolve(binding.binding_name);
            CosNaming::NamingContext_var nc =
                CosNaming::NamingContext::_narrow(objRef);
            printNamingContext(nc, level+1, out);
        } catch(const CosNaming::NamingContext::NotFound &ex) {
            std::cerr << "Caught CosNaming::NamingContext::NotFound"
                <<std::endl;
        } catch(const CosNaming::NamingContext::CannotProceed &ex) {
            std::cerr << "Caught CosNaming::NamingContext::CannotProceed"
                <<std::endl;
        } catch(const CosNaming::NamingContext::InvalidName &ex) {
            std::cerr << "Caught CosNaming::NamingContext::InvalidName"
                <<std::endl;
        }
    }
}

static void printNamingContext(CosNaming::NamingContext_ptr root, int level,
        std::ostringstream& out) {
    CosNaming::BindingList_var bindingList;
    CosNaming::BindingIterator_var bindingIterator;
    root->list(0, bindingList, bindingIterator);

    CosNaming::Binding_var binding;
    if (!CORBA::is_nil(bindingIterator)) {
        while (bindingIterator->next_one(binding)) {
            printBinding(binding, root, level, out);
        }
        bindingIterator->destroy();
    }
}

int main(int argc, char** argv) {
    try {
        // Create and initialize the ORB
        CORBA::ORB_var orb = CORBA::ORB_init(argc, argv);

        // Get the root naming context for the transient name service.
        CORBA::Object_var objRef =
            orb->resolve_initial_references("NameService");
        CosNaming::NamingContext_var root =
            CosNaming::NamingContext::_narrow(objRef);

        // Print all bindings starting from root NamingContext
        long long t0 = currentTimeMillis();
        for (int i = 0; i < NUM_LOOPS; i++) {
            if (i%10000 == 0 && i != 0) {
                long long t1 = currentTimeMillis();
                std::cout << "NumCalls: " << i << "   Calls/s: "
                    << (static_cast<double>(i * 1000) /
                    static_cast<double>(t1-t0)) << std::endl;
            }
            std::ostringstream out;
            printNamingContext(root, 0, out);
            if (out.str().length() > 0) {
                std::cout << out.str() << std::endl;
            }
        }
        orb->destroy();
    } catch(const CORBA::Exception &ex) {
        std::cerr << "Caught CORBA::Exception" << std::endl;
    }
}

---------------------------------------------------------------------------------------------------------------
BindingBrowser.java
---------------------------------------------------------------------------------------------------------------
import org.omg.CosNaming.*;
import org.omg.CosNaming.NamingContextPackage.*;
import org.omg.CORBA.*;
import org.omg.PortableServer.*;
import org.omg.PortableServer.POA;

public class BindingBrowser {
    private static final int NUM_LOOPS = 1;

    public static void main(String args[]) {
        ORB orb = null;
        try{
            // Create and initialize the ORB
            orb = ORB.init(args, null);

            // Get the root naming context for the transient name service.
            org.omg.CORBA.Object objRef =
                orb.resolve_initial_references("TNameService");
            NamingContext root = NamingContextHelper.narrow(objRef);

            // Print all bindings starting from root NamingContext
            long t0 = System.currentTimeMillis();
            for(int i = 0; i < NUM_LOOPS; i++) {
                if (i%10000 == 0 && i != 0) {
                    long t1 = System.currentTimeMillis();
                    System.out.println("NumCalls: " + i + "   Calls/s: " +
                        (double)(i * 1000) / (double)(t1-t0));
                }
                StringBuilder sb = new StringBuilder();
                printNamingContext(root, 0, sb);

                String bindings = sb.toString();
                if (bindings.length() > 0) {
                    System.out.println(bindings);
                }
            }

        } catch (Exception ex) {
            ex.printStackTrace();
        } finally {
            if (orb != null) {
                orb.destroy();
            }
        }
    }

    private static void printNamingContext(NamingContext root, int level,
            StringBuilder sb) {
        BindingListHolder bindingListHolder =
            new BindingListHolder();
        BindingIteratorHolder bindingIteratorHolder =
            new BindingIteratorHolder();
        root.list(0, bindingListHolder, bindingIteratorHolder);

        BindingIterator bindingIterator = bindingIteratorHolder.value;
        BindingHolder bindingHolder = new BindingHolder();
        if (bindingIterator != null) {
            while (bindingIterator.next_one(bindingHolder)) {
                Binding binding = bindingHolder.value;
                printBinding(binding, root, level, sb);
            }
            bindingIterator.destroy();
        }
    }
    private static void printBinding(Binding binding, NamingContext root,
            int level, StringBuilder sb) {
        NameComponent[] nameComponents = binding.binding_name;
        for (NameComponent nameComponent : nameComponents) {
            addLevel(sb, level);
            sb.append(nameComponent.id);
            if (nameComponent.kind != null && nameComponent.kind.length() > 0) {
                sb.append("[");
                sb.append(nameComponent.kind);
                sb.append("]");
            }
            sb.append("\n");
        }
        if (binding.binding_type == BindingType.ncontext) {
            try {
                org.omg.CORBA.Object obj = root.resolve(nameComponents);
                NamingContext nc = NamingContextHelper.narrow(obj);
                printNamingContext(nc, level+1, sb);
            } catch (org.omg.CosNaming.NamingContextPackage.NotFound ex) {
                ex.printStackTrace();
            } catch (org.omg.CosNaming.NamingContextPackage.CannotProceed ex) {
                ex.printStackTrace();
            } catch (org.omg.CosNaming.NamingContextPackage.InvalidName ex) {
                ex.printStackTrace();
            }
        }
    }

    private static void addLevel(StringBuilder sb, int level) {
        for (int i = 0; i < level; i++) {
            sb.append("    ");
        }
    }
}

---------- END SOURCE ----------

Comments
A profiler snapshot or even a stacktrace dump of the ORBD process at time of 100% CPU would have been useful. Closing as incomplete.
03-10-2013

EVALUATION The problem is around a channel that was originally registered with Selector for i/o gets closed on the server side (due to early client side exit). But the server side can know about such channel only when it does i/o (read/write) and thereby getting into an IO exception. In this case, (bug 6595055)there are times (erroneous) when server side (selector) did not know the channel is already closed (peer-reset), but continue to do the selection cycle on a key set whose associated channel is alreay closed or invalid. Hence, selector's slect(..) keep spinging with zero return without blocking for the timeout period One fix is to have a provision in the application, to check if any of the Selector's keyset is having a closed channel/or invalid registration due to channel closure.
30-01-2008