Bug ID: JDK-6322825 (se) Selector fails with invalid argument on Solaris 10

JDK-6322825 : (se) Selector fails with invalid argument on Solaris 10

Type: Bug
Component: core-libs
Sub-Component: java.nio
Affected Version: 9.0ee,9.0pe,5.0u6,5.0u7,6

Priority: P3
Status: Resolved
Resolution: Fixed
OS: solaris_9,solaris_10
CPU: generic,sparc

Submitted: 2005-09-12
Updated: 2011-03-23
Resolved: 2005-10-14

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.

JDK 6
6 b57Fixed

Related Reports

Duplicate :	JDK-6547113 - ulimit issue with 5.0 U7 on Solaris 10
Duplicate :	JDK-6387136 - orbd consumes one cpu when ulimit -n is used
Relates :	JDK-6782668 - (se) IOException: Invalid argument" thrown on a call to Selector.select(value) with -d64

Description

Starting glassfish using JDK6 fails with the following exceptions in the log file.


>>----------------------------------
>>>>
>>>>[#|2005-09-12T13:36:11.002-0500|SEVERE|sun-appserver-pe9.0|javax.enterprise.system.core|_ThreadID=10;com.sun.enterprise.admin.server.core.JmxConnectorLifecycle@141b736;com.sun.appserv.server.ServerLifecycleException:
>>>>Cannot bind to URL
>>>>[rmi://dhcp-uaus09-147-55:8686/management/rmi-jmx-connector]:
>>>>javax.naming.CommunicationException [Root exception is
>>>>java.rmi.ConnectIOException: error during JRMP connection establishment;
>>>>nested exception is:
>>>>       java.io.IOException];|Service
>>>>com.sun.enterprise.admin.server.core.JmxConnectorLifecycle@141b736
>>>>cannot be started! : com.sun.appserv.server.ServerLifecycleException:
>>>>Cannot bind to URL
>>>>[rmi://dhcp-uaus09-147-55:8686/management/rmi-jmx-connector]:
>>>>javax.naming.CommunicationException [Root exception is
>>>>java.rmi.ConnectIOException: error during JRMP connection establishment;
>>>>nested exception is:
>>>>       java.io.IOException]|#]
>>>>
>>>>[#|2005-09-12T13:36:11.046-0500|SEVERE|sun-appserver-pe9.0|javax.enterprise.system.core|_ThreadID=10;|CORE5071:
>>>>An error occured during initialization
>>>>com.sun.appserv.server.ServerLifecycleException: Cannot bind to URL
>>>>[rmi://dhcp-uaus09-147-55:8686/management/rmi-jmx-connector]:
>>>>javax.naming.CommunicationException [Root exception is
>>>>java.rmi.ConnectIOException: error during JRMP connection establishment;
>>>>nested exception is:
>>>>       java.io.IOException]
>>>>       at
>>>>com.sun.enterprise.admin.server.core.JmxConnectorLifecycle.onStartup(JmxConnectorLifecycle.java:123)
>>>>       at
>>>>com.sun.enterprise.server.ApplicationServer.onStartup(ApplicationServer.java:324)
>>>>       at
>>>>com.sun.enterprise.server.ondemand.OnDemandServer.onStartup(OnDemandServer.java:110)
>>>>       at com.sun.enterprise.server.PEMain.run(PEMain.java:312)
>>>>       at com.sun.enterprise.server.PEMain.main(PEMain.java:246)
>>>>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>       at
>>>>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>       at
>>>>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>       at java.lang.reflect.Method.invoke(Method.java:585)
>>>>       at com.sun.enterprise.server.PELaunch.main(PELaunch.java:202)
>>>>Caused by: java.io.IOException: Cannot bind to URL
>>>>[rmi://dhcp-uaus09-147-55:8686/management/rmi-jmx-connector]:
>>>>javax.naming.CommunicationException [Root exception is
>>>>java.rmi.ConnectIOException: error during JRMP connection establishment;
>>>>nested exception is:
>>>>       java.io.IOException]
>>>>       at
>>>>javax.management.remote.rmi.RMIConnectorServer.newIOException(RMIConnectorServer.java:812)
>>>>       at
>>>>javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:422)
>>>>       at
>>>>com.sun.enterprise.admin.jmx.remote.server.rmi.JmxConnectorServerDriver.startConnectorServer(JmxConnectorServerDriver.java:190)
>>>>       at
>>>>com.sun.enterprise.admin.server.core.JmxConnectorLifecycle.onStartup(JmxConnectorLifecycle.java:115)
>>>>       ... 9 more
>>>>Caused by: javax.naming.CommunicationException [Root exception is
>>>>java.rmi.ConnectIOException: error during JRMP connection establishment;
>>>>nested exception is:
>>>>       java.io.IOException]
>>>>       at
>>>>com.sun.jndi.rmi.registry.RegistryContext.bind(RegistryContext.java:126)
>>>>       at
>>>>com.sun.jndi.toolkit.url.GenericURLContext.bind(GenericURLContext.java:208)
>>>>       at javax.naming.InitialContext.bind(InitialContext.java:359)
>>>>       at
>>>>javax.management.remote.rmi.RMIConnectorServer.bind(RMIConnectorServer.java:633)
>>>>       at
>>>>javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:418)
>>>>       ... 11 more
>>>>Caused by: java.rmi.ConnectIOException: error during JRMP connection
>>>>establishment; nested exception is:
>>>>       java.io.IOException
>>>>       at
>>>>sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:274)
>>>>       at
>>>>sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:171)
>>>>       at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:323)
>>>>       at sun.rmi.registry.RegistryImpl_Stub.bind(Unknown Source)
>>>>       at
>>>>com.sun.jndi.rmi.registry.RegistryContext.bind(RegistryContext.java:120)
>>>>       ... 15 more
>>>>Caused by: java.io.IOException
>>>>       at
>>>>com.sun.enterprise.server.ss.provider.ASOutputStream.waitForSelect(ASOutputStream.java:101)
>>>>       at
>>>>com.sun.enterprise.server.ss.provider.ASOutputStream.write(ASOutputStream.java:128)
>>>>       at
>>>>java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>>>>       at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
>>>>       at java.io.DataOutputStream.flush(DataOutputStream.java:106)
>>>>       at
>>>>sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198)
>>>>       ... 19 more
>>>>Caused by: java.io.IOException: Invalid argument
>>>>       at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
>>>>       at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:158)
>>>>       at
>>>>sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68)
>>>>       at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>>>>       at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>>>>       at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
>>>>       at
>>>>com.sun.enterprise.server.ss.provider.ASSelector.select(ASSelector.java:126)
>>>>       at
>>>>com.sun.enterprise.server.ss.provider.ASOutputStream.waitForSelect(ASOutputStream.java:91)
>>>>       ... 24 more
>>>>|#]
>>>>
>>>> 
>>>>
>>>>    
>>>>
>
>>
>>  
>>

Comments

EVALUATION --
05-12-2006
EVALUATION -- This bug will also be fixed in 5.0 update 8.
25-02-2006
WORK AROUND There are two possible workarounds: 1. Use the poll based SelectorProvider by setting the java.nio.channels.spi.SelectorProvider system property to sun.nio.ch.PollSelectorProvider 2. Increase the hard limit on the number of file descriptors to 8193 or higher, ie: check the hard limit with "ulimit -n -H" and if less than 8193 edit /etc/system and add a "set rlim_fd_max=8193" command and reboot.
28-09-2005
EVALUATION -- The /dev/poll Selector allocates an array of 8192 pollfd entries for use by the Selector. This is exceeeds the nofiles ulimit and causing it to fail with an "invalid argument" error. The submitter can workaround the issue by setting the java.nio.channels.spi.SelectorProvider system property to sun.nio.ch.PollSelectorProvider, eg: java -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider App This bug needs to be fixed in mustang and probably a 5.0 update release too.
27-09-2005
EVALUATION -- I was able to duplicate on dhcp-uaus09-147-55 but not on other Solaris 10 machines. It duplicates with mustang-b53 and the latest build of 5.0u6-b02. The submitter noted that it didn't duplicate on 5.0 but I assume this is because the poll selector was used instead of /dev/poll (6286011). We need to track down if this is a patch or configuration issue. Alternatively this could be lurking bug in the /dev/poll selector implementation that only surfaces on Solaris 10.
24-09-2005
EVALUATION Finally the bug was reproducable in Vivek's Solaris 10 machine (dhcp-uaus09-147-55.central.sun.com). SO FAR I HAVE SEEN THIS APPEARING IN ONLY THAT MACHINE. It wasnt reproducable in another Solaris 10 machine, that I tried in India domain. After more debugging, I could write a small standalone java program that reproduces the error. Even simple select opration throw an IOException as follows. java.io.IOException: Invalid argument at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method) at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:158) at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:68) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84) at t$HelperThread.run(t.java:32) at t.test(t.java:21) at t.main(t.java:8) The program binds a server socket channel on a specific port, sets the channel to non-blocking mode ,register a selector on OP_ACCEPT and then do select() on the selector. The example program is attatched. The example program works as expected in JDK 1.5 and fails in mustang b51. I am transferring this bug to JDK team, so that they can give their analysis about the issue.
24-09-2005
EVALUATION From the stacktrace given in the description, the IOException seems to be appearing from mustang native code. ASSocketService's selector.select() operation fails with this IOException. So far could not found any thing wrong in the code. I tried with the latest mustang build (51) in my linux and solaris box. I am not able to reproduce the bug. I am marking the bug as incomplete until I receive more information about reproducing this bug. Please update the bug with the information about which combination of JDK/GlassFish/OS can reproduce this bug. Here are the steps I followed in both the OS's 1. Took the latest GlassFish binary 2. edit AS_HOME/config/asenv.conf to point to mustang buld. See the version output below.. 3. start-domain, stop-domain, again start-domain, stop-domain 4. asadmin list-jdbc-resources. Everything works fine. I have attached the server.log for both the cases. <snip> java version "1.6.0-ea" Java(TM) 2 Runtime Environment, Standard Edition (build 1.6.0-ea-b51) Java HotSpot(TM) Server VM (build 1.6.0-ea-b51, mixed mode) </snip>
16-09-2005