OPERATING SYSTEM(S)
ALL
FULL JDK VERSION(S):
1.3.1, 1.4.2, 5.0
DESCRIPTION:
Deadlock is caused because of the frequent synchronization being used by DriverManager code to access the "drivers" vector.
Following 2 threads shows deadlock::
3XMTHREADINFO "Worker-1" (TID:0x7EF26078, sys_thread_t:0x3D7757A0, state:MW, native ID:0x518A) prio=5
4XESTACKTRACE at java.sql.DriverManager.getDrivers(DriverManager.java(Compiled Code)) <<<<<<< CHECK
4XESTACKTRACE at com.sybase.jdbc2.jdbc.SybDriver.registerWithDriverManager(SybDriver.java:551)
4XESTACKTRACE at com.sybase.jdbc2.jdbc.SybDriver.<init>(SybDriver.java:109)
4XESTACKTRACE at com.sybase.jdbc2.jdbc.SybDriver.<clinit>(SybDriver.java:75)
4XESTACKTRACE at java.lang.Class.forName0(Native Method)
4XESTACKTRACE at com.ssc.faw.util.GenConnectionPool.loadDriver(GenConnectionPool.java(Inlined Compiled Code))
4XESTACKTRACE at com.ssc.faw.util.GenConnectionPool.registerDataSources(GenConnectionPool.java(Compiled Code))
4XESTACKTRACE at com.ssc.faw.util.GenConnectionPool.<init>(GenConnectionPool.java:21)
4XESTACKTRACE at com.ssc.faw.util.GenConnectionPool.<clinit>(GenConnectionPool.java:10)
4XESTACKTRACE at com.ssc.faw.intref2.IDF_Data.hasConnectionSelector(IDF_Data.java:230)
4XESTACKTRACE at com.ssc.faw.intref2.Service.process(Service.java:54)
4XESTACKTRACE at com.ssc.faw.intref2.Service.process(Service.java:185)
4XESTACKTRACE at com.ssc.eis.api.retriever.FundDateRetriever.getFund(FundDateRetriever.java:452)
4XESTACKTRACE at com.ssc.eis.api.retriever.FundDateRetriever.run(FundDateRetriever.java:97)
4XESTACKTRACE at com.ssc.faw.util.WorkerThread.run(WorkerThread.java:48)
3XMTHREADINFO "ORB.thread.pool : 5" (TID:0x73D0B650, sys_thread_t:0x3CE0EF50, state:CW, native ID:0x437C) prio=5
4XESTACKTRACE at java.lang.Class.forName0(Native Method)
4XESTACKTRACE at java.lang.Class.forName(Class.java(Compiled Code))
4XESTACKTRACE at java.sql.DriverManager.getCallerClass(DriverManager.java(Inlined Compiled Code))
4XESTACKTRACE at java.sql.DriverManager.getDrivers(DriverManager.java(Compiled Code)) <<<<<< CHECK
4XESTACKTRACE at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java(Compiled Code))
4XESTACKTRACE at java.sql.DriverManager.getConnection(DriverManager.java(Compiled Code))
4XESTACKTRACE at java.sql.DriverManager.getConnection(DriverManager.java(Compiled Code))
4XESTACKTRACE at com.ssc.eis.ntd.mss.framework.ConnectionManager.getJDBCConnection(ConnectionManager.java(Compiled Code))
4XESTACKTRACE at com.ssc.eis.ntd.mss.framework.ConnectionManager.getConnection(ConnectionManager.java(Compiled Code))
4XESTACKTRACE at com.ssc.eis.ntd.mss.ds.mds.MetaDataService.getConnection(MetaDataService.java(Compiled Code))
4XESTACKTRACE at com.ssc.eis.ntd.mss.ds.mds.MetaDataService.getConnection(MetaDataService.java(Inlined Compiled Code))
4XESTACKTRACE at com.ssc.eis.ntd.mss.ds.mds.MetaDataService.getDataviewSourceObjects(MetaDataService.java(Compiled Code))
4XESTACKTRACE at com.ssc.eis.ntd.mss.ds.mds.DataviewSourceFS.getObjects(DataviewSourceFS.java(Compiled Code))
4XESTACKTRACE at com.ssc.eis.ntd.mss.ds.mds.DataviewSourceFS.getNumObjects(DataviewSourceFS.java:114)
4XESTACKTRACE at com.ssc.eis.ntd.mss.ds.cswdata.map2db.RMapCSWData2DB.process(RMapCSWData2DB.java:198)
4XESTACKTRACE at com.ssc.eis.ntd.mss.ds.dataservice.Map2DBView.getView(Map2DBView.java:81)
4XESTACKTRACE at com.ssc.mss.services.ReportQueryManagerBean.executeScheduledQuery(Unknown Source)
4XESTACKTRACE at com.ssc.mss.services.EJSRemoteStatelessReportQueryManager_7332c029.executeScheduledQuery(Unknown Source)
4XESTACKTRACE at com.ssc.mss.services._EJSRemoteStatelessReportQueryManager_7332c029_Tie.executeScheduledQuery(Unknown Source)
4XESTACKTRACE at com.ssc.mss.services._EJSRemoteStatelessReportQueryManager_7332c029_Tie._invoke(Unknown Source)
4XESTACKTRACE at com.ibm.CORBA.iiop.ServerDelegate.dispatchInvokeHandler(ServerDelegate.java:604)
4XESTACKTRACE at com.ibm.CORBA.iiop.ServerDelegate.dispatch(ServerDelegate.java:457)
4XESTACKTRACE at com.ibm.rmi.iiop.ORB.process(ORB.java:399)
4XESTACKTRACE at com.ibm.CORBA.iiop.ORB.process(ORB.java:1598)
4XESTACKTRACE at com.ibm.rmi.iiop.Connection.doWork(Connection.java:2166)
4XESTACKTRACE at com.ibm.rmi.iiop.WorkUnitImpl.doWork(WorkUnitImpl.java:63)
4XESTACKTRACE at com.ibm.ejs.oa.pool.PooledThread.run(ThreadPool.java:95)
4XESTACKTRACE at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:672)
This problem is not recreatable, but is seen when Thread 1 is initialising a SQL driver which has two calls to synchronized methods in DriverManager: namely registerDriver() and getDrivers(). Each of these calls acquires and releases the lock on DriverManager.class.
On Thread 2, we take the lock on DriverManager.class, and then we iterate thru the drivers, running code that requires that the class of each driver is fully-initialized ("getCallerClass" ).
Thread 1:
initialize SQL Driver --> out of TWO static synchronized methods calls on DriverManager, it has finished calling registerDriver(), and about to call getDrivers(),
At this point of time, Thread 2 comes in aquires the DriverManager.class lock , so inorder to proceed further, now this thread is waiting for
DriverManager.class lock to be released by Thread 2.
Thread 2:
DriverManager.getDrivers() --> getCallerClass() call causes waiting (in classloader code) on initializing of SybDriver (and every other driver in
DriverManager vector) which is happening on the other thread, Thread 1. Leading to the deadlock kind of situation.
PROBLEM RESOLUTION:
Here the problem is caused by frequent locking on DriverManager.class by static methods implemented in DriverManager code, which also does
getCallerClass (which waits until the class intialization to be complete).
The solution is to have a copy of the shared datastructure for reading purposes (contents of the datastructure can be read without holding locks), however to get the address of this read copy of the shared datastructure we need to hold a lock.
For updating the write copy of shared datastructure, we hold a lock and once the update is done on the write copy of shared datastructure, we make a new copy of this and update the address of the read copy under the locks to this new copy. This will ensure that we achieve necessary synchronization in updating/reading of the shared datastructure.
Fix Details:
1> Remove synchronized keyword from the following methods::
getConnection()
getDriver()
getDrivers()
2> Instead of drivers vector, use 2 copies for this (one write copy and another write copy) "writeDrivers" and "readDrivers" .
3> In methods we want to just read the drivers vector (like getConnection() , getDriver() and getDrivers() ) read the "readDrivers" in a synchronized block on DriverManager.class .
4> In registerDriver() and deregisterDriver(), where we want update the drivers vector, update the "writeDrivers" and make a clone of this and assign it to "readDrivers".
Changes are in ...../java/sql/DriverManager.java file.
[This bug is being submitted as a courtesy, in order to maintain uniformity between Sun & IBM JDKs. It has been fixed in IBM JDKs.
Please contact ###@###.### if you have questions.]