Question

1
Replies
220
Views
Anandh Palanisamy Member since 2013 62 posts
Ford Motor Company
Posted: 2 years ago
Last activity: 2 years 7 months ago
Closed
Solved

System Hung or Thread hung during Load Declarative Pega Definition Cache in WAS 8.5.5

Hi Team,

We have upgraded from 6.2 to 7.3, We are seeing this error and system is very slow.

[3/19/18 11:08:58:244 EDT] 00000089 SystemOut     O 2018-03-19 11:08:58,243 [as3548.md1q.ford.com] [  STANDARD] [                    ] [                    ] (   external.async.IStartupTask) INFO    - load Declarative Page Definition Cache  ..starting
[3/19/18 11:09:21:382 EDT] 00000082 ThreadMonitor W   WSVR0605W: Thread "server.startup : 0" (00000066) has been active for 713448 milliseconds and may be hung.  There is/are 1 thread(s) in total in the server that may be hung.
	at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:719)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1178)
	at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1383)
	at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:639)
	at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:972)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:777)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl$5.run(ApplicationMgrImpl.java:2197)
	at com.ibm.ws.security.auth.ContextManagerImpl.runAs(ContextManagerImpl.java:5486)
	at com.ibm.ws.security.auth.ContextManagerImpl.runAsSystem(ContextManagerImpl.java:5612)
	at com.ibm.ws.security.core.SecurityContext.runAsSystem(SecurityContext.java:255)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:2202)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:435)
	at com.ibm.ws.runtime.component.CompositionUnitImpl.start(CompositionUnitImpl.java:123)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:378)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.access$500(CompositionUnitMgrImpl.java:126)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$CUInitializer.run(CompositionUnitMgrImpl.java:984)
	at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:502)
	at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1881)
[3/19/18 11:09:21:383 EDT] 00000082 ThreadMonitor W   WSVR0605W: Thread "server.startup : 1" (00000067) has been active for 713154 milliseconds and may be hung.  There is/are 2 thread(s) in total in the server that may be hung.
	at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:719)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1178)
	at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1383)
	at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:639)
	at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:972)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:777)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl$5.run(ApplicationMgrImpl.java:2197)
	at com.ibm.ws.security.auth.ContextManagerImpl.runAs(ContextManagerImpl.java:5486)
	at com.ibm.ws.security.auth.ContextManagerImpl.runAsSystem(ContextManagerImpl.java:5612)
	at com.ibm.ws.security.core.SecurityContext.runAsSystem(SecurityContext.java:255)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:2202)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:435)
	at com.ibm.ws.runtime.component.CompositionUnitImpl.start(CompositionUnitImpl.java:123)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:378)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.access$500(CompositionUnitMgrImpl.java:126)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$CUInitializer.run(CompositionUnitMgrImpl.java:984)
	at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:502)
	at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1881)
[3/19/18 11:09:21:385 EDT] 00000082 ThreadMonitor W   WSVR0605W: Thread "eccvas3548.md1q.ford.com" (00000076) has been active for 713455 milliseconds and may be hung.  There is/are 3 thread(s) in total in the server that may be hung.
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:127)
	at java.net.SocketInputStream.read(SocketInputStream.java:181)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1782)
	at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:4838)
	at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:6154)
	at com.microsoft.sqlserver.jdbc.SQLServerResultSet$CursorFetchCommand.doExecute(SQLServerResultSet.java:4903)
	at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
	at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)
	at com.microsoft.sqlserver.jdbc.SQLServerResultSet.doServerFetch(SQLServerResultSet.java:4940)
	at com.microsoft.sqlserver.jdbc.SQLServerResultSet.next(SQLServerResultSet.java:1058)
	at com.ibm.ws.rsadapter.jdbc.WSJdbcResultSet.next(WSJdbcResultSet.java:3114)
	at com.pega.pegarules.data.internal.store.rdbms.DatabaseResultSetForwardOnly.next(DatabaseResultSetForwardOnly.java:120)
	at com.pega.pegarules.data.internal.access.ResultSetIteratorImpl.<init>(ResultSetIteratorImpl.java:195)
	at com.pega.pegarules.data.internal.access.ResultSetPackagerImpl.packageResults(ResultSetPackagerImpl.java:206)
	at com.pega.pegarules.data.internal.access.Lister.listWithResultPackager(Lister.java:407)
	at com.pega.pegarules.data.internal.access.Lister.list(Lister.java:185)
	at com.pega.pegarules.data.internal.access.DatabaseImpl.list(DatabaseImpl.java:2941)
	at com.pega.pegarules.data.internal.access.DatabaseImpl.list(DatabaseImpl.java:2933)
	at com.pega.pegarules.exec.internal.declare.infengine.cbi.GlobalDeclarativeClass.getAllPropertyDefinitions(GlobalDeclarativeClass.java:359)
	at com.pega.pegarules.exec.internal.declare.infengine.cbi.GlobalDeclarativeClass.<init>(GlobalDeclarativeClass.java:244)
	at com.pega.pegarules.exec.internal.declare.infengine.cbi.GlobalCache.getGlobalClass(GlobalCache.java:704)
	at com.pega.pegarules.exec.internal.declare.infengine.cbi.DeclarativeCacheManagerImpl.getGlobalClass(DeclarativeCacheManagerImpl.java:1796)
	at com.pega.pegarules.exec.internal.declare.infengine.cbi.RuntimeDeclarativeClassCore.resolveRules(RuntimeDeclarativeClassCore.java:398)
	at com.pega.pegarules.exec.internal.declare.infengine.cbi.RuntimeDeclarativeClassCore.<init>(RuntimeDeclarativeClassCore.java:354)
	at com.pega.pegarules.exec.internal.declare.infengine.cbi.RuntimeDeclarativeClass.<init>(RuntimeDeclarativeClass.java:134)
	at com.pega.pegarules.exec.internal.declare.infengine.cbi.RuntimeCacheImpl.getRuntimeClass(RuntimeCacheImpl.java:313)
	at com.pega.pegarules.exec.internal.declare.infengine.cbi.DeclarativeCacheManagerImpl.getRuntimeClass(DeclarativeCacheManagerImpl.java:1638)
	at com.pega.pegarules.exec.internal.declare.infengine.cbi.ProcessingSupportImpl.getPropertyCandidates(ProcessingSupportImpl.java:108)
	at com.pega.pegarules.exec.internal.declare.infengine.cbi.ProcessingSupportImpl.getAutoChainCandidates(ProcessingSupportImpl.java:165)
	at com.pega.pegarules.data.internal.clipboard.DeclarativeSupportImpl.getAutoChainCandidates(DeclarativeSupportImpl.java:408)
	at com.pega.pegarules.data.internal.clipboard.ClipboardPropertyImpl.autoChain(ClipboardPropertyImpl.java:1073)
	at com.pega.pegarules.data.internal.clipboard.ClipboardPropertyImpl.autoChain(ClipboardPropertyImpl.java:951)
	at com.pega.pegarules.data.internal.clipboard.ClipboardPropertyBase.autoChain(ClipboardPropertyBase.java:3423)
	at com.pega.pegarules.data.internal.clipboard.ClipboardPropertyImpl.getValue(ClipboardPropertyImpl.java:4684)
	at com.pega.pegarules.data.internal.clipboard.ClipboardPropertyImpl.getValue(ClipboardPropertyImpl.java:4675)
	at com.pega.pegarules.data.internal.clipboard.ClipboardPropertyBase.getValue(ClipboardPropertyBase.java:3362)
	at com.pega.pegarules.data.internal.clipboard.ClipboardPropertyImpl.toString(ClipboardPropertyImpl.java:5338)
	at com.pega.pegarules.exec.internal.declare.infengine.pages.DeclarativePageDefinitionImpl.initialize(DeclarativePageDefinitionImpl.java:447)
	at com.pega.pegarules.exec.internal.declare.infengine.pages.DeclarativePageDefinitionFactory.acquire(DeclarativePageDefinitionFactory.java:71)
	at com.pega.pegarules.exec.internal.declare.infengine.pages.DeclarativePageDefinitionCacheImpl.initialize(DeclarativePageDefinitionCacheImpl.java:307)
	at com.pega.pegarules.session.internal.async.startup.DeclarativePageDefinitionsCacheTask.runTask(DeclarativePageDefinitionsCacheTask.java:21)
	at com.pega.pegarules.session.internal.async.startup.StartupTaskUtil.runTask(StartupTaskUtil.java:68)
	at com.pega.pegarules.session.internal.mgmt.PRNodeImpl.initializeSystem(PRNodeImpl.java:894)
	at com.pega.pegarules.session.internal.mgmt.PREnvironment.getThreadAndInitialize(PREnvironment.java:379)
	at com.pega.pegarules.session.internal.PRSessionProviderImpl.getThreadAndInitialize(PRSessionProviderImpl.java:2084)
	at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineStartup.initEngine(EngineStartup.java:671)
	at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl._initEngine_privact(EngineImpl.java:172)
	at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.doStartup(EngineImpl.java:145)
	at com.pega.pegarules.session.internal.engineinterface.etier.ejb.EngineBean.doStartup(EngineBean.java:121)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:507)
	at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:370)
	at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:411)
	at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowable(AppServerBridgeToPega.java:224)
	at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingException(AppServerBridgeToPega.java:246)
	at com.pega.pegarules.internal.etier.ejb.EngineBeanBoot.doStartup(EngineBeanBoot.java:130)
	at com.pega.pegarules.internal.etier.interfaces.EJSLocalStatelessEngineBMT_f2439d86.doStartup(Unknown Source)
	at com.pega.pegarules.web.servlet.WebAppLifeCycleListener._contextInitialized_privact(WebAppLifeCycleListener.java:216)
	at com.pega.pegarules.web.servlet.AbstractLifeCycleListener._contextInitialized_privact(AbstractLifeCycleListener.java:159)
	at com.pega.pegarules.web.servlet.AbstractLifeCycleListener.contextInitialized(AbstractLifeCycleListener.java:73)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:507)
	at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:370)
	at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:411)
	at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowable(AppServerBridgeToPega.java:224)
	at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethod(AppServerBridgeToPega.java:273)
	at com.pega.pegarules.internal.web.servlet.WebAppLifeCycleListenerBoot.contextInitialized(WebAppLifeCycleListenerBoot.java:83)
	at com.ibm.ws.webcontainer.webapp.WebApp.notifyServletContextCreated(WebApp.java:1709)
	at com.ibm.ws.webcontainer.webapp.WebAppImpl.initialize(WebAppImpl.java:414)
	at com.ibm.ws.webcontainer.webapp.WebGroupImpl.addWebApplication(WebGroupImpl.java:88)
	at com.ibm.ws.webcontainer.VirtualHostImpl.addWebApplication(VirtualHostImpl.java:170)
	at com.ibm.ws.webcontainer.WSWebContainer.addWebApp(WSWebContainer.java:904)
	at com.ibm.ws.webcontainer.WSWebContainer.addWebApplication(WSWebContainer.java:789)
	at com.ibm.ws.webcontainer.component.WebContainerImpl.install(WebContainerImpl.java:427)
	at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:719)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1178)
	at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1383)
	at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:639)
	at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:972)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:777)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl$5.run(ApplicationMgrImpl.java:2197)
	at com.ibm.ws.security.auth.ContextManagerImpl.runAs(ContextManagerImpl.java:5486)
	at com.ibm.ws.security.auth.ContextManagerImpl.runAsSystem(ContextManagerImpl.java:5612)
	at com.ibm.ws.security.core.SecurityContext.runAsSystem(SecurityContext.java:255)
	at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:2202)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:435)
	at com.ibm.ws.runtime.component.CompositionUnitImpl.start(CompositionUnitImpl.java:123)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:378)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.access$500(CompositionUnitMgrImpl.java:126)
	at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$CUInitializer.run(CompositionUnitMgrImpl.java:984)
	at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:502)
	at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1881)

***Edited by Moderator Marissa to update categories***

Low-Code App Development Upgrades
Moderation Team has archived post
Share this page LinkedIn