REST service times out SocketTimeoutException (PRPC 6.3sp1)
REST service times out with an error as below :-
Mon Oct 12 13:25:00 IST 2015:INFO:Error getting response for [.:]; java.net.SocketTimeoutException: Read timed out
On analyzing the SMA, I am seeing the Idle count of threads as "0" and only then this issue comes up.. This is a stateful authenticated service. Below screenshot from SMA.
I had purposefully set the max idle count as "0" and Max Active count as "0" in the pooling tab. This should not affect as this is a stateful-authneticated package. I also made it as - setting max active to 1 and the issue comes up again.
Any idea on how to debug this ?
I checked the weblogic logs and saw the below error :-
####<Oct 9, 2015 12:14:03 AM PDT> <Error> <WebLogicServer> <stage2cpp087> <stage2cpp087_unify_L2_1> <[ACTIVE] ExecuteThread: '40' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1444374843831> <BEA-000337> <[STUCK] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "608" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 608245 ms
", which is more than the configured time (StuckThreadMaxTime) of "600" seconds in "server-failure-trigger". Stack trace:
Is consistent with the fact that you have sized the pool at zero - there are no requestors in the pool to borrow....
Basically I think we need to verify this statement:
[The sizing of the Requestor Pool] should not affect as this is a stateful-authneticated package
Because if this is the case (I'm not saying it is , or is not at this point) - then wouldn't that imply that 'stateful requestors' cannot be pooled at all ?
What is the behaviour you are expecting here ? That 'stateful requestors' are not pooled at all ? Or they that they are pooled, and their state gets perisisted alongside the requestor when it is returned to the pool ?
I have to be honest here: I have only every worked with 'stateless' requestors with regard to pooling - so I have never thought about what the behaviour of stateful requestors should be with regard to pooling.....
As per help, this is what stateful-authenticated with regard to pooling states ( Pooling tab is disabled for stateful / authenticated ):-
Use the Pooling tab to configure a requestor pool for the services in this service package when they are stateless and unauthenticated. This tab is disabled if, on the Context tab, the Processing Mode field is set to Stateful or the Requires authentication? option is selected.
If this is the case, why would the pool count number come into picture ? ( I see this contradiction in the saved XML into DB as well .. there again for stateful-auth , I am seeing the value being read from pooling tab as opposed to being null since this is a stateful-auth package).
If that is the case, if I set the pool idle count to 1 and if that 1 requester is borrowed for processing; if the pool does not do the house keeping job of re generating the borrowed requestor, this error MAY come up right ?
As confirmed by product specialist in code it looks like for stateless authenticated requestors we also rely on requestor pooling, the only difference is while returning a requestor, we return a bogus requestor-id(a blank string) back to the pool , and actually destroy the requestor, we return a blank id to maintain the SMA counters in sync.
In your case you’re giving MaxActive 0 ,MaxIdle 0 and MAxWait 0, this configuration itself is incorrect, as the code takes it literally to be zero, requestors will be available in the pool ever , hence the application requests for a new requestor for a pool and waits till the requestor/Containers Work Manager times out. In case if maxactive is negative it will allow unlimited requestors.
This is the desired behavior and the documentation is incorrect if it says stateless/authenticated requestors do not depend on the pool. This is the same behavior for stateful services as well.
Please note that there are 2 phases in execution of a soap service
-In the first phase we open up the service package - for this we use requestor pooling
-In the second phase we execute the service activity and we dont use requestor pooling here
So overall in stateful case we use requestor pooling