Question
Capgemini
US
Last activity: 9 Jun 2017 11:35 EDT
Rule lookup failed : com.pega.pegarules.pub.PRRuntimeException: Rule-Connect-Rest lookup failed
The system is working fine for several months and all of a sudden it is not able to connect to external rest service and throwing this exception and not sending any request to service host:
2017-05-10 11:26:54,999 [BTHREAD1, eon_id=932] [TABTHREAD1] [ ] [ ValStudio:01.01.01] ( internal.mgmt.Executable) ERROR andreit - Exception
com.pega.pegarules.pub.PRRuntimeException: Rule-Connect-Rest lookup failed
at com.pegarules.generated.activity.ra_action_logitrakaction_28677068a22119b9e7d950d2d1e647ab.step4_circum0(ra_action_logitrakaction_28677068a22119b9e7d950d2d1e647ab.java:374)
at com.pegarules.generated.activity.ra_action_logitrakaction_28677068a22119b9e7d950d2d1e647ab.perform(ra_action_logitrakaction_28677068a22119b9e7d950d2d1e647ab.java:134)
I have validated the ruleset and then it started working fine. Since it is a production, I don't want to do validate rule anymore.
Have anybody faced the similar issue before? I would like to know the root cause of it and how can we prevent it in future?
Thanks
Faheem
Pegasystems Inc.
IN
The log file says that the Connect REST rule might have got corrupted, is this happening on production environment?
PEG
US
I have no ideas on the source of the corruption, but I want to point out that there is a less intrusive way to fix a corrupted cache entry (which this seems to be) that does not use revalidation.
I'm assuming you are on a 7 series product for what follows.
Activity cache entries are maintained in the VTable cache. It is possible to clear just the cache entries (there can be more than one) for a single activity by means of the SMA tool.
Use "Advanced >> Virtual Rule Table Cache Management" and enter the Virtual Rule Key for the activity in trouble. You will need to open the XML of the rule to get the precise form of the key required by this tool. In earlier releases, you had to construct the string "pxObjClass" + ":" + "pxInsId". Later releases are a bit more human friendly.
Once you enter the key, click on the button to open the cache entry. As I said, there may be more than one entry for a single activity. Delete all of the entries you see for this activity via the "reassemble candidates" button. The next time the activity is referenced by any process, the cache entry will be rebuilt. If the corruption was a transient event, it will get built correctly.
This is much less intrusive than a revalidation, and can also be done even if the target rule is in a locked ruleset.
Not all rules are in the VTable cache. UI rules in particular still use the old-style ABA cache.
I have no ideas on the source of the corruption, but I want to point out that there is a less intrusive way to fix a corrupted cache entry (which this seems to be) that does not use revalidation.
I'm assuming you are on a 7 series product for what follows.
Activity cache entries are maintained in the VTable cache. It is possible to clear just the cache entries (there can be more than one) for a single activity by means of the SMA tool.
Use "Advanced >> Virtual Rule Table Cache Management" and enter the Virtual Rule Key for the activity in trouble. You will need to open the XML of the rule to get the precise form of the key required by this tool. In earlier releases, you had to construct the string "pxObjClass" + ":" + "pxInsId". Later releases are a bit more human friendly.
Once you enter the key, click on the button to open the cache entry. As I said, there may be more than one entry for a single activity. Delete all of the entries you see for this activity via the "reassemble candidates" button. The next time the activity is referenced by any process, the cache entry will be rebuilt. If the corruption was a transient event, it will get built correctly.
This is much less intrusive than a revalidation, and can also be done even if the target rule is in a locked ruleset.
Not all rules are in the VTable cache. UI rules in particular still use the old-style ABA cache.
As each node maintains its own cache, you will need to do this on each node on a multi-node system.
Capgemini
US
Hi It is a really good suggestion but we can get use this suggestion only after facing the issue. I am more interested in avoiding the issue in future.
How can I avoid it in future ??
PEG
US
Do you have any idea what happened to cause the problem? Was it after a rule migration? You don't show the entire stack, but was there a reference to a Rule-Utility-Function which could not be found. Sometimes rules which interface with external system can become activated after a system restart during which Libraries are being rebuilt. If an external request comes in which forces the Connect rule to try to access a Library which is not currently available, the reference will fail and the referencing rule (the Connect rule) will be corrupt.
If you are currently clearing or truncating cache tables when you restart, please do not do that. Pega may have give such advice a long time ago, but we have since discovered it is more likely to cause a problem than solve a problem. And, as far as Service rules and other external-facing pieces of the system, try to not utilize them until you are sure the system has fully initialized after a restart.