Posted: 22 May 2019 15:14 EDT Last activity: 25 Feb 2020 9:26 EST
Rule Search Index Not Being Updated / pyFTSIncrementalIndexer Exception Pega 8.1.3
The Rule Search Index is not being updated.
Upon checking the PegaRULES log file, the pyFTSIncrementalIndexer is throwing out exception.
The following is the first couple lines from the PegaRULE log file:
2019-05-22 14:57:14,048 [CHEDULER_THREAD_POOL] [ STANDARD] [ ] [ ] (ueueProcessorFailedRunsManager) ERROR - Unable to re-start failed queue processor pyFTSIncrementalIndexer. Caught exception: Cannot resubmit the run [pyFTSIncrementalIndexer] because some of its partitions are not in end state
java.lang.IllegalStateException: Cannot resubmit the run [pyFTSIncrementalIndexer] because some of its partitions are not in end state
PRPC Version: 8.1.3
***Edited by Moderator: Lochan to update SR details***
Call the API function: Data-Decision-DDF-RunOptions pxStartRunByID
This API function resets the state of the partitions of a (failed, in this case) queue processor and starts the run. You will pass it the ID of the queue processor that is stuck in a failure state - in this case, pyFTSIncrementalIndexer.
Starting in 8.1.6 and 8.2.3, this resetting the state of the partitions will be used when the engine performs automatic restarting of failed queue processors so that this scenario where the partition is in an unexpected state is handled by the automatic restart.
You can check that the queue processor is running properly after doing this by going to Admin Studio > Resources > Queue Processors
Hi - what version are you using? The above recommendation relates to 8.1.x prior to 8.1.6, and 8.2.x prior to 8.2.3. Perhaps make a new thread. 7.x indexing uses different mechanisms entirely so your error would not be identical, nor would the API functions be available.
Also, if you're attempting to find this rule in your environment using the search bar, you need to check the enable diagnostic checkbox in your operator profile (click your initials/profile picture in the lower left of dev studio > preferences)
As per tSR-D17907 notes,The root cause of this issue is When a Queue Processor's Kafka partition ends up in an unknown state, the automatic restart does not clear the partition which results in the exception, and the queue processor fails to be restarted.
It got resolved by using the API function pxStartRunByID, and pass it the failed queue processor's name. This API function will reset the state of the partition when it starts the run. You can verify the Queue Processor is running normally from the Admin Studio > Resources > Queue Processor page.
In 8.1.6, this is addressed by having the automatic restart functionality also reset the state of the kafka partition when a failed run is restarted.