Question
How to process records from DB table by running agents on multi node
System : Pega 7.2.2 with multiple nodes
Requirement : We will receive data feed (from external system) in a DB table which needs to processed by agents (running on multinode) which in turn creates/updates work objects
Problem Statement : The record processed by Agent on node 1 should not be processed by Agent on Node 2
***Edited by Moderator Marissa to update categories***
Hi Asif, Thanks for the reply but the problem is during the fraction of seconds while agent on node 1 is updating the column (the flag to identify that it has been picked up by agent 1) agent on node 2 can pick up the very same record (until the flag is updated and committed to DB by agent on node 1). There is very high chance for this to happen. Hence want to avoid this problem and also want to know if the solution is available OOTB ( similar to auto queue management for task driven agent processsing).
We can run the agent on a single node to avoid this problem but as per our requirement we cannot comprise DR situation which mandates the agent to up in multiple nodes
If you are accessing the external db table via the pega class . In obj-open of the record, enable lock and Release on commit, so that a lock is acquired on the db record and its released only after commit is done. With this you can avoid 2 agents picking the same record.
Hi Vanaa,
Thanks for your response. Your solution will avoid the agent on node 2 from picking up the record processed by agent on node1. The only problem i see with this approach is there will be unnecessary contention as the records browsed by agent on node 2 will be exactly same as that of agent on node1 hence there will be lots of obj-open failures (which is expected). Just wondering if it would impact the performance of the batch considering the fact that there will be thousands of records which needs to be processed by these batch jobs. Any thoughts ?
The no of obj-open failures should be minimal, this would occur only when other agent has already started processing of the same record.
However in your case you have to open the record always to know whether it is processed or not by checking the flag value. So going with approach should be fine and it shouldn't be a huge performance impact. Thanks.
I am not sure about exact implementation, though could suggest a high level idea.
You could have a column in the table having (Updated Information such as timestamp or any flag) which gets updated for a record. In agent, you might be using an activity (pre-assumed) were you can configure a check about the above mentioned flag or updated information to conditionally process it.