Hello A_K,

So you have 8 nodes & 2 are slow. You can try the below approach to troubleshoot.

1. Are both the slow nodes are webnodes?

2. Hope PDC is installed for the system, You can verify health of both the nodes.

3.Check major alerts & exceptions for that node using pdc.

4.If PDC is not installed then download pega rules & pega alert logs. Check for a pattern of alerts & take necessary actions according to it fro example If it is Pega0005 then DB related issue, Pega0028 then memory related issue etc

Thanks,

Abhinav

Hi Abinav,

Thank you so much for the response. 

Apart from Checking Nodes and logs, Is there any other way we can troubleshoot in production? 

Hi,

it depends on the business hosted by your node. 

If these nodes are problematic you could shut them down, logically 6 are enough to handle your production. 

You can check at DB level whether there's any abnormal activity running from these nodes.

If you've got agents, you might have a poison message killing these nodes.

You can check CPU, RAM of these nodes, as well as FS.

You can also check which operators are connected on these nodes and trying to identify what kind of activity they're doing.

Are these nodes cross datacenter? running on same DC than the DB.

if they are down, from logs it should be easier to find what was happening before the crash. Actually it can also be a human error and a clean stop. Yes it happens...

Or some bad tracing activity

You also usually don't suddenly start on a Prod system without knowledges of the application & infra.

You either worked on the test env if new application so you also know the app itself.

Or the existing Production Support Team should have some knowledges for basic checks as per their experience.

And when you still don't find the root cause, make a report of all elements you checked and which are not the root cause :)

Regards

Anthony