Question

5
Replies
760
Views
 Srinivasu Anjuri (SrinivasuA)
TCS

TCS
IN
SrinivasuA Member since 2015 16 posts
TCS
Posted: July 5, 2017
Last activity: July 27, 2017
Posted: 5 Jul 2017 7:36 EDT
Last activity: 27 Jul 2017 9:35 EDT
Closed

AES - CPU Alert - Above 100%

Hi

Frequently receiving CPU alerts from AES for application. And observed the usage is more than 100% some times. But the server cpu utilization is normal (never crossed 40%).

Alert "The current process CPU usage 110 has exceeded the critical threshold limit 70 percent"

Is it a false alert ? or will it calculate the usage based on the CPU cores?

Details: AES 7.13 , PRPC 7.1.9, Linux Server, CPU Cores - 8.

Thanks

Srinivasu

System Administration
Moderation Team has archived post, This thread is closed to future replies. Content and links will no longer be updated. If you have the same/similar Question, please write a new Question.
Posted: 4 years ago
Posted: 5 Jul 2017 18:10 EDT
Wai-Kei Kwok (WaikeiKwok_GCS)
PEGA
Principal Software Solutions Engineer
Pegasystems Inc.
US

You can run 'top -H' in linux to find the threads consuming high CPU.  If it is a java thread (or multiple) from the JVM, you can then take a thread dump from the JVM and see if you can map the native thread id back to the thread id in the result of top -H.  You will be able to identify the call stack of the thread(s) consuming CPU.
Posted: 4 years ago
Posted: 10 Jul 2017 7:38 EDT
Andy Werden (WERDA)
PEGA
Sr. Director, Product Management, Pega Application Management Suite
Pegasystems Inc.
US

Every two minutes the 'management daemon' running on a node takes a snapshot of requestor count, agent count, heap usage and CPU usage reported by the application server and sends that to AES. It is quite possible that CPU usage is below 40% on average but might have spiked to 100% at the time when the daemon had taken the snapshot.


What are you using to measure and record server CPU? What is the data that drives the assertion "never crossed 40%"? How often / how granular is the CPU usage data that you are using for comparison? How often are you sampling CPU load for your reporting?


What is the nature of your system architecture? Is it physical or virtual? How many virtual CPU's are available to the operating system hosting your application server / Pega application?


Typical things that may contribute to 'spikiness' that are outside the application are garbage collection and JIT compilation. Suggest you review your alerts and verbose GC files to see if there is anything interesting happening around times that your system is reporting 100% CPU usage?
Posted: 4 years ago
Posted: 21 Jul 2017 8:31 EDT
Andy Werden (WERDA)
PEGA
Sr. Director, Product Management, Pega Application Management Suite
Pegasystems Inc.
US

I've contacted a SME to look into how the management daemon measures CPU.


What Java version are you using? What operating system? What is your hardware topology (physical and virtual)? Are you using docker or any other secondary virtualization?


I do know that in Linux 'top' command, CPU is measured relative to a single CPU and a multi-threaded process such as java can exceed 100%. We will need to see what Java API the management daemon uses, then cross-check your specific JVM implementation to see if it returns CPU utilization in terms of a single processor or the entire server.


if indeed we do find that there are instances where Java can legitimately be reporting CPU usage well in excess of 100% we'll need to be sure that AES can handle the bigger threshold.