Diagnosing High CPU problems with Java Mission Control

How to diagnose high CPU problems with Java Mission Control?

Recently we worked on couple of customer escalations where we have extensively used Java Mission Control to resolve high CPU problems. That prompted me to write an article on Java Flight Recording analysis using Java Mission Control (JMC)

In this posting, I’m going to teach you an easier way to analyze the JFR files using Java Mission Control. To use JMC you need to be on JDK 7u40 or above versions. This article assumes that you have already captured a JFR from a live Java process.

1.  Incase If your JFR file is quite big you can start JMC from the command line by specifying a custom heap size.  i.e. $JAVA_HOME/bin/jmc -J-Xms8G -J-Xmx8G

2. Now open the JFR file that you captured earlier.

3. When you open the JFR file in Java Mission Control, your initial screen will look like as below. i.e. CPU and heap usage for the duration of JFR recording. To demonstrate this use case, I ran the JFR recording for 5 minutes. Ideally you run the JFR during your peak usage or during the time server is experiencing high CPU usage. My test case simulates high CPU usage every few minutes as shown below. You will notice the CPU was at 20% (8:47 AM) at the time of recording and pegged up to 100% at the end of recording(8:51 AM).

 

ScreenHunter_73 Jun. 26 08.56

 

4. On the left hand side, you will see various tabs like General, Memory, Code,  Threads, I/O, System and Events. Since the focus of this posting is to analyze high CPU, click on the 1st tab i.e. General

5. Now zoom in on the spiked area. All you have to do is to point your mouse and drag over the area as shown below. For e.g. I have zoomed in the area where the CPU usage exceeded 90%

ScreenHunter_74 Jun. 26 09.03

 

6. If you wish, you can zoom in couple of more times to get more granular data.

7. At this time, click on the Threads >> Hot Threads tab at the bottom of the window and you will be able to determine the Thread that’s causing the higher CPU. Here you also have an option to further drill down to the complete call stack.  Click on the right arrow (green) in the Thread section and you will see the Stack Trace in the bottom window.

 

ScreenHunter_75 Jun. 26 09.06

As we can tell from the above screenshot, ExecuteThreads #6 and #8 were causing the high CPU. We can also see the corresponding call stack in the lower pane. e.g. com.ateam.LongRunningEJB.threadCalc(int)

Now let’s look at my code snippet that’s causing the high CPU

public void threadCalc(int seconds) {

double start = 0;
long t0 = System.currentTimeMillis();

while (((System.currentTimeMillis() – t0) / 1000) < seconds) {
double x = Math.sin(start++);
}

}

From the my jsp/servlet code, I’m calling above method by specifying the numbers of seconds to run in a while loop. , I’m purposefully causing a high CPU usage by calling a CPU intensive operation. Ofcourse this may not be the case in a real production environment but you get the idea by analyzing the hot threads in JFR and looking at your source code to determine the high CPU usage.

That’s about it.  You are ready to troubleshoot high CPU problems using Java Mission Control 🙂 System Administrators, you can go back to your development teams and demonstrate the problematic area. Same is true for the developers who can see it themselves for the higher CPU usage areas.

 

Conclusion

As you have seen from the above analysis, Java Mission Control is a very powerful arsenal to analyze high CPU usage problems. Starting from JDK 7 u40, Java Mission Control is included as part of standard JDK install.

Please visit http://www.oracle.com/technetwork/java/javaseproducts/mission-control/java-mission-control-1998576.html  to know more about Java Mission Control.

 

 

 

 

Add Your Comment