Recently a smart engineering manager from a major financial institution asked the question: ‘We are already using APM tool (AppDynamics), why do we need to use GC log analysis tool (GCeasy)?’. Fair question that deserves detailed answer. There are angles to this question:

(a). Purpose

(b). Metrics

Let’s review them in this article.

(a). Purpose

APM tools are fantastic tools to monitor your applications performance, health in production environment. They give entire big picture on application’s availability, performance in production environment. However, they aren’t meant to optimize & tune your application’s Garbage Collection and memory settings.

APM tools are typically meant to be used in production environment. On the other hand, GC log analysis tools can be used in Dev, Test and production environment.

(b). Metrics

GC Log analysis tools by provides rich set of Garbage collection, memory related micrometrics (which aren’t provided by APM tools). Here is a overview of such metrics that are provided only by GCeasy.

1. Garbage Collection KPIs

There are 3 Key Performance Indicators when it comes to Garbage Collection analysis:

  1. Throughput
  2. Latency (i.e. GC Pause time)
  3. Footprint

APM tools reports only Footprint, but not other two critical KPI. More details on each Garbage Collection KPI is described in this article.

img4
Fig: GC KPIs reported by GCeasy

2. Phase metrics

Within one single GC event there are multiple phases. Example G1 GC algorithm has following phases: initial mark, root region scanning, concurrent marking, remark, cleanup. CMS GC algorithm has following phases: initial mark, concurrent mark, concurrent preclean, concurrent abortable preclean, final remark, concurrent sweep, concurrent reset.

APM tools doesn’t give metrics on these individual phases, whereas GCeasy provides detailed metrics on each phase. These metrics are crucial when you are tuning, optimizing the GC/memory settings of your application.

img1
Fig: Individual GC Phases metrics generated by GCeasy

3. GC Causes

Garbage collection by itself doesn’t add value in processing customer transactions. It is a necessary evil, to clean-up unreferenced objects, make room to handle new incoming requests. However, Garbage collection consumes considerable amount of CPU and causes application to pause. Thus, one should try to reduce number of time GC events run and amount of time it takes to run.

GCeasy reports the reasons that are triggering Garbage collection events. If these reasons can be addressed GC event counts and pause times, caused by them can be minimized. These ‘GC Causes’ aren’t reported by APM tools.

img3
Fig: Reasons triggering GC events

4. ML algorithms to detect problems

GCeasy employs machine learning algorithms to detect various memory/GC related problems. Detected problems are reported in report. GCeasy not only detects problems, but it also recommends solutions to fix the detected problems. Below are few such problems and solutions reported by GCeasy tool:

img2
Fig: GCeasy detects GC/memory related problems automagically

5. ML algorithms – memory tuning recommendations

GCeasy’s machine learning algorithms provides tips for optimizing and tuning GC and memory related settings in the JVM.

img5

Fig: GC Tuning tips recommended by GCeasy

6. REST API – for CI/CD Integration

GCeasy provides JSON based REST API to analyze Garbage collection logs. Instead of manually uploading GC logs and analyzing, you can use this API to analyze hundreds/thousands of GC logs in automated manner. API provides detailed micrometrics on various aspects of application’s memory and GC utilization. These APIs are used by enterprises in their CI/CD pipeline. It facilitates to catch various GC and memory related performance problems right during code commit time.

7. Amount of memory wasted

Today enterprises wastes anywhere between 30 – 80% of memory because of inefficient programming practices such as: Duplication of String, Inefficient Data structure implementation, Suboptimal data type definitions, overallocation and underutilization of generation sizes…. GCeasy’s sister product HeapHero is the first tool in the industry to report the amount of memory wasted due to inefficient programming.

img6
Fig: HeapHero reporting amount of memory wasted due to inefficient programming practices.

Conclusion

Garbage Collection log analysis tools such as GCeasy, Garbage Cat, HP Jmeter aren’t replacement to APM tools such as AppDynamics, NewRelic, Dynatrace, Wily Introscope. Garbage collection log analysis tool compliments APM tools by providing additional metrics and purposes.