‘How will you know whether your application’s memory is over allocated or not?’ Over allocation of memory will not result in any performance problems, so most of us don’t know whether memory is over allocated or not. However, it also brings another question: ‘Memory is cheap, so why do I need to worry about memory over-allocation?’. Fair question. Yes, it’s true – ‘Memory is cheap’. In the 1970s 1 byte of memory cost USD $1, whereas now it’s a fraction of that cost. However, consider your cloud hosting bill. We are billed based on the number of EC2 instances (or the services) in which our applications are running. If you take an EC2 instance, it consists of four computing resources:

1. CPU

2. Memory

3. Network

4. Storage

Among these resources, most of our applications tend to saturate memory first, before hitting the limits of the other three resources. When memory becomes a bottleneck, you end up provisioning additional computing capacity even though the other three resources (CPU, network, and storage) are partially or under-utilized. This causes our cloud hosting computing costs to spike up. Thus, if we want to reduce computing costs, we need to reduce our memory footprint. One easy and effective way to do it is to reduce the memory size. In this post, we will discuss how to determine whether your application’s memory is over allocated or not. If it’s over allocated, how much can we reduce further?

Video

In one of our recent webinars, we explored how to evaluate and optimize your application’s memory allocation for peak performance. Attendees discovered practical techniques to avoid over-provisioning and under-provisioning, ensuring seamless application functionality while minimizing unnecessary costs. Access the presentation deck here and gain valuable insights and strategies shared by our experts in the video below:

Under allocated memory

To answer the question whether your application’s memory is over allocated or not, we first need to answer the question whether your app’s memory is under allocated or not. 

Symptoms of Under Allocated Memory

When your application’s memory is under-allocated, it struggles to keep up with workload demands, leading to noticeable performance issues. Below are the key symptoms that indicate application is running with insufficient memory:

1. Spike in CPU Consumption:  When memory is under allocated, garbage collection will be triggered more frequently to free up the space. Garbage Collection is a highly CPU intensive operation, because it must scan tons of objects that are present in memory and their complete hierarchy to identify whether they are eligible for garbage collection or not. Thus, the application’s CPU consumption will spike up.

2. Degradation in Overall Response Time: Also, another side-effect of automatic garbage collection is: It will stop your application from processing customer transactions. Since applications stopped, it will result in degraded response time and cause unpleasant customer experience. 

3. Intermittent Unresponsiveness: If garbage collection runs consecutively to free up the memory, it will cause prolonged pauses in the application, which will cause intermittent unresponsiveness. It can result in transaction timing out and application health check failures.

How to Identify Under Allocated Memory?

If your application is experiencing above mentioned symptoms, then you need to study closely your application’s Garbage Collection (GC) performance.  GC Logs is the lightweight, non-intrusive and accurate way to study the GC performance. To study the GC log performance, you need to first enable GC log by passing the following JVM arguments to your application:

Java 8 & below versions: If your application is running on Java 8 & below versions, then pass below arguments:

-XX:+PrintGCDetails -Xloggc:<gc-log-file-path>

Example:

-XX:+PrintGCDetails -Xloggc:/opt/tmp/myapp-gc.log

Java 9 & above versions: If your application is running on Java 9 & above versions, then pass below arguments:

-Xlog:gc*:file=<gc-log-file-path>

Example:

-Xlog:gc*:file=/opt/tmp/myapp-gc.log

We highly recommend collecting GC logs from your production servers, because your GC behavior is influenced by the incoming traffic. Since it’s hard to simulate production traffic in a test environment, we recommend capturing GC logs from your production servers. You may refer to this post, to learn more best practices on GC log capturing.

GC log adds very negligible overhead to your JVM. If you want to learn more about the overhead added by GC log, read this ‘zero overhead added by enabling GC log’ white paper.

Once you have collected GC log, you can upload the GC log to the tools like GCeasy, IBM GC Visualizer, HP JMeter, Garbage Cat. These tools provide detailed reports on GC performance.

Key GC Indicators of Memory Under-Allocation

When an application suffers from memory under allocation, you will observe either one (or both) of the characteristics: 

1. Consecutive Full GCs: If the allocated memory size is smaller than the actual demand, memory will fill up quickly, causing the garbage collector to run more frequently. During peak load, the garbage collector may run back-to-back, causing the CPU consumption to spike up and response time to degrade.

2. Poor GC Throughput: GC Throughput is one of the primary KPI when analyzing GC performance. It indicates the percentage of time the JVM spends processing customer transactions versus garbage collection. For instance, a GC throughput of 98% means the application spends 98% of its time in processing customer transactions and the remaining 2% on GC activities. Thus high GC throughput is desirable, however in under allocated memory scenarios, GC throughput will drop below 95%.

Real-World Example of Under Allocated Memory

Here is the GCeasy tool’s real GC log analysis report of an application whose heap size was under allocated. The report highlights both consecutive Full GCs and poor GC throughput issues as shown in the below screenshots:

Fig: Consecutive Full GCs error reported by GCeasy

Fig: Poor GC Throughput warning reported by GCeasy

Over Allocated Memory

Now let’s discuss the central focus of this post i.e. what are the symptoms of over allocated memory, how to detect it and how much can you reduce it further. 

Symptoms of Over Allocated Memory

Unlike Under Allocated Memory, Over Allocated Memory doesn’t exhibit any symptom. It’s like a silent killer 😊. Your application will perform properly without exhibiting any hiccups, however you will end up paying unnecessary bills to your cloud hosting provider.

How GCeasy Detects Over-Allocated Memory

Just like how we studied the GC performance to identify the memory under allocation, we should study the GC performance to identify the memory over allocation as well. To identify if your Java application’s memory is over allocated, you can use the GCeasy, an advanced garbage collection log analysis tool, which provides actionable insights into your application’s memory allocation. Here’s how it identifies over-allocation:

  1. Advanced Algorithms: GCeasy uses sophisticated algorithms to thoroughly analyze your GC logs, ensuring that none of the under-allocation symptoms, such as consecutive Full GCs or poor GC throughput, are present. This helps rule out under-allocation as a contributing factor.
  2. Mathematical Modeling: The tool applies mathematical models to evaluate your application’s memory usage patterns. These models simulate the effect of reducing heap size and predict whether such changes would adversely impact application performance.
  3. Precise Recommendations: If GCeasy determines that your application’s memory is over-allocated, it provides specific recommendations. For example, the tool will calculate and suggest a percentage by which the heap size can be reduced without affecting performance.

Real-World Example of Over Allocated Memory

Here is a real GC log analysis report of an application whose heap size was over allocated. This report illustrates how GCeasy identifies over allocation and provides actionable insights. GCeasy tool not only identifies over allocation but also suggests by what percentage the heap size can be reduced. In this example, you can notice that the tool is raising a warning that memory is over-allocated by 47%. Thus this application’s memory size can be reduced from its original allocated size of 4gb to 2gb.

Fig: Over Allocated Memory warning reported by GCeasy

Conclusion

In this post I attempted to explain the performance degradation experienced by the applications whose memory is under allocated, and the unnecessary computing cost incurred by the applications whose memory is over allocated. I hope the solutions discussed in this post will help you to allocate optimal memory for your application, which will ultimately help to reduce your cloud hosting bill.