Sizing your Java heap correctly is the one configuration task that has more impact on performance and costs than any other.
Too small, and the garbage collector (GC) works overtime, hogging vital CPU resources. There’s also the risk of system crashes caused by OutOfMemoryErrors. Too big, and either cloud costs shoot up, or the performance of the device or container degrades due to lack of memory resources for other tasks.
Can we hand over the entire job of JVM heap sizing to AI? Unfortunately, at the moment, the simple answer is: not with any degree of accuracy. On the other hand, an exciting new development in this field is Deterministic AI.
In this article, we’ll explore how GCeasy’s Deterministic AI integrates with your LLM of choice to produce accurate guidelines for this important task.
What Is Deterministic AI and How Can it Streamline JVM Heap Sizing?
What’s the difference between deterministic problem solving and AI?
Deterministic problem solving applies a fixed set of rules and algorithms to a set of data, and produces a reproducible result. AI works differently. It uses probability to come up with the most likely solution to a given problem. AI won’t necessarily always come up with identical answers, even when it’s fed the same set of information.
AI is becoming more impressive by the day. When it comes to fact-dependent and high-accuracy tasks like troubleshooting production problems, it still has some severe limitations.
- AI sometimes “hallucinates”, or, in other words, it tends to represent probability as fact. You really don’t need this when you’re troubleshooting an urgent production problem in the small hours.
- It’s not always easy to know the right questions to ask.
- Quality AI is often expensive when we need to upload large datasets such as GC logs.
- For performance tuning, we need precise accuracy rather than probability.
GCeasy’s deterministic AI, on the other hand, calculates precise metrics first before presenting them to an LLM for interactive exploration. It processes large amounts of raw data into a small subset of relevant facts in JSON format. This is illustrated in the diagram below.
Fig: Deterministic AI Process
GCeasy reduces the large logs to concise metrics before passing the data to the LLM, significantly reducing costs. Since the LLM is using a precise set of facts for its tasks, results become reproducible.
What’s also invaluable is that GCeasy gives us the option to view useful graphs and charts, so we can absorb trends at a glance.
Fig: Selection of GCeasy Graphics
See this article for more information on Deterministic AI.
How to Determine the Optimum JVM Heap Size
Before we talk about JVM heap sizing, let’s just clarify how the JVM organizes run-time memory.
We can visualize it like this:
Fig: JVM Runtime Memory
The heap is just one of several memory pools. It’s used to store all objects created by the application. For more information, it’s worth watching this video: JVM Explained in 10 Minutes. We should always remember this when sizing containers or calculating hardware requirements. As a quick rule-of-thumb for estimating memory needs, the heap generally occupies around 70% of the total JVM memory.
Unfortunately, we can’t just apply a formula to size the JVM heap accurately. Every application behaves differently, because there are so many factors that affect the decision, including:
- Peak heap usage;
- Allocation rate;
- Object lifetimes;
- Allocation patterns;
- Latency and throughput targets;
- Number of CPUs;
- The GC algorithm currently enabled in the configuration.
On the command line, we can either specify initial and maximum heap sizes, or we can specify the heap size as a percentage of available host memory. We’d use these command line JVM arguments to specify the heap size.
| Argument | Purpose | Syntax |
| -Xmn | Set heap size at start-up | -Xmn plus size plus single-letter suffix, e.g. –Xmn5G to set initial size to 5GB |
| -Xmx | Set maximum heap size | -Xms plus size plus single-letter suffix, e.g.-Xms100M to set initial size to 100MB |
| -XX:InitialRAMPercentage | Set heap size at start-up as a percentage | -XX:InitialRAMPercentage= plus percentage, e.g. -XX:InitialRAMPercentage=20 |
| -XX:MaxRAMPercentage | Set maximum heap size as a percentage | -XX:MaxRAMPercentage= plus percentage, e.g. -XX:MaxRAMPercentage=20 |
Using percentages rather than fixed values is recommended when working with containers.
Sizing the heap can’t reliably be done by formula. We need to test the memory settings under a fixed load, recursively trying different settings and monitoring the result until we find the sweet spot, where increasing the heap size no longer improves performance. Obviously, we need a sensible starting point for the first test. As a rule of thumb, analyze the GC log for a period that includes both peak and off-peak loads to get the maximum heap usage. We then need to multiply this by a safety margin to get a rough guide that we can use to begin testing. Depending on the type of application, the table below shows suggested safety margins.
| Type of Application | Safety Margin |
| Stable server | 1.3–1.5 |
| Typical web application | 1.5–2.0 |
| Sporadic Creation Cycles | 2.0–3.0 |
| Batch processing | 2.5–4.0 |
GCeasy provides the peak heap usage:
Fig: Memory Usage Chart
See this article for a full discussion of how to find the optimum heap size.
This process can be time consuming, as it involves comparing several logs to find the point where the key performance indicators no longer improve when we increase the heap size. Can AI help us? We’ll see in the next section.
How Deterministic AI Eliminates Guesswork from JVM Heap Sizing
Let’s use a worked example to see how Deterministic AI takes the pain out of heap sizing.
For this exercise, we took a small Jenkins test server, and ran it for several hours with each of six different heap settings of approximately:
- 80MB
- 150MB
- 250MB
- 512MB
- 1GB
- 1.5GB
We ran each iteration with a similar workload, which simulated peak and off-peak usage.
Let’s see what happened when we loaded all six logs into GCeasy’s Deterministic AI. For this exercise. GCeasy used Claude as the LLM, but you could also configure it to use OpenAI or Gemini.
GCeasy is running on our test server, and we access it via a web interface.
On the web page, we have a choice of using Classic GCeasy analysis, or Deterministic AI, as shown below. Selecting this lets us upload a log and ask a question.
Fig: GCeasy Deterministic AI
We explained the task we needed the AI to carry out, then uploaded the first of the logs by pressing the + sign.
The response is shown below:
Fig: AI Response to the Task Request
The response continued by displaying some key metrics from the first log. A section of this is shown below, and gives some guidelines on the metrics we should see if the heap is optimally sized.:
Fig: Key Metrics and Guidelines
We then uploaded each of the remaining five logs by loading the log using the + sign, and stating what we wanted done with it.
Fig: Uploading Additional Logs
The system responded by giving some metrics from each new log.
When we loaded the final log, we asked the LLM to compare the six logs and come up with heap sizing recommendations.
Once more, it gives some metrics from the current log, then explains its sizing formula:
Fig: Sizing Formula Explained
Finally, the LLM gives recommendations on how to achieve best performance for this application without using excessive memory:
Fig: Configuration Recommendations
We could have achieved the same thing by laboriously comparing the KPI of each log, but this is much faster and easier.
We know the LLM is not hallucinating because it’s working on a fixed set of calculated facts, and we can safely implement this change.
Conclusion
Accurate JVM heap sizing is essential so we can keep costs low and performance high. Good GC log analysis tools transform this task from guesswork to a science, but it still involves time and effort.
Deterministic AI combines the accuracy of traditional log analyzers with the power of LLMs, taking the pain away from calculating the optimum Java heap size.