GC Log Analysis Using Deterministic AI vs. LLM: A Real-World Comparison (Part 1)

If you want to understand how a Java application actually behaves under memory pressure, there is no better way than a thorough analysis of GC logs. For many years, one approach has dominated the industry – deterministic AI, which transforms raw logs into structured metrics using pre-defined rules. However, large-scale language models (LLMs) have introduced a completely new analysis paradigm: they do not rely on fixed rules, but directly read raw logs and describe the situation in natural language.

This experiment is very interesting- we analyze the same Java GC log in two different ways. One is GCeasy‘s deterministic AI and the other is a general-purpose LLM. This is not a contest of who is better, but rather an attempt to compare the “thought processes” the two go through in identifying GC behavior, presenting rationales, interpreting potential problems, and suggesting countermeasures.

For fairness, the GC logs used in both analyzes were exactly the same, and all comparisons were based on the same data source.

How We Compared Deterministic AI and LLMs 

To ensure the accuracy of the comparison, we used something like a “double-blind study”; exactly the same GC log samples were used in both analyses.

The original analysis was left to a general-purpose LLM. It instructed them to download raw GC logs directly and “identify key GC-related issues and perform a comprehensive assessment that includes problem symptoms, chain of evidence, root cause inference, and a list of suggested fixes.”

We then fed a copy of this log into GCeasy‘s deterministic AI system, which generated the familiar GC Intelligence report.The objective is not to replace one approach with the other. 

The important thing is that I did not draw my conclusion beforehand. Instead, we looked at how, when faced with the same data, the two systems each captured GC behavior characteristics, pause time patterns, heap usage, object allocation/promotion activity, potential inherent problems, etc., and ultimately produced a list of optimization suggestions.

Our goal was not to judge which is better or worse, but to highlight interesting differences in how deterministic logic and LLM inference react differently when interpreting the same JVM workload.

GC Log Analysis Test Environment and Methodology 

Both engines analyzed the identical gc.log generated under Java 21 with G1 GC and a fixed 256 MB heap.

Java GC Log Analysis Test Environment 

The configured capacity of the heap was around 256 MB, which was nearly exhausted by the maximum occupied value of roughly 230.24 MB, and the workload is close to hitting the hard limit.

The workloads were created in two phases. In the first phase, an allocation of short-lived byte arrays of 20 KB was initiated, while some of the created arrays were retained as long-lived references. The second phase was characterized by the allocation of 50 KB arrays with retaining about 2% of the allocated arrays as long-lived references to increase the pressure on the young generation, thereby causing frequent young garbage collection pauses and the live set and old generation size increase after each collection.

 The test was compiled with javac GCLogComparison.java and executed with the following JVM configuration:

java -Xms256m -Xmx256m -XX:+UseG1GC -Xlog:gc*:file=gc.log:time,uptime,level,tags GCLogComparison

The first generated gc.log was loaded into GCeasy‘s deterministic AI interface and analyzed using comprehensive instructions. We then independently supplied the same raw log to the LLM model, being careful not to mix it with the GCeasy analysis results. This cross-validation approach allows both tools to operate independently while working on the same source data.

Fig:  The same gc.log is loaded into GCeasy’s deterministic AI and analyzed

This screenshot displays the same gc.log file being sent to GCeasy for deterministic analysis. The report generated from this file is used, in this article, as a comparison basis against the general-purpose LLM results.

GCeasy’s deterministic AI does more than just generate standard GC reports. By using the structured data obtained from the log analysis as the context of the analysis, we are able to produce output that combines accurate GC measurements with natural language explanations of JVM operations. This feature has important consequences for later comparisons: the deterministic module provides the numerical basis, while the AI layer interprets the meaning behind these measurements.

What the Java GC Log Reveals 

Without relying on the interpretation of any analysis tool, let’s see what the raw logs themselves tell us:

• GC Collector: G1 GC
• GC Events: 89
• Full GC: none observed
• Humongous regions: none observed
• Heap: 256 MB

A representative line from the raw log looks like this:

Pause Young (Normal) (G1 Evacuation Pause)
230M->107M(256M)

The workload generated frequent Young GC activity, and the amount of memory remaining after each collection increased steadily toward the end of the test.

This comparison focuses on Java heap behavior. Metaspace, Code Cache, Direct Buffer memory, thread stacks, and other native-memory regions require additional JVM diagnostics.

What Deterministic AI Found in the GC Log 

GCeasy‘s Deterministic AI analyzed the uploaded GC log and generated a structured GC Intelligence Report containing measured GC statistics, heap trends, GC behavior, allocation and promotion metrics, GC causes, and tuning recommendations.

The first important result was that the JVM did not appear to be suffering from a serious GC pause-time problem.

GCeasy reported a GC throughput of 99.437%, indicating that only a very small portion of the observed runtime was being consumed by garbage collection.

Fig: GCeasy Deterministic AI – Key Performance Indicators and GC Duration Time Range

The dashboard has 99,437% GC throughput, 2.08 ms average pause time and 10 ms maximum pause time. Thus, we can assume that pauses are frequent but short.

The report shows:

• Throughput: 99.437%
• Avg Pause GC Time: 2.08 ms
• Max Pause GC Time: 10.0 ms

And the pause-time distribution:

• 0–1 ms → 4 GCs
• 1–2 ms → 44 GCs
• 2–3 ms → 28 GCs
• 3–4 ms → 8 GCs
• 9–10 ms → 2 GCs

1. GC Pause Time Analysis 

Despite the relatively frequent GC activity, GCeasy showed that the individual pauses remained very short.

The report calculated an average GC pause time of approximately 2.08 ms, while the maximum reported pause was 10 ms. Most pauses fell between 1 and 3 ms.

This is important because frequent garbage collection does not automatically mean that GC is causing severe application latency. In this workload, the collections were frequent, but individually inexpensive.

GCeasy also reported approximately 179 ms of total GC pause time, shown in the pause-time distribution above.

2. Post-GC Heap Occupancy Is Increasing 

Looking at the graph of heap memory usage created by GCeasy (after GC collection), a clear stepwise increase can be observed.

Just like climbing a flight of stairs, you can see that almost zero objects survive the first GC, but as the process progresses, more and more objects remain in memory.

Fig: Heap usage after GC – residual memory grows as work progresses

The graph shows a stepwise increase in memory remaining after each GC cycle. This upward trend of the post-GC heap floor suggests that the amount of objects surviving GC was growing throughout the application’s lifetime.

This diagram confirms what the previous large-scale language model showed: In this application, more and more objects persist over time.

What is interesting is the difference in the wording of the two tools. Language pattern is an abstract concept that we explained as “object dwell time” and “old generation utilization growth”, but GCeasy conveys the key trends at a glance with an intuitive heap graph. [Memory load is constantly increasing]

3. Pre-GC Heap Usage and Allocation Pressure 

Memory usage before the GC starts is like a slowly rising thermometer, initially hovering around 150MB, but as the amount of work increases, it crosses the 220MB mark.

Fig: Heap usage before the GC grows at the end of the task

The pre-GC heap utilization grows from about 150 MB to over 220 MB, which suggests that the application was under continuous memory allocation pressure towards the end of the workload.

The progression of before- and after-GC graphs shows an ongoing allocation pressure and a steadily increasing post-GC heap floor – suggesting increasing live-set retention, but not proving a memory leak.

4. Old Generation Growth and Object Promotion 

The growth curve of the old generation shows this preservation pattern even more clearly:

The Old Generation increased from the initial level, reaching up to nearly 90 MB by the end of the measurement. Young Generation collections were effective in removing short-lived objects, while the promotion of objects to Old Generation caused a constant increase in the used memory. This trend can lead to java.lang.OutOfMemoryError: Java heap space if the application continues to allocate objects in the Old Generation.

Fig: The utilization rate of the old generation increases in the second half of processing

The Old Generation size reached almost 90 MB in the test’s second half, which explains why the long-lived objects were gradually promoted and retained.

This visually supports the language model analysis that “more and more objects survive the Young Generation GC and become long-lived objects”.

However, there are significant differences. The language model warned separately that “this model alone cannot prove a memory leak.” Likewise, GCeasy’s report also states:

“No significant memory leaks detected”

This consensus is important. Although both approaches detect an increase in object lifetime, they say that it is not possible to determine a Java memory leak based solely on the increase in utilization of the old generation.

5. GC Allocation and Promotion Rate Analysis 

In the area of memory object allocation and promotion analysis, the GCeasy tool provides a more in-depth quantitative analysis.

The report clearly records the following key indicators:

• Total created bytes: 12.56 GB
• Total promoted bytes: 98 MB
• Average creation rate: 405.13 MB/sec
• Average promotion rate: 3.09 MB/sec

Fig: GCeasy Deterministic AI – Pause Time, Object Stats, and CPU Stats

The report gives a quantitative assessment of the workload, presenting such metrics as allocated data (about 12.56 GB), promoted data (about 98 MB), allocation rate (about 405.13 MB/s), and promotion rate (about 3.09 MB/s). This information allows an accurate understanding of the load, while the LLM only identified trends in the changes.

This is one of the clearest differences between the two approaches.

The LLM correctly inferred that the workload had a high allocation rate, but it did not independently quantify that rate. GCeasy converted that observation into a measurable value: approximately 405 MB/sec.

Likewise, the LLM observed increasing promotion and Old Generation retention conceptually, whereas GCeasy quantified the average promotion rate at approximately 3.09 MB/sec.

This is where deterministic parsing has a clear advantage: it can turn a qualitative observation into a reproducible metric.

6. GC Cause Analysis

GCeasy classified the observed GC activity under a single cause: G1 Evacuation Pause.

The report attributed the following to this GC cause:

• Count: 89
• Average: 2.01 ms
• Maximum: 10.0 ms
• Total: 179 ms

Fig: GCeasy Deterministic AI – GC Causes breakdown

The report classifies the observed activity under G1 Evacuation Pause and shows 89 of them on average, which take 2.01 ms each, and the maximum recorded is 10 ms.

GCeasy reports 89 G1 Evacuation Pause records in the GC Causes and G1 Collection Phase Statistics views, while the Pause Time aggregation in Figure 6 includes 86 pause events. These values represent different reporting scopes: overall classified events versus the subset included in the displayed pause-time aggregation.

The 89-versus-86 difference should therefore be treated as a reporting-scope distinction rather than evidence of a deterministic parsing error.

7. G1 GC Tuning Recommendations from GC Log Analysis 

The report identifies repeated G1 Evacuation Pauses and recommends validating allocation pressure and memory-recovery behavior before applying JVM-level tuning.

• Review JVM memory-related parameter settings
• Evaluate whether the heap size should be increased
• Correct expansion of heap size
• -XX:InitiatingHeapOccupancyPercent threshold validation
• Evaluate the number of concurrent GC threads configured by -XX:ConcGCThreads
• Review the reserve-space ratio configured by -XX:G1ReservePercent
• Consider -XX:+UseStringDeduplication where appropriate

These suggestions should only be considered as a reference for improvement, and mechanical setting changes should be avoided.

The workload already achieves 99.437% GC throughput and very short pauses. Changing multiple G1 parameters without first understanding allocation hotspots and live-object retention could introduce unnecessary tuning risk.

Of particular note is the impact of the Large Language Model: GC settings should not be changed until object allocation “hotspots” and survivability have been investigated in detail with JFR tools or allocation profilers.

Note: If post-GC occupancy continues to rise under a stable workload, capture a heap dump when memory pressure is high and inspect retained size and dominator trees with yCrash or HeapHero. Use Java Flight Recorder or an allocation profiler to identify allocation hotspots and object survivability before changing G1 parameters. 

Related guidance: G1 GC tuning; Java Flight Recorder; and OutOfMemoryError and heap-dump troubleshooting.

What the General-Purpose LLM Found in the GC Log 

For the same raw GC log data, a general-purpose large-scale language model performed the following analysis with comprehensive prompts:

Fig: Raw GC log input to general-purpose LLM

This screenshot shows the same raw gc.log being ingested into the general-purpose LLM, prompted for details about GC, pauses, frequencies, heap usage, troubleshooting, root-cause analysis, and supporting evidence.

For the same raw GC log, we asked a general-purpose LLM to analyze the JVM’s GC behavior using the following evaluation criteria: 

1. General GC behavior 
2. Collection frequency 
3. Pause time 
4. Heap usage 
5. Significant GC-related issues 
6. Possible root causes 
7. Log evidence to support your conclusion.

The core conclusion of the model: Although GC operation is generally good in terms of downtime and recovery efficiency, the frequent occurrence of young GCs and increased utilization of old areas require further investigation.

The main findings are as follows:

• 89 Young GC events occurred during the test.
• No Full GC events were observed.
• GC pause times remained short, indicating that GC was not causing significant application downtime.
• Young GCs occurred frequently throughout the workload.
• Post-GC heap occupancy increased over time, indicating that more objects were surviving each collection.
• Old Generation utilization increased as more objects remained live or were promoted.
• The analysis did not prove a memory leak, but the increasing live set warranted further investigation.
• Investigate object allocation and object lifetime to understand the increasing memory retention.
• Java Flight Recorder (JFR) or an allocation profiler can be used to identify allocation hotspots and object survivability.

Fig: Figure  Actual LLM Executive Summary

The LLM concluded that pause latency is healthy, and there were no Full GC or heap exhaustion events. However, in addition to that, the LLM noted the occurrence of frequent young GC and growing old generation as potential areas of concern.

Fig: Actual LLM Pause-time Analysis

The LLM detected 89 GC events with an average pause time of approximately 1.98 ms, a maximum pause time of 6.47 ms, and a total pause time of approximately 176 ms. Thus, the pauses are frequent, yet short.

Fig: Actual LLM analysis showing progressive post-GC live-set growth and increasing Old Generation occupancy

The LLM results show post-GC heap growth from 24 MB to 107 MB and Old Generation regions increasing from 14 to 97, indicating progressive live-set retention. 

It’s worth noting that the large-scale language model doesn’t simply dismiss the ever-increasing live set as a memory leak, but cautiously treats it as an “observation requiring further investigation”. Such restrained consideration is a unique strength of general-purpose large-scale language models.

Deterministic AI vs. LLM: GC Log Analysis Comparison 

In general, both methods allowed arriving at approximately the same conclusions regarding GC behavior, number of events, average pause, heap growth, and the absence of Full GC events. The tool provided more accurate data on throughput, allocation rate, promotion rate, and pause duration, while the general-purpose LLM could give a more contextualized analysis of allocation and retention trends. The difference between 10 ms in GCeasy and 6.47 ms in the LLM analysis also shows that deterministic parsing is better for obtaining exact figures. Finally, the 89 and 86 numbers seen in two different GCeasy windows actually represent different metrics and therefore are not contradictory.

Key Differences in GC Log Analysis Results 

I found one discrepancy in the numbers that troubled me.

The general-purpose LLM calculated the maximum Young GC pause to be approximately 6.47 ms, while GCeasy reports it to be 10.0 ms. As for cumulative pause time, the numbers are 176 ms vs. 179 ms.

While the general conclusion remains the same, even though the downtimes are all very short, this shows that this is the limit of direct processing of raw logs with a general-purpose language model. It’s like trying to do professional engineering calculations on your smartphone: while you get a rough idea, you can’t expect it to be as accurate as a professional calculator.

This difference reaffirms that traditional analysis tools have a significant advantage in situations where accurate event extraction and numerical aggregation are required. The GCeasy blog also discussed this challenge in detail with AI-based GC log analysis.

The 89-versus-86 values displayed in the two separate GCeasy reports probably represent different categories of events – overall classified events versus aggregated pause-time-related events.

Before making any comparisons based on the values presented in different reports, it is critical to make sure that the metrics and report scopes are compatible.

Deterministic AI vs. LLM: Strengths and Limitations 

Although they both reached similar conclusions from the same GC log, their paths were completely different.

The language model excels in its “interpretation of behavior.” I think it’s wise to conclude that “even if GC occurs frequently, it’s not a serious problem as long as the stopping time is short” and associate the increase in heap usage with “allocating and retaining objects,” and it’s not easy to conclude that it’s a “memory leak.”

GCeasy’s AI went beyond the field of “numerical measurement”.

• Accurate calculation of 99.437% GC performance
• Measurement of allocation speed 405.13 MB/s and promotion speed 3.09 MB/s
• Classification of GC causes and visualization of memory changes for each generation

Features that stood out in the comparison:

• Language patterns are used to answer the question: “What does this behavior mean? What should I research next?”
• If you have a question such as the following, use GCeasy: “What specifically happened and how many times? What are the measured values?”

Neither approach was perfect.

Conclusion: Deterministic AI vs. LLM for GC Log Analysis 

The maximum value of the stopping time produced by the largest-scale language model had a slight difference from the GCeasy results – accurate numerical analysis with probabilistic word processing is definitely a risk. In fact, GCeasy also uses its own tools, and the amount of reporting of GC events has not been standardized. But the JVM tuning suggestions are quite useful, so I think it’s worth adopting them after you have checked the application-level profiling.

The important thing is, it’s not about switching completely to one or the other.

Qualitative analysis tools are responsible for the measurement, and the LLM complements the interpretation. For example, if both agree and give advice such as, “We should increase the retention rate of the Live set and the old generation,” this double-check is very convincing.

In the next article, we’ll dig deeper and ask the same questions of both approaches, checking which one is more reliable in a conversational GC log analysis workflow.

Deterministic analysis gives us measurements, but the LLM provides us with context. And when both approaches agree that something is happening (such as an increase in live-set retention and Old Gen occupancy), that’s when it gets interesting.

In next week’s post, we’ll ask the same set of questions to both the deterministic and probabilistic methods and see which one gives us better support for a given line of inquiry during a conversational GC log analysis.