|
Tuning Garbage Collection
with the 1.3.1 Java Virtual Machine
|
See also Performance Docs |
Introduction
The Java 2 Platform is increasingly used for large server applications such as web services.
These applications demand scalability, and directly benefit from large numbers
of threads, processors, sockets and memory. Yet 'big iron' performance
has a reputation as an art form, requiring special expertise beyond what
is needed for performance on smaller systems. Fortunately, the Java
Virtual Machine (JVM)* and Solaris operating
environment provide effective implementations
of threads, I/O and memory management. This document addresses a common
speed bump on the road to scalable high performance: poorly tuned garbage
collection (GC).
Amdahl observed that most workloads cannot be perfectly parallelized; some
portion is always sequential and does not benefit from parallelism.
This is also true for the Java 2 Platform. In particular, JVMs up to and including
version 1.3.1 do not have parallel garbage collection, so the impact of GC
on a multiprocessor system grows relative to an otherwise parallel
application.
The graph below models an ideal system that is perfectly scalable with
the exception of GC. The top line (red) is an application spending
only 1% of the time in GC on a uniprocessor; this translates into more than
20% loss in throughput at 32 processors. At 10%, not considered an outrageous
amount of time in GC in uniprocessor applications, more than 75% of throughput
is lost when scaling up.
This demonstrates that issues that appear lost in the noise when developing
on small systems may become principal bottlenecks when scaling up.
The silver lining is that small improvements in such a bottleneck can produce
large gains in performance. For a sufficiently large system it becomes
well worthwhile to tune garbage collection.
This document is written from the perspective of 1.3.1 JVM on the
Solaris (SPARC Platform Edition) operating environment,
because that platform provides the most scalable hardware/software Java 2
platform today. However, the descriptive text applies to other
supported platforms, including Linux, Microsoft Windows, and the
Solaris (Intel Architecture) operating environment, to the extent
that scalable hardware is available. Although command line options
are consistent across platforms, some platforms may have different defaults
than described here.
Generations
One of Java 2 Platform's great strengths is that it shields the substantial complexity
of memory allocation and garbage collection from the developer. However,
once GC has become the principal bottleneck, it becomes worth understanding
aspects of this hidden implementation. Garbage collectors make assumptions
about the way applications use objects, and these are reflected in tunable
parameters that can be adjusted for improved performance without sacrificing
the power of the abstraction.
An object is garbage when it can no longer be reached from any pointer
in the running program. The most straightforward garbage collection
algorithms simply iterate over every reachable object; any objects left
over are then known to be garbage. This approach takes time proportional
to the number of living objects, which is prohibitive for large applications
maintaining lots of living data.
The 1.3 JVM incorporates a number of different garbage collection algorithms
that are combined using generational collection. While naive
garbage collection examines every living object in the heap, generational
collection exploits several empirically observed properties of most applications
to avoid extra work.
The most important of these properties is infant mortality.
The blue area in the diagram below is a typical distribution for
the lifetimes of objects. The sharp peak at the left represents objects
that can be reclaimed shortly after being allocated. Iterator objects,
for example, are often alive for the duration of a single loop.
Some objects do live longer, and so the distribution stretches out to the
the right. For instance, there are typically some objects allocated
at initialization that live until the process exits. Between these
two extremes are objects that live for the duration of some intermediate
computation, seen here as the lump to the right of the infant mortality
peak. Some applications have very different looking distributions,
but a surprisingly large number possess this general shape. Efficient
collection is made possible by focusing on the fact that a majority of objects
die young.
To do this, memory is managed in generations: memory pools holding
objects of different ages. Garbage collection occurs in each generation
when it fills up; these collections are represented on the diagram above
with vertical bars. Objects are allocated in eden, and because
of infant mortality most objects die there. When Eden fills up it causes
a minor collection, in which some surviving objects are moved to
an older generation. When older generations need to be collected there
is a major collection that is often much slower because it involves
all living objects.
The diagram shows a well-tuned system in which most objects die before
they survive to the first garbage collection. The longer an object survives,
the more collections it will endure and the slower GC becomes. By arranging
for most objects to survive less than one collection, garbage collection
can be very efficient. This happy situation can be upset by applications
with unusual lifetime distributions, or by poorly sized generations that
cause collections to be too frequent.
The default garbage collection parameters were designed to be effective
for most small applications. They aren't optimal for many server applications.
This leads to the central tenet of this document:
|
If GC has become a bottleneck, you may wish to customize the generation
sizes. Check the verbose GC output, and then explore the sensitivity
of your individual performance metric to the GC parameters.
|
Types of collection
Each generation has an associated type of garbage collection that
can be configured to make different algorithmic time, space and pause
tradeoffs. In 1.3, the JVM implements three very different garbage
collectors:
- Copying (sometimes called scavenge): this collector
very efficiently moves objects between two or more generations. The
source generations are left empty, allowing remaining dead objects to be
reclaimed quickly. However, since it requires empty space to operate,
copying requires more footprint. In 1.3.1 copying collection is used
for all minor collections.
- Mark-compact: this collector allows generations to be collected
in place without reserving extra memory; however, compaction is significantly
slower than copying. In 1.3.1 mark-compact is used for major collections.
- Incremental (sometimes called train): this collector
is used only if -Xincgc is
passed on the command line. By careful bookkeeping, incremental GC
collects just a portion of the old generation at a time, trying to spread
the large pause of a major collection over many minor collections.
However, it is even slower than mark-compact when considering overall throughput.
Since copying is very fast, a tuning goal is to collect as many objects
as possible by copying rather than by compaction or incremental collection.
The default arrangement of generations looks something like this.
At initialization, a maximum address space is virtually reserved but not
allocated physical memory unless it is needed. The complete address
space reserved for object memory can be divided into the young and old generations.
The young generation consists of eden plus two survivor spaces
. Objects are initially allocated in eden. One survivor space
is empty at any time, and serves as the destination of the next copying
collection of any living objects in eden and the other survivor space.
Objects are copied between survivor spaces in this way until they age enough
to be tenured (copied to the old generation.)
(Other virtual machines, including the production JVM version 1.2 for
the Solaris operating environment,
used two equally sized spaces for copying rather than one large eden plus
two small spaces. This means the options for sizing the young generation
are not directly comparable; see the
Performance FAQ for an example.)
The old generation is collected in place by mark-compact. One portion
called the permanent generation is special because it holds all the
reflective data of the JVM itself, such as class and method objects.
Performance considerations
There are two primary measures of garbage collection performance.
Throughput is the percentage of total time not spent in garbage
collection, considered over long periods of time.
Throughput includes time spent in allocation (but tuning
for speed of allocation is generally not needed.)
Pauses are
the times when an application appears unresponsive because garbage collection
is going on.
Users have different requirements of garbage collection. For example,
some consider the right metric for a web server to be throughput, since
pauses during garbage collection may be tolerable, or simply obscured by
network latencies. But for an interactive graphical program, even
short pauses may upset the user experience.
Some users are sensitive to other considerations. Footprint
is the working set of a process, measured in pages and cache lines.
On systems with limited physical memory or many processes, footprint may
dictate scalability. Promptness is the time between when an
object becomes dead and when the memory becomes available, an important consideration
for distributed systems including RMI.
In general, a particular generation sizing chooses a trade-off between
these considerations. For example, a very large young
generation may maximize throughput, but does so at
the expense of footprint and promptness.
Pauses can be minimized by using a small young generation and incremental
collection, at the expense of throughput.
There is no one right way to size generations; the best choice is determined
by the way the application uses memory as well as user requirements.
For this reason the JVM's default GC choices may not be optimal, and may
be overridden by the user in the form of command line options below.
Measurement
Throughput and footprint are best measured using metrics particular to the
application. For example, throughput of a web server may be tested
using a client load generator, while footprint of the server might be measured
on the Solaris operating environment using the pmap
command. On the other hand, pauses due to GC are easily estimated
by inspecting the diagnostic output of the JVM itself.
The command line argument -verbose:gc
prints information at every collection. For example, here is output
from a large server application:
[GC 325407K->83000K(776768K),
0.2300771 secs]
[GC 325816K->83372K(776768K), 0.2454258 secs]
[Full GC 267628K->83769K(776768K),
1.8479984 secs]
Here we see two minor collections and one major one. The numbers
before and after the arrow indicate the combined size of live objects before
and after the GC. (After minor collections the count includes objects
that aren't necessarily alive but can't be reclaimed, either because they
are directly alive, or because they are within or referenced from the
old generation.) The number in parenthesis is the total available space,
which is the total heap minus one of the survivor spaces.
Sizing the generations
A number of parameters affect generation size. This diagram
illustrates the ones most important to tuning the 1.3.1 JVM.
Many parameters are actually ratios x:y, and these are
depicted with black (representing x) and grey (representing
y) size bars: 
Total heap
Since collections occur when generations fill up, throughput is inversely
proprotional to the amount of memory available. Total available memory
is the most important knob affecting GC performance.
By default, the JVM grows or shrinks the heap at each collection to try to
keep the proportion of free space to living objects at each collection within
a specific range. This target range is set as a percentage by the parameters
-XX:MinHeapFreeRatio=<minimum> and
-XX:MaxHeapFreeRatio=<maximum>, and the total size is bounded below by
-Xms and above by -Xmx
. The default parameters for the Solaris (SPARC Platform Edition)
operating environment are shown in this table:
-XX:MinFreeHeapRatio=
|
40
|
-XX:MaxHeapFreeRatio=
|
70
|
-Xms
|
3584k
|
-Xmx
|
64m
|
Large server apps often experience two problems with these defaults.
One is slow startup, because the initial heap is small and must be resized
over many major collections. A more pressing problem is that the default
maximum heap size is unreasonably small for most server applications.
The rules of thumb for server applications are:
Unless
you have problems with pauses, try granting as much memory as possible to
the JVM. The default size (64MB) is often too small.
Setting -Xms and
-Xmx to the same value increases predictability by removing the
most important sizing decision from the JVM. On the other hand, the
JVM can't compensate if you make a poor choice.
Be sure to increase the memory as you increase the number of processors,
since allocation can be parallelized, but GC is not parallel.
|
The young generation
The second most influential knob is the proportion of the heap dedicated
to the young generation. The bigger the young generation, the less
often minor collections occur. However, for a bounded heap size a larger
young generation implies a smaller old generation, which will increase the
frequency of major collections. The optimal choice depends on the lifetime
distribution of the application.
By default, the young generation size is controlled by NewRatio.
For example, setting -XX:NewRatio=3
means that the ratio between the young and old generation is 1:3; in other
words, the combined size of eden and the survivor spaces will be one fourth
of the heap.
The parameters NewSize and
MaxNewSize bound the young
generation size below and above. Setting these equal to one another
fixes the young generation, just as setting
-Xms and -Xmx equal
fixes the total heap size. This is useful for tuning the young generation
at a finer granularity than the integral multiples allowed by
NewRatio.
Because the young generation uses copying collection, enough free memory
must be reserved in the old generation to ensure that a minor collection
can complete. In the worst case, this reserved memory is equal to the
size of eden plus the objects in non-empty survivor space. When there isn't enough memory
available in the old generation for this worst case, a major collection will
occur instead. This policy is fine for small applications, because
the memory reserved in the old generation is typically only virtually committed
but not actually used. But for applications needing the largest possible
heap, an eden bigger than half the virtually committed size of the heap
is useless: only major collections would occur.
If desired, the parameter SurvivorRatio
can be used to tune the size of the survivor spaces, but this is often
not as important to performance. For example,
-XX:SurvivorRatio=6 sets the ratio between each survivor space
and eden to be 1:6; in other words, each survivor space will be one eighth
of the young generation (not one seventh, because there are two survivor
spaces).
If survivor spaces are too small, copying collection overflows directly
into the old generation. If survivor spaces are too large, they will
be uselessly empty. At each garbage collection the JVM chooses a threshold
number of times an object can be copied before it is tenured. This
threshold is chosen to keep the survivors half full. (For the intrepid,
a 1.3.1 option -XX:+PrintTenuringDistribution
can be used to show this threshold and the ages of objects in the new
generation. It is also useful for observing the lifetime distribution
of an application.)
Here are the default values for the Solaris (SPARC Platform Edition)
operating environment:
NewRatio
|
2 (client JVM:
8)
|
NewSize
|
2172k
|
MaxNewSize
|
32m
|
SurvivorRatio
|
25
|
The rules of thumb for server applications are:
First
decide the total amount of memory you can afford to give the JVM.
Then graph your own performance metric against young generation sizes to
find the best setting.
Unless you find problems with excessive major collection or pause times, grant plenty
of memory to the young generation. The default
MaxNewSize (32MB) is generally too small.
Increasing the young generation becomes counterproductive at half the total
heap or less.
Be sure to increase the young generation as you increase the number of
processors, since allocation can be parallelized, but GC is not parallel.
|
Other considerations
For most applications the permanent generation is not relevant to GC performance.
However, some applications dynamically generate and load many classes.
For instance, some implementations of JSPs do this. If necessary,
the maximum permanent generation size can be increased with
MaxPermSize.
Some applications interact with garbage collection by using finalization
and weak/soft/phantom references. These features can create performance
artifacts at a Java-programming-language level; an example is relying on finalization to close
file descriptors, which makes an external resource (descriptors) dependent
on GC promptness. Relying on GC to manage resources other than memory
is almost always a bad idea.
Another way apps can interact with garbage collection is by invoking GCs
explicitly, such as through the
System.gc() call. These calls force major collection,
and inhibit scalability on large systems. The performance impact of
explicit GCs can be measured using the unsupported flag
-XX:+DisableExplicitGC.
One of the most commonly encountered uses of explicit GC occurs with RMI's
distributed garbage collection (DGC). Applications using RMI refer
to objects in other JVMs. Garbage can't be collected in these distributed
applications without occasional local collection, so RMI forces periodic
full collection. The frequency of these collections can be controlled
with properties. For example,
java -Dsun.rmi.dgc.client.gcInterval=3600000
-Dsun.rmi.dgc.server.gcInterval=3600000 ...
specifies explicit collection once per hour instead of the default rate
of once per minute. However, this may also cause some objects to take much
longer to be reclaimed. These properties can be set as high as Long.MAX_VALUE
to make the time between explicit collections effectively infinite, if there
is no desire for an upper bound on the timeliness of DGC activity.
The Solaris 8 operating environment supports an alternate version of libthread that binds threads to
LWPs directly; this may help avoid starvation of the finalization thread.
To try this, set the environment variable
LD_LIBRARY_PATH to include
/usr/lib/lwp before launching the JVM.
Soft references are cleared less aggressively in the server JVM than the
client. The rate of clearing can be slowed by increasing a parameter
in this way: -XX:SoftRefLRUPolicyMSPerMB=10000. The default is value 1000, or one second per megabyte.
For large dedicated systems, there are other
special options available to boost performance.
Conclusion
Garbage collection can become a bottleneck in highly parallel systems.
By understanding how GC works, it is possible to use a variety of command
line options to minimize that impact.
The demands of large servers are being met with larger hardware configurations
that ever before. For these systems, the 1.4 line of JVMs will provide
additional solutions, including a 64 bit address space for even larger generations
and concurrent collection to hide the pauses associated with major collection.
As used on the web site, the terms "Java Virtual
Machine and "JVM" mean a virtual machine for the Java platform.
