The Java HotSpot Virtual Machine, v1.4.1

Print-friendly Version

[Table of Contents/Page 1]  [Page 2]  [Page 3]  [Page 4]  [Page 5]  

The Java HotSpot Virtual Machine is Sun's VM for the Java platform. It delivers the optimal performance for Java applications using many advanced techniques, incorporating a state-of-the-art memory model, garbage collector, and adaptive optimizer. It is written in a high-level, object-oriented style, and features:

• Uniform object model

• Interpreted, compiled, and native frames all use the same stack

• Preemptive multithreading based on native threads

• Accurate generational and compacting garbage collection

• Ultra-fast thread synchronization

• Dynamic deoptimization and aggressive compiler optimizations

• System-specific runtime routines generated at VM startup time

• Compiler interface supporting parallel compilations

• Run-time profiling focuses compilation effort only on "hot" methods

The J2SE SDK version 1.4.1 release includes two flavors of the VM -- a client-side offering, and a VM tuned for server applications. These two solutions share the Java HotSpot runtime environment code base, but use different compilers that are suited to the distinctly unique performance characteristics of clients and servers. These differences include the compilation inlining policy and heap defaults.

The J2SE SDK contains both of the these systems in the distribution, so developers can choose which system they want by specifying -client or -server.

Although the Server and the Client VMs are similar, the Server VM has been specially tuned to maximize peak operating speed. It is intended for executing long-running server applications, which need the fastest possible operating speed more than a fast start-up time or smaller runtime memory footprint.

The Client VM compiler serves as an upgrade for both the Classic VM and the just-in-time (JIT) compilers used by previous versions of the Java SDK. The Client VM offers improved run time performance for applications and applets. The Java HotSpot Client VM has been specially tuned to reduce application start-up time and memory footprint, making it particularly well suited for client environments. In general, the client system is better for GUIs.

The Client VM compiler does not try to execute many of the more complex optimizations performed by the compiler in the Server VM, but in exchange, it requires less time to analyze and compile a piece of code. This means the Client VM can start up faster and requires a smaller memory footprint.

Figure 2-2: The Java HotSpot Client VM, on the left, and the Java HotSpot Server VM, on the right, use a different compiler but otherwise interface to the same virtual machine, using the same garbage collection (GC) routine, interpreter, thread and lock subsystems, and so on.

The Server VM contains an advanced adaptive compiler that supports many of the same types of optimizations performed by optimizing C++ compilers, as well as some optimizations that cannot be done by traditional compilers, such as aggressive inlining across virtual method invocations. This is a competitive and performance advantage over static compilers. Adaptive optimization technology is very flexible in its approach, and typically outperforms even advanced static analysis and compilation techniques.

Both solutions deliver extremely reliable, secure, and maintainable environments to meet the demands of today's enterprise customers.

In previous versions of the Java virtual machine, such as the Classic VM, indirect handles are used to represent object references. While this makes relocating objects easier during garbage collection, it represents a significant performance bottleneck, because accesses to the instance variables of Java programming language objects require two levels of indirection.

In the Java HotSpot VM, no handles are used by Java code. Object references are implemented as direct pointers. This provides C-speed access to instance variables. When the object is relocated during memory reclamation, the garbage collector is responsible for finding and updating all references to the object in place.

The Java HotSpot VM uses a two machine-word object header, as opposed to three words in the Classic VM. Since the average Java object size is small, this has a significant impact on space consumption -- saving approximately eight percent in heap size for typical applications. The first header word contains information such as the identity hash code and GC status information. The second is a reference to the object's class. Only arrays have a third header field, for the array size.

Classes, methods, and other internal reflective data are represented directly as objects on the heap (although those objects may not be directly accessible to Java technology-based programs). This not only simplifies the VM internal object model, but also allows classes to be collected by the same garbage collector used for other Java programming language objects.

Per-thread method activation stacks are represented using the host operating system's stack and thread model. Both Java programming language methods and native methods share the same stack, allowing fast calls between the C and Java programming languages. Fully preemptive Java programming language threads are supported using the host operating system's thread scheduling mechanism.

A major advantage of using native OS threads and scheduling is the ability to take advantage of native OS multiprocessing support transparently. Because the Java HotSpot VM is designed to be insensitive to race conditions caused by preemption and/or multiprocessing while executing Java programming language code, the Java programming language threads will automatically take advantage of whatever scheduling and processor allocation policies the native OS provides.

Back to Top

The generational nature of the HotSpot memory system provides the flexibility to use specific garbage collection algorithms suited to the needs of a diverse set of applications. In addition to the garbage collection algorithms available in the previous version of the Java HotSpot VM, two new collectors are available as options in this release: a multithreaded synchronous copying collectorfor use in the young generation, and a "mostly concurrent" asynchronous non--copying marksweep collector best suited for the old generation.

A major attraction of the Java programming language for programmers is that it is the first mainstream programming language to provide built-in automatic memory management, or garbage collection (GC). In traditional languages, dynamic memory is allocated using an explicit allocate/free model. In practice, this turns out to be not only a major source of memory leaks, program bugs, and crashes in programs written in traditional languages, but also a performance bottleneck and a major impediment to modular, reusable code. (Determining free points across module boundaries is nearly impossible without explicit and hard-to-understand cooperation between modules.) In the Java programming language, garbage collection is also an important part of the "safe" execution semantics required to support the security model.

A garbage collector automatically handles freeing of unused object memory behind the scenes by reclaiming an object only when it can prove that the object is no longer accessible to the running program. Automation of this process completely eliminates not only the memory leaks caused by freeing too little, but also the program crashes and hard-to-find reference bugs caused by freeing too much.

Traditionally, garbage collection has been considered an inefficient process that impeded performance, relative to an explicit-free model. In fact, with modern garbage collection technology, performance has improved so much that overall performance is actually substantially better than that provided by explicit freeing of objects.

In addition to including the state-of-the-art features described below, the memory system is designed as a clean, object-oriented framework that can easily be instrumented, experimented with, or extended to use new garbage collection algorithms.

The major features of the Java HotSpot garbage collector are presented below. Overall, these capabilities are well-suited both for applications where the highest possible performance is needed, and for long-running applications where memory leaks and memory inaccessibility due to fragmentation are highly undesirable.

The Java HotSpot garbage collector is a fully accurate collector.In contrast,many other garbage collectors are conservative or partially accurate. While conservative garbage collection can be attractive because it is very easy to add to a system without garbage collection support, it has certain drawbacks. In general, conservative garbage collectors are prone to memory leaks, disallow object migration, and can cause heap fragmentation.

A conservative collector does not know for sure where all object references are located. As a result, it must be conservative by assuming that memory words that appear to refer to an object are in fact object references. This means that it can make certain kinds of mistakes, such as confusing an integer for an object pointer. Memory cells that look like a pointer are regarded as a pointer -- and GC becomes inaccurate. This has several negative impacts.First,when such mistakes are made (which in practice is not very often),memory leaks can occur unpredictably in ways that are virtually impossible for application programmers to reproduce or debug. Second, since it might have made a mistake, a conservative collector must either use handles to refer indi- rectly to objects -- decreasing performance -- or avoid relocating objects, because relocating handleless objects requires updating all references to the objects. This cannot be done if the collector does not know for sure whether an apparent reference is a real reference. The inability to relocate objects causes object memory fragmentation and, more importantly, prevents use of the advanced generational copying collection algorithms described below.

Because the Java HotSpot collector is fully accurate, it can make several strong design guarantees that a conservative collector cannot make:

• All inaccessible object memory can be reclaimed reliably.

• All objects can be relocated, allowing object memory compaction, which eliminates object memory fragmentation and increases memory locality.

  An accurate garbage collection mechanism avoids accidental memory leaks, enables object migration, and provides for full heap compaction. The GC mechanism in the Java Hotspot VM scales well to very large heaps.

The Java HotSpot VM employs a state-of-the-art generational copying collector, which provides two major benefits:

• Increased allocation speed and overall garbage collection efficiency for most programs, compared to nongenerational collectors

• Corresponding decrease in the frequency and duration of user-perceivable garbage collection pauses

A generational collector takes advantage of the fact that in most programs,the vast majority of objects (often greater than 95 percent) are very short lived (for example, they are used as temporary data structures). By segregating newly created objects into an object nursery, a generational collector can accomplish several things. First, because new objects are allocated contiguously in stack-like fashion in the object nursery, allocation becomes extremely fast, since it merely involves updating a single pointer and performing a single check for nursery overflow. Secondly, by the time the nursery overflows, most of the objects in the nursery are already dead, allowing the garbage collector to simply move the few surviving objects elsewhere, and avoid doing any reclamation work for dead objects in the nursery.

The single-threaded copying collector described above,while suitable for many deployments, could become a bottleneck to scaling in an application that is otherwise parallelized to take advantage of multiple processors. To take full advantage of all available CPUs on a multiprocessor machine, version 1.4.1 of the HotSpot JVM offers an optional multithreaded collector for the young generation, in which the tracing and copying of live objects is accomplished by multiple threads working in parallel. The implementation has been carefully tuned to balance the collection work between all available processors, allowing the collector to scale up to large numbers of processors. This reduces the pause times for collecting young space and maximizes garbage collection throughput. The parallel collector has been tested with systems containing more than 100 CPU's and 0.5 terabytes of heap.

When moving objects, the parallel collector tries to keep related objects together, resulting in improved memory locality and cache utilization, and leading to improved mutator performance. This is accomplished by copying objects in depth first order.

The parallel collector also uses available memory more optimally. It does not need to keep a portion of the old object space in reserve to guarantee space for copying all live objects. Instead, it uses a novel technique to speculatively attempt to copy objects. If old object space is scarce this technique allows the collector to switch smoothly to compacting the heap without the need for holding any space in reserve. This results in better utilization of the available heap space.

Finally, the parallel collector is able to dynamically adjust its tunable parameters in response to the application's heap allocation behavior, leading to improved garbage collection performance over a wide range of applications and environments. This means less hand-tuning work for customers. This capability is currently available only for the parallel collector, but is expected to be further refined and extended to other collectors in future releases.

In comparison with the default single-threaded collector, the break-even point for the parallel collector appears to be somewhere between two and four CPUs, depending on the platform and the application. This is expected to further improve in future releases.

As mentioned above, this collector parallelizes only the work for minor (young space) collections. Major (old space) collections are also expected to be parallelized in future releases.

Although the generational copying collector collects most dead objects efficiently, longer-lived objects still accumulate in the old object memory area. Occasionally, based on low-memory conditions or programmatic requests, an old object garbage collection must be performed. The Java HotSpot VM can use a standard mark-compact collection algorithm, which traverses the entire graph of live objects from its roots, then sweeps through memory, compacting away the gaps left by dead objects. By compacting gaps in the heap, rather than collecting them into a freelist, memory fragmentation is eliminated, and old object allocation is streamlined by eliminating freelist searching.

The mark-compact collector does not eliminate all user-perceivable pauses. User-perceived GC pauses occur when old objects (objects that have lived for awhile in machine terms) need to be garbage collected, and these pauses are proportional to the amount of live object data that exists. This means that the pauses can become arbitrarily large as more data is manipulated, which is a very undesirable property for server applications, animation, or other soft-real time applications.

• Server applications,especially high-availability applications

• Applications that manipulate very large live object data sets

• Applications where all user-noticeable pauses are undesirable,such as games,animation,or other highly interactive applications

The low-pause collector works by using an incremental old space collection scheme referred to as the train algorithm. This algorithm breaks up old space collection pauses into many tiny pauses (typically less than ten milliseconds)that can be spread out over time so that the user never notices a pause. Since the train algorithm is not a hard-real time algorithm, it cannot guarantee an upper limit on pause times. However, in practice much larger pauses are extremely rare, and are not caused directly by large data sets.

The low-pause collector also has the highly desirable side benefit of improving memory locality. This happens because the algorithm attempts to relocate groups of tightly coupled objects into regions of adjacent memory, providing excellent paging and cache locality properties for those objects. This can also benefit highly multithreaded applications that operate on distinct sets of object data.

For applications that require large heaps, collection pauses induced by the default old generation mark-compact collector can often cause disruptions, as application threads are paused for a period that is proportional to the size of the heap. Version 1.4.1 of the Java HotSpot VM has implemented an optional concurrent collector for the old object space that can take advantage of spare processor cycles (or spare processors) to collect large heaps while pausing the application threads for very short periods. This is accomplished by doing the bulk of the tracing and sweeping work while the application threads are executing. In some cases, there may be a small decline in peak application throughput as some processor cycles are devoted to concurrent collection activity; however, both average-and worst-case garbage collection pause times are often reduced by one or two orders of magnitude, allowing much smoother application response without the burstiness sometimes seen when the default synchronous mark-compact algorithm operates on large heaps.

Back to Top

Previous releases of the Java HotSpot VM were limited to addressing four gigabytes of memory -- even on 64-bit operating systems such as the Solaris OE. While four gigabytes is a lot for a desktop system, modern servers can contain far more memory. For example, the new Sun Fire 15K server comes in a standard configuration with 288 gigabytes of memory. With a 64-bit JVM, Java technology-based applications can now utilize the full memory of such a system.

There are several classes of applications where using 64-bit addressing can be useful. For example, those that store very large data sets in memory. Applications can now avoid the overhead of paging data from disk or extracting it from a RDBMS. This can lead to dramatic performance improvements in applications of this type.

The Java HotSpot Server VM is now 64-bit safe, and the Server VM includes support for both 32-bit and 64-bit operations.Users can select either 32-bit or 64-bit operation by using commandline flags -d32 or -d64 respectively. Users of the Java Native Interface will need to recompile their code to run it on the 64-bit VM.

Object packing functionality has been added to minimize the wasted space between data types of different sizes. This is primarily a benefit in 64-bit environments, but offers a small advantage even in 32-bit VMs.

For example:

public class Button {
  char shape;
  String label;
  int xposition;
  int yposition;
  char color;
  int joe;
  object mike;
  char armed;
}

This would waste space between: color and joe (three bytes to pad to an int boundary) joe and mike (four bytes on a 64-bit VM to pad to a pointer boundary) Now, the fields are reordered to look like this:

...
  object mike;
  int joe;
  char color;
  char armed;
...

In this example, no memory space is wasted.

The Java programming language allows for use of multiple,concurrent paths of program execution -- threads. The Java programming language provides language-level thread synchronization, which makes it easy to express multithreaded programs with fine-grained locking. Previous synchronization implementations were highly inefficient relative to other micro-operations in the Java programming language, making use of fine-grain synchronization a major performance bottleneck.

The Java HotSpot VM incorporates a breakthrough in thread synchronization implementation that boosts synchronization performance by a large factor. As a result, synchronization performance becomes so fast that it is not a significant performance issue for the vast majority of real-world programs.

In addition to the space benefits mentioned in the Memory Model section,the synchronization mechanism delivers its performance benefits by providing ultra-fast, constant-time performance for all uncontended synchronizations, which dynamically comprise the majority of synchronizations.

The Java HotSpot VM provides a leaner, speedier thread-handling capability that is designed to scale readily for use in large, shared-memory multiprocessor servers.

The new I/O (NIO) APIs introduced in version 1.4 provide new features and improved performance in the areas of buffer management, scalable network and file I/O, character-set support, and regular-expression matching. These new APIs are supported in both client and server compilers. With NIO, developers can now write ultra-scalable, high-performance server applications such as Web, application, file, and database servers. They can also write compute-intensive scientific, technical, and graphics applications that require fast access to large quantities of data. I/O operations that previously required programming in C or C++ can now be performed using the Java language, but with the performance of a native C or C++ application.

• The new network I/O package dramatically increases the number of simultaneous connections that a server can handle by removing the need to dedicate one thread to every open connection.

• New file I/O supports read, write, copy, and transfer operations that are up to twice as fast as the current file I/O facilities. It also supports file locking, memory-mapped files, and multiple concurrent read/write operations.

Back to Top

[Table of Contents/Page 1]  [Page 2]  [Page 3]  [Page 4]  [Page 5]