From Rockets to Power Plants to Automobiles:
A Conversation With Real-Time Specification for Java Expert Greg Bollella

Print-friendly Version

First, can you sort out the difference between the ground software and the software on the Mars Rover?

In the current mission, the software used on the ground to create commands to send to the Rover, and the control software that actually sits on the Rover, are two very different systems with nothing in common whatsoever. What's being used up on the Rover is a well-known real-time operating system from Wind River Systems. The ground-side control system used Java software. I am working on a project called "Golden Gate" which will create code that would replace the proprietary APIs and real-time operating system code (Wind River) in future missions. Sun Labs, Carnegie Mellon-West, a campus of Carnegie Mellon located near Sun Labs in Silicon Valley, and the Jet Propulsion Labs (JPL) are working together on this project. We're working on an implementation of a software architecture developed at JPL called MDS, or Mission Data System. We're creating greater commonality between the flight system on the Rover and the ground system -- all essentially using Real-Time Specification for Java (RTSJ), and a more seamless development environment for the entire system.

"We're creating greater commonality between the flight system actually on the Rover, and the ground system -- all essentially using Real-Time Specification for Java (RTSJ)." - Greg Bollella, Distinguished Engineer and Senior Staff Engineer, Sun Microsystems

Java technology today is good for general purpose computing and GUIs, but it was not ready for use with control systems like the software on the Rover. The Golden Gate project seeks to use RTSJ to develop a system of control software that can be used on a Rover.

There is a growing trend in the device world to create software that takes charge of larger and larger parts of a system. JPL wants to equip the Rover with artificial intelligence capability so that it can maneuver on its own on the surface of Mars rather than depend on ground control. At the 2003 JavaOne Conference, James Gosling demonstrated such a system in his keynote there. To see James and Dan Dvorak of the Jet Propulsion Lab talking for six minutes about the Golden Gate project. You'll see a video of a Rover driven on Golden Gate code, which is Java code, running on an implementation of the RTSJ. The goal is to implement this Mission Data System on top of the Real-Time Specification for Java.

Controlling Physical Systems With RTSJ

What is the basic problem you are trying to solve?

If you have a physical system that needs to be controlled, it usually has three interacting parts -- sensing, control, and actuation. "Sensing" means there are sensors in the environment that measure physical quantities -- temperature, pressure, humidity, force, and so forth. "Control" takes the values from those sensors, and creates an output command for what's called an actuator. "Actuators" are hydraulic rods, rocket engines, attitude thrusters, heaters, and coolers, etc., that physically impact the environment. You can see how this might apply to a rover on Mars in constant interaction with its environment.

The Mission Data System (MDS) is an architecture that abstracts the problem of controlling such systems. All elements must talk to one another. I recognized early that MDS lacked the realization that it will execute on a real computer with all the limitations that digital computers have. For instance, threads, execution models, scheduling problems, interrupts, IO, and so on must be taken into account, along with serious time constraints.

If you're going to control something you must make sure that the time between sensing and command is limited by some maximum amount. You can't go off to the Internet and download something in the middle of trying to control something, because what you are trying to control will then fail. It's a fundamentally different system than we're used to today. My challenge is to give MDS the execution architecture so that we can map the functional pieces on to a real computer.

From Rockets to Power Plants

Why is Sun interested in rockets?

It has to do with a lot more than rockets - these physical systems are everywhere. Let me give you an example. We're implementing a version of the RTSJ in the Java HotSpot performance engine for an important customer. They're going to use this implementation for an application system that they're developing that controls fossil fuel power plants. These power plants have big turbines that you spin by heating water with fossil fuel, which makes the water hot, and creates steam. The steam gets compressed and routed along pipes; then you shoot it out of a nozzle at the blades of the turbine, which start to spin. The turbine is connected to the shaft, their generator. So both the turbine and generator spin, creating electricity that runs through the grid and cooks coffee or runs your computer.

The electricity generated at the plant must be at a specific phase and frequency, and the speed of the turbine must be kept constant, neither too fast nor too slow. The issues are complex, involving numerous sensors and ongoing feedback systems. The system has to figure out what to do next, and issue commands to the nozzles that spray steam on the turbine blades.

"Spacecrafts and power plants employ similar software systems." - Greg Bollella, Distinguished Engineer and Senior Staff Engineer, Sun Microsystems

This is similar to a spacecraft problem. Control systems with sensors, control mechanisms, and actuators are everywhere. Whenever computers control physical devices in the world, a control system is required. Spacecraft and power plants employ similar software systems. So Sun is interested in using computers to control physical systems.

We think that general-purpose computing and control-specific, or device computing, will begin to merge over the next few years. The two worlds, which were divided in the 1950s by IBM, who decided to separate business and scientific device computing, are merging again, in part, because of the Web.

The problem is that general-purpose computing success -- such as the Web and business computing transactions and performance -- is measured by speed. In the control space, when you have to manipulate physical devices, just being fast enough isn't good enough, because you also have to be predictable. And that's not so easy to do in general-purpose machines. In the future, devices will play an increasingly important role in the computing infrastructure. To manage such devices, real-time capability is essential, which is why we're engaged in our Golden Gate project.

The Mackinac Project: An RTSJ Implementation Based on a HotSpot Java Virtual Machine (JVM)

Your new RTSJ software project, Mackinac, addresses the challenges of physical systems. Tell us about it.

The Mackinac team is doing something very difficult. We're taking a commercial, highly-engineered, highly-optimized, extremely well-used system platform, the HotSpot JVM, and modifying it so that it not only retains its value for non-real-time programs, but so that it can also do real-time programming. This is much more challenging than building a small, deterministic, real-time virtual machine.

So why are you doing it this way?

We want to see systems coalesce so that the real-time bits -- the real-time programs and the non-real-time programs -- exist on the same JVM, rather than disparate machines on the system. Distributed systems are hard to build, even when they're limited like this. The Jet Propulsion Lab doesn't want a bunch of processors on the Rover. They want one processor, but they will have both real-time and non-real-time code, and both soft and hard real-time code on the same system. If you look at the power plant customer we're building a system for, their architecture is three-tiered now. They have control, management, and thin clients.

Mackinac is targeted to control the turbine. Management is the thing that collects data from the turbine, and displays it for the clients, which are browsers. So, the management node in the middle is a big server with a Web server on it. We want to coalesce the management node and the control node, because there's no reason you can't do both control and management on the same JVM with RTSJ. Automobiles are the same way. There's a move to coalesce the processors into a few big ones, rather than many little ones. We're doing this with the HotSpot JVM, and taking the more difficult route and not just building a deterministic JVM, because we want to support traditional non-real-time Java code as well as the HotSpot JVM does, in addition to doing real-time code. We've dug way, way deep into the HotSpot JVM, and we're changing some fundamental things there.

I want to make it clear that the RTSJ and any conformant implementation do not change the semantics of the Java Language Specification or the Java Virtual Machine Specification. The RTSJ only tightens semantics where needed, but the tightening does not introduce new semantics. Any non-real-time Java program will execute fine on an RTSJ implementation.

Scoped Memory: A Response to Garbage Collection Challenges

You are also doing interesting things with memory allocation and garbage collection.

The problem with garbage collection is that, although it makes programming really nice for developers, so far, it's been very difficult to get a collector that's predictable, because the application code isn't predictable. Memory allocation is kind of up and down. No one really allocates memory in a deterministic way in a Java program. So, RTSJ said, "Okay, we have to give the developers the ability to write some code that does not suffer from interference from the garbage collector in significant ways." In fact, the interference from the collector should be zero.

"We think that general purpose computing and control-specific, or device computing, will begin to merge over the next few years." - Greg Bollella, Distinguished Engineer and Senior Staff Engineer, Sun Microsystems

And to do that, we created this new model of memory design we call scoped memory and immortal memory. And the difference between the regular Java heap memory and immortal and scoped memory has to do with what determines the lifetime of an object. So, back in the days of C, we had memory allocation free. And so when an object came into being, it was dependent on the application code. When it disappeared, free, it was also dependent on the application code.

The Java language changed that by saying, "Objects come into being at the behest of the application code, but then they go away when they're no longer visible." And the garbage collector comes in behind the scene. So, it frees developers from having to release the objects. With RTSJ we said, "Okay, we have to have a slightly different lifetime model for objects." And so we have two kinds of memory, at least. There are others, but they're not as important. There is immortal memory, in which objects that get put into immortal stay there forever. As far as the application is concerned, they never go away.

Objects that are put in scoped memory stay there as long as any thread is executing in that particular scope. You can think of scoped memory as being like a method call or even a thread itself. So, you can create a no heap thread and give it a scoped memory area. And as long as that thread is running, the objects in the scoped memory are going to stay there. When the thread quits, when the run method ends, the objects will all go away. So, we collect objects on scoped memory areas rather than on individual objects, which enables us to reduce the interference from the traditional GC to zero during the execution of the logic of the thread.

Programming with scoped memory is much more difficult than with regular code. You have to think about memory again. For Real-Time embedded programmers, scoped memory is much easier than what they're used to. For Java programmers, it's going to be a challenge.

RTSJ also contains another class of threads, real-time threads, that the collector interferes with. We hope that real-time garbage collection technology in many future real-time systems will advance to the point where much of the code can be written in the traditional way where you don't pay attention to deallocation, but instead have a very smart real-time collector operating in the background. So, you will still understand the predictability of your threads. But we're not quite there yet. There are some real-time garbage collectors, and eventually Mackinac will include one, but we have to get scoped memory right first, because the customers want that now so they can have the predictability they need. And they don't mind doing the hard work with scoped memory.

"We've dug way, way deep into the HotSpot JVM, and we're changing some fundamental things there." - Greg Bollella, Distinguished Engineer and Senior Staff Engineer, Sun Microsystems

Another challenge involves scheduling issues, because the Java Language Specification doesn't require strict scheduling semantics. So, we had to require this in the RTSJ. We had to use classic real-time synchronization protocols and priority inversion avoidance algorithms to avoid low-priority threads from blocking high-priority threads, due to locks. We have incorporated much asynchronous support because the real world doesn't have much use for mental simplifications. And it's asynchronous with respect to the execution of computer logic. So, Mackinac has asynchronous events and asynchronous transfer control.

In addition, Mackinac includes access to physical memory. Writing code on top of it, which is real-time HotSpot JVM, will be more like writing system code than writing application code. You'll have access to physical memory. You can write device drivers. You can write very predictable code, and basically get down to direct access to the hardware, if you need it.

What about required dispatching, and the idea that it needs to be increased and made explicit?

Unfortunately RTSJ uses scheduling a bit too loosely. The quote and base scheduler required are really nothing more than a dispatcher. Higher-level scheduling policies like RMA (Rate Monotonic Analysis) or EDF (Earliest Deadline First) may come into play later when customers want it, but we currently require explicit dispatching.

Opportunities for Developers

What opportunities exist for developers here?

At the application level, developers who currently write code in the Java language can begin to write code that requires more temporal predictability. Let's say you are a developer for a financial institution that does trading, and they want a new trading application. Now, a developer who creates code for this can think about the problem in a way that includes time as a key element of his or her thinking in order to give more value to users.

Also, developers can expand the kinds of systems for which they have expertise. They can write code that controls power plants, automobiles, or spacecraft, so this opens opportunities in those areas as well.

In physical systems that make use of RTSJ code, a lot of non-real-time Java code will have to be written as well, another opportunity for Java programmers. In my opinion, there's no way that the code for the 2009 mission for the Mars Science Laboratory (MSL) will be written in C++. It's just too complex. I think that Java technology is going to be a requirement for that system.

See Also

Biography of Greg Bollella
Abstract of Greg Bollella's Dissertation
Java Technology and the Mars Mission
Real-Time Specification for Java
Standardizing Real-Time Development with Java Technology