2006 JavaOne Conference - A Robotic Dune Buggy Named Tommy (TS-3966)

Paul Perrone, founder and CEO of Perrone Robotics, and creator of Tommy, presented "A Robotic Dune Buggy Named Tommy" (TS-3966) on May 16. Perrone described how the company developed a 100 percent Java technology-based robotic dune buggy for a historic race of driverless robotic vehicles, and he told the unique and dramatic story behind Tommy.

"This was an extreme application of Java technology in an extreme environment," remarked Perrone, "an application that enabled a dune buggy to navigate through vast stretches of desert at a fairly high clip in real time."

Autonomous Robotic Vehicles

What is an autonomous ground vehicle (AGV)? The idea is to transfer control from humans to machines. Unmanned ground vehicles (UGVs) require no human on board, are tele-operated, and can be either autonomous or hybrid -- a mix of autonomy and human tele-operation control. AGVs also have no human on board, operate autonomously and remotely, and can navigate over terrain and avoid obstacles. Although such vehicles are rapidly evolving in autonomy, speed, and the distances they can successfully travel, Perrone was quick to point out that AGVs are a form of experimental technology with imperfect vehicle control, so the capacity for humans to override and control the vehicle in an emergency is critical.

The Uses of AGVs

Aside from being great fun, AGVs are beginning to demonstrate enormous value as their autonomy increases. "Autonomous vehicles have been around for a while, but it is only recently that we have seen that they can do actual work and accomplish actual missions," said Perrone. "In the near future, they will drive us in urban environments."

AGVs can be used in surveillance and security, for payload delivery and transport, for explosive ordinance disposal (EOD) and mine detection, for force protection and replacement in the military, in farming, as hospital carts, and ultimately, as self-driving cars and personal robotics.

The DARPA Grand Challenge

When Perrone asked how many people in the audience had heard of the DARPA Grand Challenge, everyone raised their hands, a stark contrast from Perrone's experience at the 2005 JavaOne conference, when only half the room had heard of it. Tommy was invited to participate in the 2005 Defense Advanced Research Projects Agency (DARPA) Grand Challenge, an AGV robot competition held in October 2005, as one of 40 robots that would race 150 miles through desert terrain. The first to complete the course in under 10 hours would win a cash prize of USD $2 million.

Racing teams received the coordinates only two hours before the race. The vehicles were entirely autonomous, with no external communication. DARPA had emergency-stop (e-stop) controls for all vehicles. Also, no federally funded technology could be involved. Vehicles first had to compete in the National Qualifying Event (NQE) at the California Speedway in Fontana, California, to make the desert finals.

Major technological progress took place between the 2004 and 2005 Challenge events. The 2004 Challenge, which required travel over 142 miles of rough terrain, offered a $1 million prize. But the farthest any vehicle traveled that year was 7.4 miles, with a few others going five to six miles. The 2005 Challenge required navigation over a roughly two-mile obstacle course in the NQEs, offered a $2 million prize, and required navigation over 132 miles of smoother terrain.

The winner finished in roughly seven hours. Three robots completed the course within the required 10-hour limit, and one vehicle completed it in roughly 13 hours. "This shows major advances in the technology over a year," said Perrone. Clearly, AGVs are a rapidly developing technology.

The Silver Egg That Could

Tommy, which Perrone referred to as the "silver egg that could," is a custom-built gas-powered dune buggy with a shell like a silver egg. It uses commercial off-the-shelf (COTS) electronics, COTS sensors, COTS actuators, and Perrone Robotics' patent-pending Java technology-based Mobile Autonomous X-bot (MAX) software, a general purpose robotics platform with configured, built-in services that rely on available and added drivers and extensions.

When asked why he chose a silver shell design, Perrone said that his team originally considered creating a self-righting capability for Tommy, to prevent the vehicle from rolling over. "We rejected that design feature but kept the shape because we thought it looked pretty cool," said Perrone. Tommy has a standard Subaru legacy engine inside.

System Architecture Overview

Perrone presented an overview of the system architecture. Tommy receives GPS and other information constantly. Its laser-radar system searches for obstacles or dents in the road, while its standard radar sees far ahead, and a lower laser radar perceives immediate obstacles. A number of other low-level sensors monitor the engine's RPM speed for real-time feedback control. Everything runs on a single microprocessor, where the main artificial intelligence (AI) takes place. Two microcontrollers provide lower-level feedback control for driving, shifting, steering, and braking.

"We went with a core robotics software engine which showed how fast and cheap a Java technology-based approach could be," explained Perrone. "And we made it configurable so it would be easier to focus on the programming challenge."

Meet Max

MAX runs on the Java Platform, Standard Edition (Java SE, formerly known as J2SE) and uses the Java Communications API. A general-purpose UGV framework, MAX-UGV, also written in the Java programming language, resides on the MAX platform in support of the DARPA Grand Challenge application. These "brains" process synchronous and asynchronous sensor information and events from other Java technology-based processors and microcontrollers. Using this data to formulate navigation plans, determine position and orientation, and resolve obstacles, Tommy can establish appropriate speed, steering, and braking actions.

Tommy's microprocessors rely on a hardware-based Java Virtual Machine (JVM) running Java Platform, Micro Edition (Java ME, formerly known as J2ME). The microcontrol platform is a version of MAX known as MAX, micro profile. Commands are sent to controllers over serial ports, and controllers receive feedback data from motor encoders. Controllers must act in real time, at high rates, and with the low latency that a feedback-driven dynamic mechanical system inherently requires.

Tommy's MAX Code

Perrone presented a lower-level generic MAX operation code example to demonstrate how his team programs with Java software. "There is first some initialization where you obtain a handle to your sensor which has already been defined and configured," said Perrone. "You must obtain a handle to your actuator and a component that may engage in decision making. Then, depending on how your component is triggered -- by a real-time synchronous thread or even another component -- you get a handle to your sensor state and some information, and then you set an actuator position."

Note: All code samples in this article are simplified for purposes of illustration.

General MAX Operation Example

Perrone next presented a Java ME micro profile example and explained: "Here we first get a handle to the command center and setup functions and then to a feedback control component and actuator. Then we register the steering feedback sensor and steering motor with the feedback control component. When this object is triggered, we get sensor value and the current state and the desired steering control."

MAX Micro Profile Example

Perrone followed with a simplified Java SE MAX standard profile example. "When we started, we expected to run Java software on multiple processors," he recalled, "but at the end of the day, we kept data crunching and adding sensors and more logic, and we never reached a performance bottleneck. So for those who think Java software is too slow or not high performance enough, we were able to do what we needed to do on a single microprocessor."

He explained that from a current position, the software first establishes the handles and components through the GPS sensor, motion sensor, steering motor, and a route. "The route," said Perrone, "keeps track of where you are: You get the current position from the GPS sensor and the desired position from your route. You calculate the desired direction based on that information, get your orientation information to calculate the directional difference between your current direction and your desired direction, and set some positional difference for your steering command."

Perrone explained that, although the example is oversimplified, it nevertheless illustrates the power of Java technology. "It has simple syntax, a high-level programming paradigm environment that makes it a lot easier." He said that Java technology enabled their faster, cheaper approach.

MAX Standard Profile Example

The Tommy Story

Perrone told the story of Tommy's origins and adventures. His group, Team Jefferson, originates from Charlottesville, Virginia, the home of Thomas Jefferson. That, along with the team's love of the rock opera Tommy, are the sources of Tommy's name. To create Tommy, they initially approached 30 dune buggy manufacturers -- to no avail. Finally, a company in Utah that produces sand rails, a type of dune buggy, took an interest and built Tommy.

A documentary film crew was embedded with the team as Tommy competed in the DARPA competition. Look for the documentary Autopilots.

The Tommy crew arrived at the California Speedway for the NQE and was doing well. After the third run, which was virtually perfect, Tommy ranked 10th in the field. Then, on the fourth day, disaster struck: A power surge led the controller responsible for speed control to stop responding, and Tommy accelerated out of control.

"We had a triple failure," explained Perrone. "We had an electrical-mechanical problem that we will never be able to explain. Then in the control software, we had a watchdog approach that focused on functionality and made certain assumptions where we thought the outputs would fall in a certain way, but our assumptions were violated. Finally, there was human error on the part of the race officials. We were screaming for 13 seconds at the race officials to give an e-stop signal, but they did not respond. And Tommy crashed at 60 miles an hour into a barrier and was nearly totaled."

The officials assumed that Tommy was finished, but Perrone's team managed to find a new engine and replace the now-demolished front end. Two hours before game time, Team Jefferson was still calibrating the steering in the dark. "Amazingly, all the sensors worked -- a testimony to the vendors," said Perrone. "Some of the feedback mechanisms were misaligned, but we got Tommy back on track and functional again, but we were not allowed into the Grand Challenge."

Despite these grave setbacks, the team took Tommy for test runs in the desert, runs, including one venture at 80 miles per hour. "We had a two-degree steering offset that we had to correct," said Perrone. "After that, Tommy did fine."

From a financial perspective, Tommy, which was built on a shoestring, yielded impressive results. The project cost $60,000 in parts and tools and $30,000 in travel. It entailed 10 months of software-development time and required a single processor card that cost less than $200 retail. Some of Tommy's competitors came from well-financed institutions with prior or current federal funding and involved more than $3 million, 100 man-years, and banks of powerful processors. Moreover, some teams had multiple spare vehicles.

Not the End of the Story: Tommy Jr.

Perrone has plans for the future: Tommy 2 will have a smaller platform than its predecessor, with the same electronics and software approach built into a more economical unit. The next DARPA Challenge will occur in an urban zone and require entrants to parallel park and negotiate traffic circles and merging vehicles.

So why did Perrone and his team create Tommy? "I wanted to see how general purpose the software would be," said Perrone. "We would not have gotten this far without this simple programming language and open-source components and the huge support base for Java software. We are currently working on the platform and have an SDK that people can use for free to help make Tommy 2 bigger, better, and meaner. So we're still at it and going strong."

For More Information