Java Device Test Suite

Today's service providers and device manufacturers face a variety of challenges when creating products to meet the increasing demand for mobile devices that take advantage of Java technology. The growing complexity and diversity of devices—with their varying operating systems, processors, and memory configurations—increases the need for thorough testing to ensure customers will be satisfied. At the same time, service providers and manufacturers face the challenge of managing—and, if possible, lowering—internal costs caused by excessive engineering overhead, disorganized development of test cases, or the impact of new data services on support operations.

Java Platform, Micro Edition (Java ME) technology is designed for consumer and embedded devices such as mobile phones, PDAs, TV set-top boxes, in-vehicle telematics systems, and a broad range of embedded devices. Over the past two years, more than 250 million Java technology-based mobile devices have been deployed worldwide. "The Yankee Group estimates a current installed base of 128 million Java technology-enabled handsets worldwide, and expects continued rapid growth for Java ME penetration globally," John Jackson, Director of Sun's Professional Services Migration Practices, said in November 2003. Before a manufacturer can make a device commercially available, however, and before a service provider can add new devices to its offerings, they must perform a significant amount of testing to ensure a high-quality, dependable product that will please their customers.

The Java Device Test Suite simplifies quality assurance and reduces time-to-market for Java ME implementations by providing comprehensive tests and a robust test manager. These enable suite users to evaluate, validate, and verify the quality of implementations of the Connected Limited Device Configuration (CLDC) and the Mobile Information Device Profile (MIDP) on a particular device. The Java Device Test Suite helps device manufacturers and service providers ensure their reputation for quality, while building customer satisfaction and loyalty.

Designed to reduce complex, in-house development and test management of multiple devices, the Java Device Test Suite meets the industry's need for a comprehensive, off-the-shelf software solution. It helps lower engineering costs by standardizing and simplifying testing, and by minimizing the need to write quality assurance tests manually.

With the Java Device Test Suite, each device can be tested against thousands of test cases that have been written and tested against the reference implementation for each technology. These test cases can be divided into four categories:

• Functional tests assure that the implementation behaves correctly, as intended. They imitate real applications and their interaction with external components, such as message senders and receivers.
• Stress tests reveal the robustness of a device by stretching it to the limits of memory and processor resources. These tests also exercise implementation response to boundary conditions, such as maximum message sizes and multithreaded invocations.
• Performance tests measure the time a device consumes to perform operations or combinations of operations.
• Security tests verify sandbox boundaries and privileges of MIDlets, and also verify that devices correctly implement the MIDP 2.0 security model, including authentication, signing, and permissions.

Developers may choose from automated tests, which don't require user interaction, and interactive tests, which rely on users' feedback. All tests cover devices that adhere to CLDC and MIDP specifications.

Users can easily select, configure, and execute test suites through the Test Console, which also automatically generates security certificates for MIDP testing. These highly customizable test suites may be configured to match device specifications. Tests are executed on the target device through a Test Agent, which has a small footprint that allows it to run on most devices, and enables developers to browse test results and device reports conveniently. The Test Console provides access to the central installation for device configurations, test suites, and specifications, simplifying the entire testing procedure.

The modular design of the Java Device Test Suite enables users to plug in additional test suites for new APIs and to execute selective test cases, as well as to build or plug in customized tests of specific functional areas, easily and rapidly. To match device configurations, the Test Manager makes possible the creation and editing of unlimited device profiles.

The comprehensive Java Device Test Suite includes several features designed to speed test execution:

• Parallel execution is supported, enabling the test execution load to be distributed over a number of devices.
• Multiple test suites can be executed at once.
• Several Test Consoles can be connected to the Test Manager concurrently, each of them testing multiple devices of the same configuration.

The Java Device Test Suite and the Technology Compatibility Kit (TCK) play complementary roles. The TCK verifies that the implementation is compatible and logically consistent with standards written in the specification at the API level, while the Java Device Test Suite tests the quality of the implementation: whether it is correct and robust. It does so by assembling into small applications separate features that the TCK tested independently, and confirming that they work together with the underlying operating system and hardware. To complete this task, the Java Device Test Suite actually performs operations, sends and receives data, simulates error conditions, and verifies that the implementation handles those conditions correctly.

A broad range of support services is available for the Java Device Test Suite, including product upgrades, new features and enhancements, new test suites, on-site installation, configuration, training, technical consultation, and test development. Developers can take advantage of two weeks of on-site engineering services per year, technical assistance by phone and e-mail, and technical information on the Web.

The Java Device Test Suite:

• Comprehensive Tests: Unmatched breadth and depth of test coverage covering multiple Java technologies including JTWI, CLDC, MIDP, MMAPI, WMA and 3D Graphics
• Benchmark Test Suite: Evaluate performance of Java on devices more thoroughly than any other existing benchmark on the market
• Developer's Kit: Extend test coverage or include customized, user-defined test suites
• Convenient Reporting: Provides test results and test execution reports in HTML format.
• Parallel Execution: To distribute test execution load over multiple devices, speeding test execution time
• Central Installation: Centrally stores device configurations, test suites, specifications, and administration to reduce maintenance costs and system complexity
• Test Manager: enables creation and editing of unlimited device profiles to match device configurations
• Test Console: Enables easy selection, configuration, and execution of test suites.
• Device Readiness Suite: Quickly diagnose a new device's readiness to run standard JDTS test suites