Articles: Zooming In On Accessibility

Print-friendly Version

Articles Index

Futurists in the 1950s predicted that computers would be doing all the work by now, and we would be free to write poetry and dance through fields of flowers. That cyber-utopia didn't quite pan out, but we can expect future computers to serve our needs as unique individuals. The Java Accessibility APIs are helping to bring those expectations closer to reality.

The World Health Organization estimates that 750 million people worldwide are dealing with a significant level of disability. Computer technologies must do a better job at fulfilling the needs of this huge segment of the population. This is what accessibility is all about.

The basic categories of disability are:

• Visual: Blindness/low vision; difficulty with screen-based content and limited- or inability to use a mouse.
• Physical: Difficulty using standard input devices like keyboard and mouse.
• Hearing: Difficulty with auditory content.
• Learning: Difficulty with auditory and/or visual content.

Computer-based assistive technologies fall under two major headings:

• Augmented output devices
• Screen Reader: presents audio or Braille output of screen GUI content
• Screen Magnifier: provides options for magnification of screen-based events
• Augmented input devices
• Speech Recognition: facilitates verbal command of keyboard and mouse functions
• Alternative keyboards (e.g. on-screen keyboards)
• Alternative mice (e.g. hands-free, head pointing devices)

The Java Accessibility APIs and Utility APIs help to make applications and assistive technologies more responsive to our individual differences. For those with significant disabilities, accommodation of those differences can be crucial.

One Woman's Accessibility Odyssey

As the technical writer for Sun's Accessibility Team, Dena Shumila's responsibilities are broad ranging. She prepares all Accessibility API documents, product release announcements, and makes sure that the java.sun site is accessible and up-to-date. "Pretty much anything connected with verbiage and accessibility is my domain," she says in rapid-fire cadence.

Shumila was born with congenital glaucoma, a genetic eye disorder. She had some sight as a child, but by age 10 was learning Braille and walking with a white cane. Computers entered the scene early.

"I got my first machine at age 12," she says. "It was an Apple IIe with no hard drive, two 5 1/4 inch floppies, and a speech synthesizer." She used it in conjunction with a keypad device called VersaBraille, which stored typed notes on an audio cassette while displaying the data on a Braille display. It also allowed her to transfer the cassette data to her Apple IIe for print or speech synthesis output. The two devices got her through high school. At the University of Winnipeg, she acquired a DOS-based, 20-MHz, 386-powered Toshiba laptop. "It was a rockin' machine," she says, laughing. She used it as her personal system until about two years ago. Now she has migrated to a Wintel desktop box with a refreshable Braille display positioned at the bottom edge of her keyboard. It operates as a peripheral device, echoing her keystrokes and interrogating screen outputs and icon annotations. The data is presented by a software "screen reader."

Destination: Sun

Shumila graduated from the University of Winnipeg after working part-time at the University of Toronto during hiatus periods. Her background with U of T led to a position on the team that designed the audio "look & feel" for Notepad and the concept demo for the Java Foundation Classes (JFC) Project Swing (Project Swing) Set.

"It's the exact analog of a visual look & feel," she says. In the realm of audio, "look" is defined as the information coming out of the machine, and "feel" is the way information is presented to the machine. "We demonstrated how a radio button, or a check box, or a tab pane might sound. We also designed sounds to represent a component's change of state. The audio look & feel was designed to interact simultaneously with the visual interface, and provide sound cues for a hand-free, eye-free environment."

Each of the Project Swing set components was represented by a sound icon. And the interfaces operated in conjunction with Java Look and Feel. Shumila's demo used keyboard input exclusively for the feel component. There is no commercially viable way for a blind person to track the movements of a mouse.

The goal of the project was to provide a unique audio signature for Java components, while maintaining a familiarity factor for people who had used audio interfaces before. Sun was impressed with the results, and when the job as technical writer for the Accessibility Team opened up, Dena Shumila was at the top of the list.

Accessibility by Design

At the CSUN "Technology and Persons with Disabilities" conference, Sun's Peter Korn noted that, in the past, screen readers had to unravel complicated graphics calls as they unfolded in real time to determine the characteristics of GUI components. For instance, if text was being drawn, then a line was drawn in close proximity, it might indicate an underline or a strike-through, depending on subtle differences in orientation.

"Instead of figuring out what's going on from reading the tea leaves of the graphics calls, you can just talk to the object directly and ask, 'Hey, what are your attributes?' And you'll get back a wealth of information, such as, 'I'm 14 point, I'm bold italic Times Roman, and I'm red,'" Korn says. In Java 2, all this is rendered under Windows via direct text, no screen reader will ever get a single pixel—which is a good thing. "By going to the objects directly you can get more information than you could ever deduce from data that is pouring chaotically out to the screen. There are many compelling reasons for sidestepping the native facilities and rendering the text under the Java Platform. The most important is complete platform independence, and absolute consistency of the look & feel across platforms."

The Inner-Workings of Java Accessibility

With Project Swing user interface components, assistive technologies like screen readers, speech recognition, and screen magnification are automatically supported. The Pluggable Look and Feel (PLAF) architecture ensures that accessible interfaces are interchangeable with the standard JLF. Accessibility is an intrinsic aspect of the language design.

There are two accessibility programming packages of importance:

• The Accessibility APIs
• The Accessibility Utility APIs.

The Accessibility APIs, which are contained in the javax.accessibility package, define application side interfaces and classes that make components "accessibility friendly." They provide a wide array of detailed information about GUI components. The Accessibility Utility APIs rely on this information to implement the functions of assistive technologies.

The Accessibility Utility APIs are designed for developers of assistive technologies. They offer support for locating and querying user interface objects, tracking events, installing event listeners, and loading the assistive technology into the virtual machine.

JavaSoft bundles the Accessibility APIs with the Project Swing libraries. The Accessibility Utility APIs are downloadable independently.

AccessibleContext

The Accessibility API method getAccessibleContext() returns a customized AccessibleContext object. This object can then be used to retrieve detailed information about the accessible component of interest.

The AccessibleContext class is the focal point of accessibility. It makes a broad range of information available to any assistive technology. The class has three primary functions.

• Provide access to accessible properties of the host component
• Name
• Role
• Description
• Current State
• Parent Object (if applicable)
• Child Object (if applicable)
• Register any change of events on properties
• Report accessible functionality exported by component

An accessible context object can implement one or more of the following interfaces to determine what types of functionality can be used.

• AccessibleAction - Gets information about what actions the accessible object can perform, and tells the accessible object to perform actions.
• AccessibleComponent - Provides information about the accessible object's size, position, visibility, etc. The accessible contexts for all standard Project Swing components implement this interface. Custom components should do the same.
• AccessibleText - Allows assistive technologies to query a component for information about the current selections.
• AccessibleHypertext - Indicates that an accessible object contains hyperlinks, and provides information about the links, allowing them to be traversed.
• AccessibleSelection - Reports information about the current selection state, and can facilitate modification.
• AccessibleText - Indicates that an object displays text, and returns all or part of the text, plus length and other attributes.
• AccessibleValue - Indicates that an object is numeric and provides information about current value, minimum and maximum.

The Accessibility Utility Classes

The utilities package contains classes that help assistive technologies interface with the Java¹ Virtual Machine (JVM) on startup. In addition, the classes facilitate communication with accessibility friendly components, allowing for the capture and interpretation of various system events.

The accessibility utility classes allow assistive technologies to:

• Get a list of top-level windows for all applications executing in the JVMJava V.
• Locate current mouse position.
• Locate the window with input focus.
• Register listeners for appearance and disappearance of top level windows.
• Insert events into windowing event queue and register related listeners.

Screen Reader Example

If Dena Shumila were using IBM's Self-Voicing-Kit, the first screen reader written entirely in the Java programming language, she would be relying on the Accessibility APIs to provide information about various aspects of the screen layout. That data could very easily be routed either to a speech synthesizer or a refreshable Braille display. Let's say she is interested in exiting an application. She could navigate available on-screen options with the arrow keys until she hits "Exit" which might be spoken by a speech synthesis box receiving data from her system's serial port. The descriptive data would have been established during code development through the Accessibility API AccessibleName and AccessibleDescription properties.

Accessible Names and Descriptions

The AccessibleName property is used by assistive technologies to distinguish one component from others of the same type. When named components are extended from AbstractButton, the accessible name is automatically derived from its label. For all others, the AccessibleName should be explicitly set. The names should be as succinct as possible while maintaining uniqueness. The AccessibleDescription string should be somewhat more elaborate. An example of explicitly setting name and description follows.

    
JButton exit = new JButton("Exit");
exit.getAccessibleContext().setAccessibleName(
 "Exit Button");
exit.getAccessibleContext(
      ).setAccessibleDescription(
 "Click this button to exit application.");

There are numerous other AccessibleContext properties that can be set and/or queried by the programmer to establish various kinds of accessibility information, like states, roles, and hierarchies.

ToolTips

Another way to ensure accessibility is by using ToolTips with interface components. Sighted users can invoke a ToolTip by hovering the mouse over an icon, button, or option of interest. ToolTips can also be revealed by pressing the CNTL-F1 key combination when a button has focus. A window of annotation text then pops up. The data can also be interrogated through the Accessibility APIs. Simple string-based ToolTips can be automatically set or retrieved using the toolTipText property of JComponent, as illustrated below.

JButton btn = new JButton 
  ("Press Here");btn.setToolTipText(
    "World Reset");

When the mouse hovers, the text "World Reset" pops up. Both pieces of information could be retrieved by a screen reader or other assistive technology using the Accessibility APIs. If a ToolTip is set, it is not necessary to set the AccessibleName property.

More Accessibility Programming Tips

The LabelFor Property

Setting the LabelFor property on applicable components creates a linkage between labels and related fields or components. For instance, if a JLabel is in front of an editable field, setting the LabelFor property allows users with disabilities (such as blindness) to discover the relationship between the editable field and the associated descriptive label.

Tab Navigation Order

Many disabled users navigate GUI components solely with keyboard commands. It is important that the navigational flow be well thoughtout in terms of efficiency and accepted expectations of the locale. For instance, where English is the predominant language, a left-to-right, top-to-bottom flow structure is appropriate.

Mnemonics

It is often cumbersome and inefficient for users to tab through a whole list of components to find a particular area of interest (especially if that area is directly above the current position.) In such cases, key sequences are used. Typically the ALT key combined with a letter is used to select a component. If there are no text entry fields, sub-options are chosen by a single key press. Accessible applications should provide for mnemonic key selection as part of a generalized accessible design.

A Mnemonic keystroke character is associated with a button using the setMnemonic method. This method indicates which letter will be underlined in the label. The method is found in the AbstractButton class.

The setDisplayedMnemonic method is used to tie a mnemonic to a component that is part of a larger entity, such as a text field with an associated label. LabelFor provides the linkage between the label and the field of interest. The setDisplayedMnemonic(char aChar) method indicates which letter is underlined in the label. The JLabel class contains the setDisplayedMnemonic method.

JMenuItem(String, int) is used to set mnemonics on menu components. The integer argument indicates which letter is underlined in the string argument. The JMenuItem constructor calls AbstractButton.setMnemonic() which uses the int variable to specify the mnemonic keycode that activates the button. The JMenuItem method is found in the JMenuItem class.

Accelerators are used to invoke a menu item's action via a key combination. For instance, CNTL-O might be used as an accelerator for the Open function within a menu. A listener grabs the CNTL-O and the open function is called. The setAccelerator(KeyStroke keyStroke) method is found in the JMenuItem class.

JComponent

The JComponent class does not implement the Accessible interface, so instances of its direct subclasses are not accessible. If you write a custom component that inherits directly from JComponent, you must explicitly implement the Accessible interface. JComponent does have an accessible context, called AccessibleJComponent, that implements the AccessibleComponent interface and provides a minimal amount of accessible information. You can provide an accessible context for your own custom components by creating a subclass of AccessibleJComponent and overriding the pertinent methods.

Threading

The use of multithreading is important in making sure your application supports the simultaneous execution of multiple assistive technologies. For example, if your application is dependent on data from an external source, the code should be designed to accommodate the execution of multiple threads while the application waits for data.

How Accessible Is My Code?

Testing your application with the mouse disconnected provides a basic accessibility assessment. This will give immediate feedback as to how well the navigation schemes and mnemonics have been implemented. Accelerators and sequential tab navigation should also be tested during this phase.

The Java Accessibility Utilities can be used give a very detailed picture of your application's accessibility readiness. Explorer, Java Monitor, and Accessibility Monitor are the most crucial to analyzing code for accessibility readiness.

Explorer (soon to be Ferret)

Objects are analyzed by Explorer according to these criteria:

• following focus
• following the mouse
• following caret position,
• pressing F1 when pointer is over an object

Once an object has been selected for analysis, Explorer displays the results of calling Java Accessibility API methods on that object.

Java Monitor

The Java Monitor obtains information about objects being displayed on the screen by a JVM, and provides extended support for the Java Foundation Classes.

AccessibilityMonitor

The AccessibilityMonitor will track all accessible property change events on all non-transient objects in the virtual machine. In addition, if the user clicks the right mouse over the events presented in the AccessibilityMonitor table, a popup menu is presented containing options for manipulating the associated object.

Monkey

Monkey navigates the component trees in a JVM and presents the hierarchy in two different ways: first by the actual component hierarchy, and secondly by the hierarchy viewed as accessible objects. In addition, if the user clicks the right mouse button over a tree node, Monkey will present a popup menu containing options for manipulating the object.

Linker

Linker captures the Accessible hypertext information contained in the object beneath the pointer (user presses the F1 key), and displays a table detailing the Accessible hyperlinks within that object. Selecting one of the links and clicking "Activate Selected Link" will cause the hypertext object to follow the selected link.

Building Bridges

Java programming language applications are hosted on a wide variety of operating systems (for example, UNIX, Windows, and Mac.) Most currently available assistive technologies are hosted on Windows 95 or 98 platforms. To provide access to programs written in Java, these existing technologies need a bridge between their native environment and the Java Accessibility APIs within the JVM. This bridge will be different for each unique platform, due to the fact that half lives in the JVM and the other half lives in the native platform. Bridge code is currently under development for Win32, and Sun intends to develop bridges for the other major operating platforms over time.

The Promise of the Jini Technology

Jini technology provides spontaneous network computing. Printers, cell phones, even microwave ovens and clocks could be networked under the Jini technology umbrella. Then when you plug a new product into the network, a driver inside the device allows your computer to discover it. Interconnect is seamless and automatic.

In the realm of accessibility, the Jini technology has some very exciting ramifications. It rounds out the promise of Project Swing based accessibility by allowing persons who cannot interact directly with interfaces to carry technology that facilitates this interaction. Suppose a severely disabled person approaches an elevator in a wheelchair. Say the person is not able to move his or her arms. If the elevator is equiped with Jini technology and the wheelchair is also equiped with Jini technology, then the elevator could announce itself. The wheelchair would then discover the elevator and display options in a manner appropriate for its owner, such as audio output and voice-actuated input.

This is the vision for the future of Java accessibility as computing power moves out of the desktop box and into consumer devices everywhere. Computers will know about our individual characteristics and adjust themselves accordingly.

The Importance of Being JFC Project Swing

By using the Java 2 Project Swing components for applications development, the foundation for this future scenario is set in motion. The Project Swing look & feel is extremely flexible, and can even be reset at run time, without closing the current application. In the future, if Dena Shumila put her Java ring into someone else's computer, the system might reconfigure itself on-the-fly for the audio look & feel that she likes best. This concept has implications for everyone, as computers become more and more sensitive to our needs as unique individuals.

Online References

Overview of Java Accessibility API Features

Java Accessibility API FAQ

Sun Technology and Research — Enabling Technologies

Synopsis of the Java Accessibility API Interfaces

Java Foundation Classes APIs, version 1.1
(containing the Java Accessibility API)

Java Developers Kit (JDK) software, version 1.2
(containing the Java Accessibility API)

Java Accessibility Utilities (download)

Guidelines for Writing Accessible Applications in the Java Programming Language

Accessing the Java Accessibility API with IBM's Self Voicing Kit

Henter-Joyce's JAWS for Windows Screen Reader supporting the Java Accessibility API

Reference Texts

• The Java Tutorial by Mary Campione and Kathy Walrath
• Java Swing by Robert Eckstein, Marc Loy, and Dave Wood
• Java 2 Platform by Laura Lemay and Rogers Cadenhead
• Using Java 1.2 by Mike Morgan
• Java Foundation Classes by Matthew T. Nelson
• Programming with JFC by Scott R. Weiner and Stephen Asbury

The Sun Accessibility team

access@sun.com

_______
¹ As used on this web site, the terms "Java virtual machine" or "JVM" mean a virtual machine for the Java platform.

About the Author

Michael Meloan began his professional career writing IBM mainframe and DEC PDP-11 assembly languages. He went on to code in PL/I, APL and C. In addition, his fiction has appeared in WIRED, BUZZ, Chic, L.A. Weekly, and on National Public Radio.