Just Say "Britney Spears": Multi-Modal Search and On-Device Portals

Print-friendly Version

This article discusses the importance of multi-modality in a search-driven On-Device Portal (ODP), demonstrates multi-modal search in a Java Platform, Micro Edition (Java ME) ODP, and summarizes solutions to technical challenges in developing the Java ME application.

(Note: "Britney Spears" is a common search query in ODPs that allow users to search, browse, and discover popular music.)

An On-Device Portal is a handset-resident mobile application, often preloaded, that enhances the discovery and consumption of endorsed mobile content, services, and applications.

An ODP application is provided for free by a mobile operator or publisher for the purpose of allowing users to search, browse, discover, and ultimately purchase digital content. Here, digital content could mean traditional walled-garden consumables like ringtones, music, wallpapers, videos, games, and applications, or emerging Mobile Web content from preferred or long-tail publishers.

An ODP aims to provide a more compelling experience than is possible in a mobile browser. An ODP can deliver personalized content recommendations. It can be monetized through mobile advertising. Consumers often buy content using ODPs with a greater adoption rate than traditional mobile storefronts.

Mass distribution is a tenet of ODP development, making Java ME a popular platform choice. Java ME allows the ODP developer to create a rich and consistent user experience across the landscape of mass-market featurephones and smartphones.

Figure 1. Browsing Games in Java ME ODP

Figure 2. Search Results for "obama" in Java ME ODP

Multi-modality is a key component of an addictive ODP user experience. Multi-modality allows the user to search, browse, and discover mobile content using several interaction methods. Consumers navigate a search-driven ODP application by searching with text, voice, and camera image. Search-driven ODPs generally capture all types of consumer search queries in a single searchbox. Users browse content by clicking links.

The screenshot below is an example in a Java ME ODP application of a "predictionary" - a personalized list of suggested query terms based on partially-entered text and knowledge of your previous user activity.

Figure 3. Predictionary of Search Suggestions in Java ME ODP

Multi-modality allows the consumer to use the ODP in many ways. Users search to find mobile content and services. Or, they can browse a curated collection of current offerings. Discovery of new and interesting mobile content is fostered in two ways. Advanced user interface elements, such as the left-right circular navigation at the bottom of the above screenshot, nurture curiosity and encourage the user to navigate between screens. Predictive analytics surface relevant content and service recommendations in the application screens. Ultimately, users click links to purchase the desired content.

Multi-modal search functionality drives ODP usage by encouraging the consumer to use familiar idioms on mobile phones. Text search queries are entered using the phone keypad. Users search by voice by speaking a phrase into the handset microphone. Camera images can be captured in the ODP application to enable searching by product image, barcode, or QR code.

Effective searching by voice on Java ME mobile devices requires consideration of three development challenges.

Sound Capture Methods
Java ME devices require JSR-135 with sound recording functionality to capture voice utterances. In addition, the handset must generate voice recordings using a low-bitrate codec such as AMR (adaptive multi-rate compression).

Capturing Sound in a Mobile Environment
The voice-to-text translation service must be capable of handling utterances from noisy mobile environments. Voice recordings can come from cars, bars, airports, and even popular music concerts.

Complex Voice Grammars
The voice-to-text translation service must be efficient at handling natural and unstructured voice utterances. It is insufficient for the service to understand only walled-garden grammars like song names, music artists, and titles of other digital content from a closed catalog. Web-enabled ODP applications must support searching by voice across free-text queries, mobile web domain names, and millions of music tracks. ODP developers should consider voice-to-text translation services using phoneME identification and incorporating user behavioral analysis.

On-Device Portals present two kinds of development challenges that are solved by using Java ME technology.

Operator and Publisher Integration
Because the purpose of an ODP is to surface digital content, the application must make handoffs to operator and publisher services to preview and purchase the content. These handoffs are accomplished by launching a microbrowser using HTTP 302 redirects or by launching a separate Java ME application using the Push Registry (MIDP 2.0) or Content Handler (JSR-211) APIs.

Network Efficiency
The ODP application should implement an efficient protocol for delivery of rich digital content. This can be achieved by implementing a compact binary protocol over HTTP between the Java ME application and the associated server. Network communication can be implemented asynchronously. Status updates can be piggybacked in the same transaction as a search request to maximize network efficiency.

Mobile operators and publishers use On-Device Portal applications to enhance the discovery and consumption of their mobile content, services and applications. A multi-modal and search-driven On-Device Portal in Java ME is an effective strategy for providing compelling mobile content and a consistent user experience to mass-market featurephones and smartphones.

• Java ME homepage
• Medio Systems

Gail Rahn Frederick is a mobile software architect at Medio Systems in Seattle. She leads a software team that leverages Java ME and other technologies to deliver mobile search and discovery products to operators and publishers.

We welcome your participation in our community. Please keep your comments civil and on point. You can optionally provide your email address to be notified of replies—your information is not used for any other purpose. By submitting a comment, you agree to these Terms of Use.