We have been liberated from the desktop. As mobile and embedded computing systems increasingly pervade environments, more and more information and content is available throughout our daily surroundings. Sometimes we proactively seek that information -- via our laptops or MP3 players or smart phones. Other times, the information is offered automatically, through GPS systems, for example, or sensor-triggered signage.

These pervasive appliances and applications offer new sets of technical and business challenges -- along with ubiquitous and fascinating opportunities for innovation, especially in the areas of managing information overload, making computing environments transparent to users, and enabling mobile and ambient devices to serve as intelligent interfaces to our physical surroundings.

These days, people increasingly try to multitask by juggling the simultaneous use of communications and PDA devices, entertainment gear, home appliances, and even transportation systems. Yet these entities generally do not interact in meaningful ways with each other for the purpose of responding to the user's tacit needs. The performance of high-tech gear is a specific reaction to the user's precise input commands, delivering or referencing the content accessible by only that particular device.

However, when we consider the opportunities to equip these systems with sensing capabilities, effortless user interaction, and interconnectivity capabilities, we can envision ways in which they can automatically perform for us within the context of the activities in which we are engaged and the behavior that characterizes our unique personal needs.

By observing the physical environment in which the system is in use, and by recording and analyzing user behavior, intent-aware and context-aware systems can detect people's intentions without requiring complex, real time, manual interactions. A system that can infer a user's intention can deliver information in anticipation of the user's expressed need for that material -- able to proactively deliver services and information to transform the ways in which we live, play, and work.

As sensors become less expensive and easier to connect to the Internet, they can give computing technologies eyes and ears to make sense of the physical world. Orwellian as that may sound, myriad benefits appear possible: from the simple convenience of letting devices handle mundane tasks (media playback, shopping) to the availability of applications that assist mentally and physically challenged people … that enable more efficient consumption of energy resources … that coordinate safety and rescue efforts … or that provide tighter physical and virtual security.

Creating Opportunities with Emerging Technologies

How might an individual's activities establish a context for targeted, responsive, promotional messages? PARC is exploring responsive media technologies as a platform for modeling sequential interactions and behaviors between buyers and sellers. Responsive media use computer-initiated interactions with retail shoppers to attract, detect, and maintain customer engagement with products or messages. Content segments play at specific points in the standard cycle of interactions between buyers and sellers; the system infers and predicts the shopper's intentions.

To explore opportunities for responsive media in the buying and selling of clothes, and also to explore potential responsive media design requirements in the bricks and mortar retail setting, PARC recently designed and built a complete prototype installation, the Responsive Mirror, an interactive, fashion recommendation system.

The Responsive Mirror is an implicitly controlled video technology for clothes fitting rooms. Watching his reflection in an apparently conventional mirror, a shopper may directly compare a garment he's trying on adjacent to an image display of photos of himself wearing the garments he previously tried on. In both his live reflection and the photographs of his previous outfits, his physical poses match. The system also allows live, side-by-side comparison to fashions popular with friends in the shopper's social network. PARC's user study elicited a number of design challenges and tradeoffs that designers of such technologies must consider regarding privacy, adoption, benefits to shoppers and merchants and user behaviors in fitting rooms.

When a person interacts with a fitting room mirror, he provides implicit cues about the information he seeks. When he turns his body, he is observing how the clothes look from a different angle. Sensors can be used to detect this implicit information-seeking behavior, and supplemental information can be provided. Shoppers need not be taught how to use the mirror -- they need to do nothing but behave naturally for it to work.

Kyoko is a 22-year-old woman shopping for blouses in a small clothing store. She browses the rack of recent arrivals for tops in her size, takes a selection into the changing room, and puts them on, one at a time. Each time, she exits the changing room and stands in front of the mirror, turning to view the blouse from multiple angles.

Kyoko notices the two, large, flat-screen displays, one on either side of the mirror (although she doesn't immediately notice the small video camera mounted atop the mirror, aimed directly at her). One screen shows Kyoko wearing each previously worn blouse. As Kyoko turns to view her blouse from multiple angles, the changing images of her in the previous blouse match her poses as she moves around to compare the fit and style from each angle of view: front, left, right, and back.

The results of this research contribute to our understanding of the potential impact of ubiquitous computing technology in the domain of physical shopping.

In another project, inspired by the popularity of online advertising based on the context of users typing keywords into a search engine, our scientists have noted that targeting mechanisms for ubiquitous computing environments lack analogous approaches. Moreover, none of the current, interactive electronic advertising systems explicitly models a consumer's abstract activity.

In this study, PARC has applied activity detection to targeted advertising. To research data collection issues, behavior pattern-capture, product requirements, and potential market value, we developed the Proactive Experience Sampling Tool. We deployed mobile phones to monitor subjects engaged in certain activities to enable the presentation of messages at the time they would make maximum impact. We defined and evaluated a proposed architecture and its implementation, using two mechanisms to link activity descriptions with ad content: direct keyword matching using an online advertising service, and "human computation" matching, which enhances keyword matching with help from online workers. Early research findings show that people consider advertisements related to certain types of current activity to be more relevant and more useful than nontargeted advertisements.

In the near future, activity detection will achieve sufficient accuracy rates to enable many new applications, including highly targeted advertising. Combined with location-based service delivery, PARC envisions a rapidly growing market with many avenues of opportunity.

Creating Location-based, Mobile Recommender Services

PARC's activity detection technologies have been integrated within a mobile recommender system codenamed "Magitti," in collaboration with Japan-based Dai Nippon Printing Company Ltd. DNP is one of the world's largest business service and commercial printing companies. A team of PARC social scientists, computer scientists, cognitive scientists, and linguistic scientists joined forces with DNP to collaborate in the identification of a new market and the invention of new media applications for handheld phones equipped with GPS and touch-screens.

Our work with DNP focused on developing mobile techniques and systems to find and filter leisure activities based on the device owner's personal interest and preferences, geographical location, and relevance of data about vendors and their offerings, hours and pricing.

PARC conducted initial analysis and subsequent development and in situ evaluation of a conceptual prototype in Japan. Feedback from prospective users has enabled PARC to further develop the Magitti prototype into a unique technology concept with an exceptionally easy-to-use, one-handed interface. The compelling platform portends a promising future for contextual service delivery.

This summary was adapted from a longer white paper by PARC. To read the entire article,
including descriptions and examples of different classes of applications, click here.

More detailed information about the Responsive Mirror and the technology and social science
behind it also is available at the PARC web site.

READER COMMENTS

Post a comment

Hide comment form

 | 

Read all comments (0)

Read recent comments

---

Permission to post your comment is assumed when you submit it. The name you provide will be used to identify your post, and NOT your e-mail address. We reserve the right to excerpt or edit for clarity any posted comments.
View our full privacy policy

Name:

E-mail:

Comment:

MORE: Kiplinger Recommends | Technology

TOOLS:   E-Mail Article  |  Print Article

GET: Email Alerts on Your Favorite Business Topics

SAVE, SHARE & DISCUSS:    |   |   |   |   |   |   |   |    

ADD HEADLINES: