This paper presents the study of 36 participants with a location-aware map of a town in Finland . The map and context-sensitive advertisements were displayed on a PDA for visitors to use to navigate the city. The authors evaluated the system over the course of a month using cameras on mobile phones to record user interactions with the system.
The authors exclaim that questionnaires and user interviews did not provide the design team with sufficient information, whereas, "shadowing" users with video cameras to observe their usage of the system was too disruptive of users and too difficult to conduct on a large scale. In one of the first studies of this system, researchers followed participants and videotaped their actions with a portable video camera. Because of the public nature of the device, this technique was not only disruptive, but it made recruiting participants difficult.
The authors also attempted to study usage of the system by asking participants to photograph their interactions with the system and later provide notes about the photos in a diary. Although many of the notes were useful, the photos did not provide the detail the authors were hoping for, as many depicted users starring at a PDA screen. In addition, the authors found that taking notes in a mobile situation prevented many from contributing detailed descriptions.
When using questionnaires, the authors found that pen and paper questionnaires did not work well in a mobile environment, and a questionnaire on the PDA required users to be familiar with a stylus.
Therefore, the team decided to use a new approach, which they call the "experience clip". Using this technique, the authors provided pairs of participants with two devices, the PDA (with the location-aware map installed) and a mobile phone equipped with the ability to record video clips. While using the system, one individual was instructed to use the guide, while the other was to record the actions with the mobile phone camera. Participants were also asked to complete a user profile.
The authors' goal in defining this technique, include:
- Reducing the influence of a researcher in the situation,
- Giving users freedom to explore the system,
- Supporting mobile usage situations, and
- Not altering the environment in a way that would affect the user experience.
The authors felt those mobile phones are now commonplace and their usage would not draw any extra attention to participants.
Based on their experience, the authors found that it was necessary to provide users with instructions to explain the type of material to capture, as well as to teach participants how to use the mobile phone cameras. The authors also found that they received a much wider range of usage situations and emotional responses with the experience clip than when a researcher was present (possibly due to the fact that users were more comfortable trying new things around their friends). Furthermore, when the researcher was present, users frequently turned to the researcher for help. Whereas, with the experience clip, researchers were able to see how users handled problems in a natural environment.
After reviewing the clips, the authors believe that the experience clip might also be a valid participatory design technique, in addition to an evaluation methodology.
|
