In this paper, the authors discuss the challenges of context-aware systems in situations where it is difficult to sense context. Although many have argued for systems that ask for human input in uncertain situations, others believe systems should act on users' behalf. This paper presents an evaluation of a design solution that provides users with context-aware information and informs users of the uncertainty of that information.
The studies conducted by the authors evaluate how providing users with the level of uncertainty of the contextual information affects users' behaviors. The evaluation involves two experiments, one conducted in an "idealistic desktop-setting" and the other in a more realistic ubicomp environment. The first study used a 4-factorial analysis in order to determine which issues where important to test in the more realistic setting that employed a 2-factorial analysis.
The first experiment asked participants to remember numbers from a list. To help participants recall the numbers, participants received tips and from a PC and information about the certainty of the tips. To study the effects of providing users with information about the certainty of information, the study varied:
- the task difficulty (the length of time participants had to view the numbers),
- the cost (points earned/deducted for correct/incorrect responses)
- knowledge of uncertainty (displaying uncertainty vs. not displaying it)
- level of uncertainty (varying the quality of the tip)
Based on the study, the authors found that displaying uncertainty resulted in a significant increase in correct answers when high quality tips were shown. The benefit was more pronounced when participants had less time to view the initial set of numbers.
The second study provided participants with a list of seven numbers that they were asked to remember. Based on their memory, users were asked to pack senor devices into a cardboard box. Participants were again given tips via a PC and informed that the system could sense the numbers packed and suggest the next number to be included.
The two factors varied in this study included:
- knowledge of uncertainty (displaying uncertainty vs. not displaying it)
- level of uncertainty (varying the quality of the tip)
Based on the study, the results from the first experiment were replicated, thus the authors conclude the effectiveness of displaying the level of uncertainty.
The authors also conclude that this study provides an example for others who wish to conduct evaluations of ubiquitous applications, by conducting an initial study in a controlled environment in order to determine which aspects to study in a more realistic setting.
|
