The paper presents a framework for the iterative design and evaluation of ubicomp environments. The authors stress several principles and suggest that evaluators need to gather both qualitative and quantitative data from users in their real environments. They also stress the importance of establishing a baseline assessment of the environment in order to measure the value of subsequent enhancements and modifications. This paper presents the strengths and weaknesses of several user study techniques based on an ubicomp evaluation of Labscape.
In designing Labscape, the development team used two methods to gain qualitative data and input on user needs, tasks and environment, including intensive interviewing and contextual field research.
In evaluating Labscape, the authors rule out evaluation methods, such as usability testing. The paper suggests that usability testing is a contrived situation with artificial tasks which do not represent use of ubicomp applications in the everyday, physical world. Instead, the development team chose to use evaluation techniques from psychology and anthropology such as Lag Sequential Analysis (LSA). The benefit of LSA is that it provides quantitative measures users of performing tasks in their real environment and can provide a baseline of the environment before the ubicomp application is introduced. However, it can be a time-consuming and expensive process.
The authors stress the importance of applying well-established user study techniques that involve representative users in their natural environment. They believe that it is important to gather both qualitative and quantitative data from real users in an authentic setting and stress the importance of establishing a baseline.
|
