Zhang, P., Plettenberg, L., Klavans, J., Oard, D., Soergel, D.
This paper describes a formative evaluation of an integrated multi-lingual, multimedia information system, a series of user studies designed to guide system development. The system includes automatic speech recognition for English, Chinese, and Arabic, automatic translation from Chinese and Arabic into English, and query-based and profile-based search options. The study design emphasizes repeated evaluation with the same (increasingly ex-perienced) participants, exploration of alternative task designs, rich qualitative and quantitative data collection, and rapid analysis to provide the timely feedback needed to support iterative and responsive development. Results indicate that users presented with materials in a language that they do not know can generate remarkably useful work products, but that integration of transcrip-tion, translation, search and profile management poses challenges that would be less evident were each technology to be evaluated in isolation.
The HCIL has a long, rich history of transforming the experience people have with new technologies. From understanding user needs, to developing and evaluating these technologies, the lab's faculty, staff, and students have been leading the way in HCI research and teaching.