Research Areas
The HCIL has conducted a broad range of research over the years.
Below is a SAMPLE of current or recent projects organized by topics.
Included as well are links to specific labs working on those topics.
For older projects see the Research Project Archive
Accessibility
Trace Center
The Trace Research & Development Center is a pioneer in the field of technology and disability, and is known for high-impact research and development. Our mission is to capitalize on the potential that technologies hold for people experiencing barriers due to disability, aging, or digital literacy, and to prevent emerging technologies from creating new barriers for these individuals. In doing this, we bring together disciplines such as information science, computer science, engineering, disability studies, law, and public policy. We engage in research, development, tech transfer, education, policy, and advocacy.
Clustering User Interactions in Outdoor Orientation Apps
We analyze the behavior of people who are blind, during their interaction with mobile technologies that support orientation and mobility. By representing a single blind user of a mobile navigation application as a stream of remotely collected interaction logs, we uncover underlying behavioral patterns and understand application usage. We propose a scalable solution for identifying naturally formed clusters from common interaction patterns of thousands of blind users and extracting semantic meanings for the clusters.
Intelligent Personal Assistants
For accessibility, voice-controlled, home-based intelligent personal assistants (IPAs) have the potential to greatly expand speech interaction beyond dictation and screenreader output. This research examines the accessibility of off-the-shelf IPAs (e.g., Amazon Echo) through two exploratory studies. Findings indicate preference of using IPAs over traditional computing devices for digital technology access. We identify design implications to better support use of IPAs. Both studies highlight challenges related to discoverability and the need for richer voice-based applications on IPAs.
Personal Object Recognizer
Many blind users benefit from technologies to support their movement through unfamiliar indoor or outdoor environments, including navigation to destinations and their awareness of the surroundings. We show that blind people can train a mobile application with only a few snapshots of their objects of interest (e.g. everyday items on their home). We use transfer learning to adapt a deep learning network trained for generic image recognition for user-specific recognition tasks given a few examples.
Project Sidewalk
Project Sidewalk links big data, machine learning, computer vision, and geographic information systems to help map and visualize D.C’s accessibility for people with mobile impairments. The tool allows volunteers to tag and label interactive images from GoogleStreet View for accessibility issues; the absence of curb ramps, narrow or blocked pathways and cracks in sidewalks and roads.
Technology Use and Dementia
Though a dominant view of dementia sees it as a condition that erases individuality and the ability to engage in activity, other ways of viewing dementia take into account the importance of the social context, abilities that persist long into dementia, and stigma that isolates and marginalizes people with this condition. This work explores the use of technology to challenge predominant conceptions of dementia and to support activities such as sharing online.
Citizen Science
NatureNet
NatureNet (NN) is an existing multi-touch technology platform that uses mobile devices and a website to enable participants to develop and carry out projects that support their own informal learning and foster positive scientific dispositions.
StreamBED VR
StreamBED VR is a a virtual reality training tool that teaches qualitative stream monitoring to citizen scientists. The goal of the system is to create an experience that guides novice water monitors to focus on key areas of the stream and so that they understand and evaluate streams relative to other similar streams spaces using visual, auditory and other sensory cues. This work contributes to research in the domains of citizen science, VR education, and multisensory design.
Communities
Crowd Discussions and Hidden Profile Paradigm
Online communities, from question and answer sites to general-purpose discussion forums, are increasingly working to solve hard problems together, often through a process of open sharing and discussion of ideas and information. “Wisdom of the crowd” arguments suggest that, in theory, crowds have the capacity to bring together diverse expertise, pooling distributed knowledge and thereby solving challenging and complex problems. In several ongoing projects we are testing this theory using the hidden profile paradigm, a set of problems used in small group research to evaluate whether groups pool distributed facts.
Science Everywhere
Science Everywhere is an NSF funded research study aimed at understanding how technology can engage entire communities in science learning. We utilize a design-based research approach in which we co-design innovative science learning technology with families, teachers, and leaders in a community, implement that technology in the community, and then redesign that technology in an iterative design process. Broadly, this study will contribute to theory on connected learning by developing an understanding of how to connect science learning at home, school, and community spaces with technology. This study also aims to contribute to our understanding of parent-child learning, interactive display design, and social media for learning.
Sensemaking technologies for collaboration at scale
In many collective creativity systems, the number of contributors can number in the thousands to tens of thousands, making true collaboration is a difficult challenge. Without effective mechanisms for surfacing key insights and sharing them among collaborators, many collective creativity efforts devolve into largely independent work. This often results in a preponderance of redundant, shallow, and bad ideas. We want to know: how might we build sensemaking technologies that support effective collaboration at scale? In collaboration with researchers in machine learning, we explore what kinds of sensemaking strategies are useful for creativity, what representational requirements are necessary for supporting those kinds of sensemaking (e.g., relational knowledge for analogy), and techniques from crowdsourcing that can support or work in combination with novel machine learning systems.
Design Process
Children as Design Partners (Intergenerational Design Teams)
Children, ages 7-11, work with HCIL faculty/staff/students after school and over the summer to create new technolgies for children.
Creatively reusing (old) knowledge
People build new ideas on what they know and have seen. Sometimes this a good thing; sometimes it kills creativity. We want to know: are there scientific principles that can guide creators’ interactions with sources of inspiration so that they are inspired and not hindered in their creativity? This is an old, fundamental question in the cognitive science of creativity, but has taken on a fresh significance in today’s information age, where creators can be exposed to many, many potential sources of inspiration online (e.g., Google Scholar, US Patent Database). Computational inspiration systems (such as search engines and recommender systems) can help by directing users’ attention to what is most inspirational. But what is most inspirational? (How) can we predict this beforehand?
Crowd civic systems
Classic work in design studies/research recognized policymaking as a prototypical kind of design: planning and taking actions to turn undesirable states into desirable ones. While much work has focused on how information technologies can augment the design process in other traditional domains like engineering, product, and service design, we are only just beginning to explore the potential of information technologies to augment the design of policy. Could there be computer-aided policymaking? What might that look like? Open government data, smart cities, and participatory budgeting have begun to scratch the surface of what designing policy could look like in the digital age, but much remains to be learned. We want to know: what new/better forms of governance and civic participation are possible in the digital age?atory budgeting have begun to scratch the surface of what designing policy could look like in the digital age, but much remains to be learned. We want to know: what new/better forms of governance and civic participation are possible in the digital age?
Digital Humanities
Informal STEM Learning Through Alternate Reality Games
Our Alternate Reality Games (ARGs) represent an effort to bring underrepresented populations into STEM fields by engaging them in the uniquely immersive gameplay of ARGs. These games ask players to incorporate and imagine gameplay as it takes place in their day-to-day lives, encouraging them to co-construct the fictional storyline as they play along, and to collaborate with hundreds or thousands of other players dedicated to the same tasks. Our primary audience is teenagers (13-17 years old), including those currently underrepresented in STEM (e.g., females, blacks, and hispanics). Our ARGs inspire players to take on the roles of real life scientists, programmers, engineers, technicians, artists, writers, and explorers as they solve challenges and immerse themselves in the storyworld. This project represents a multidisciplinary partnership with the National Science Foundation (award number 1323306), as well as Brigham Young University, NASA, Tinder Transmedia, Intuitive Company, and the Computer History Museum. Our ARGs bring academic researchers, writers, artists, scientists, transmedia producers, and players together as part of a larger design team.
Digital Libraries
Sensemaking technologies for collaboration at scale
In many collective creativity systems, the number of contributors can number in the thousands to tens of thousands, making true collaboration is a difficult challenge. Without effective mechanisms for surfacing key insights and sharing them among collaborators, many collective creativity efforts devolve into largely independent work. This often results in a preponderance of redundant, shallow, and bad ideas. We want to know: how might we build sensemaking technologies that support effective collaboration at scale? In collaboration with researchers in machine learning, we explore what kinds of sensemaking strategies are useful for creativity, what representational requirements are necessary for supporting those kinds of sensemaking (e.g., relational knowledge for analogy), and techniques from crowdsourcing that can support or work in combination with novel machine learning systems.
Keshif: Simplicity Driven Visual Faceted Browser
Keshif is a visual data browser that makes it easier to browse and understand your data. It presents visual summaries of your data properties, such as who, what, when and where, in its facets and timeline.
International Children’s Digital Library
A library that provides free access to children’s books from around the world. By ensuring access to books from many cultures and in diverse languages, we foster a love of reading, a readiness to learn, and a response to the challenges of world illiteracy.
Digital-Physical Interaction Design
BodyVis
A new wearable computing and electronic textiles (e-textiles) technology for learning about anatomy and physiology, and for supporting children’s scientific inquiry skills. (NSF-funded) (PI Jon Foehlich)
Technology Use and Dementia
Though a dominant view of dementia sees it as a condition that erases individuality and the ability to engage in activity, other ways of viewing dementia take into account the importance of the social context, abilities that persist long into dementia, and stigma that isolates and marginalizes people with this condition. This work explores the use of technology to challenge predominant conceptions of dementia and to support activities such as sharing online.
Education
BodyVis
A new wearable computing and electronic textiles (e-textiles) technology for learning about anatomy and physiology, and for supporting children’s scientific inquiry skills. (NSF-funded) (PI Jon Foehlich)
Children as Design Partners (Intergenerational Design Teams)
Children, ages 7-11, work with HCIL faculty/staff/students after school and over the summer to create new technolgies for children.
Enhancing Group Communication
Small groups rarely work together optimally. We developed a real-time language feedback system. It monitors communication patterns among students in discussion groups and provides real-time instructions to shape the way the group works together.
Informal STEM Learning Through Alternate Reality Games
Our Alternate Reality Games (ARGs) represent an effort to bring underrepresented populations into STEM fields by engaging them in the uniquely immersive gameplay of ARGs. These games ask players to incorporate and imagine gameplay as it takes place in their day-to-day lives, encouraging them to co-construct the fictional storyline as they play along, and to collaborate with hundreds or thousands of other players dedicated to the same tasks. Our primary audience is teenagers (13-17 years old), including those currently underrepresented in STEM (e.g., females, blacks, and hispanics). Our ARGs inspire players to take on the roles of real life scientists, programmers, engineers, technicians, artists, writers, and explorers as they solve challenges and immerse themselves in the storyworld. This project represents a multidisciplinary partnership with the National Science Foundation (award number 1323306), as well as Brigham Young University, NASA, Tinder Transmedia, Intuitive Company, and the Computer History Museum. Our ARGs bring academic researchers, writers, artists, scientists, transmedia producers, and players together as part of a larger design team.
International Children’s Digital Library
A library that provides free access to children’s books from around the world. By ensuring access to books from many cultures and in diverse languages, we foster a love of reading, a readiness to learn, and a response to the challenges of world illiteracy.
Science Everywhere
Science Everywhere is an NSF funded research study aimed at understanding how technology can engage entire communities in science learning. We utilize a design-based research approach in which we co-design innovative science learning technology with families, teachers, and leaders in a community, implement that technology in the community, and then redesign that technology in an iterative design process. Broadly, this study will contribute to theory on connected learning by developing an understanding of how to connect science learning at home, school, and community spaces with technology. This study also aims to contribute to our understanding of parent-child learning, interactive display design, and social media for learning.
Scratch Encore
This research collaboration with the University of Chicago and the Chicago Public School district seeks to answer the following question: Can we create advanced upper elementary Computer Science instructional materials that give equal value to improving equity and student learning outcomes? This research practitioner partnership is designing, developing, and evaluating advanced Scratch-based CS instructional materials for upper elementary students through a process that attends to practical barriers to equity.
StreamBED VR
StreamBED VR is a a virtual reality training tool that teaches qualitative stream monitoring to citizen scientists. The goal of the system is to create an experience that guides novice water monitors to focus on key areas of the stream and so that they understand and evaluate streams relative to other similar streams spaces using visual, auditory and other sensory cues. This work contributes to research in the domains of citizen science, VR education, and multisensory design.
Understanding the Design of Introductory Programming Environments
Blocks-based programming environments are growing in popularity and are increasingly being used in formal introductory programming contexts. To date, much of the work evaluating such tools has focused on their effectiveness in out-of-school contexts and emphasized engagement and attitudinal measures over content mastery. Given their growing presence in classrooms, it is important to understand the benefits and drawbacks to the use of the blocks-based programming approach in formal learning contexts. This project seeks to understand the affordances and drawbacks of block-based programming relative to conventional text-based languages.
Human-Centered Artificial Intelligence
Human-Centered AI – Overview
This project presents a Human-Centered AI (HCAI) framework, which bridges the gap between ethics and practice. It offers specific recommendations for designing successful technologies that augment, amplify, empower, and enhance humans rather than replace them. This project has four papers, several videos, and resources that present fresh approaches to design for successful technologies that are based on high levels of human control AND high levels of automation.
Workshop on Reliable, Safe & Trustworthy AI
This workshop explores case studies, guidelines, principles, and theories to design advanced technologies that bring the benefits of AI methods, while ensuring appropriate human control. Website includes abstracts, bios, and videos from the 11 speakers. Well-designed technologies, which offer high levels of human control and high levels of computer automation, will increase human performance, rather than replace people. These Human-Centered AI technologies are more likely to produce designs that are Trusted, Reliable & Safe (TRS). Achieving these goals will dramatically increase human performance, while supporting human self-efficacy, mastery, and responsibility.
History of the field of HCI
Main faculty contact: Catherine Plaisant
Historical CHI Video Project
Working with ACM SIGCHI, we are digitizing and archiving historical videos from the CHI Conference, the premier international conference for the field of Human-Computer Interaction. 20 years and 322 video segments are being preserved.
HCIL Early Research Histories
Summaries of the pioneering work of HCIL in hypertext, touchscreens, search interfaces, photo browsing and annotation, etc.
HCIL Archive
Historical technical reports, videos and more
Innovation, Discovery, and Creativity
Colloquium on Revisiting Cybernetic Serendipity
Our ambition is to redirect the history of ideas, restoring the Leonardo-like close linkage between art/design and science/engineering/medicine. We believe that internet-enabled collaborations can make more people more creative more of the time. This colloquium used the historical framework of Cybernetic Serendipity to look at how the context has changed, and how creativity and collaboration are impacting practice and research today. How should we re-envision research policy and educational approaches to maximize the impact of partnerships with design, art, and humanities? How can we productively engage business, government, and non-governmental organizations as research and educational partners?
Creatively reusing (old) knowledge
People build new ideas on what they know and have seen. Sometimes this a good thing; sometimes it kills creativity. We want to know: are there scientific principles that can guide creators’ interactions with sources of inspiration so that they are inspired and not hindered in their creativity? This is an old, fundamental question in the cognitive science of creativity, but has taken on a fresh significance in today’s information age, where creators can be exposed to many, many potential sources of inspiration online (e.g., Google Scholar, US Patent Database). Computational inspiration systems (such as search engines and recommender systems) can help by directing users’ attention to what is most inspirational. But what is most inspirational? (How) can we predict this beforehand?
Open Data
Open Collaboration and Data Science
The democratization of data science and open data initiatives have inspired groups from civic hackers to data journalists to use data to address social issues as well as open innovation contest platforms like Kaggle. Our ongoing work investigates ways people are collaborating to analyze open data using data science and big data techniques.
Privacy
Improving Developers’ Privacy and Security Decisionmaking
Well-intentioned software developers can put user data at risk when they make errors with complicated encryption protocols, abuse device identifiers and location data, or make other mistakes. In this work, we examine why these errors occur and how they can be prevented.
LSP2 Lab – Security Privacy People
In SP2, we research how security and privacy affect people, and how people affect security and privacy. We use methods and tools from computer science and social science to investigate how and why people make security- and privacy-relevant decisions, and to give people tools to make better decisions.
NetCHI Lab
We study people and networks and create tools to help people with their networks. We are working to facilitate an efficient, secure, and affordable Internet for all by studying how users get online, creating broadband tools, and having fun in the process.
Security
LSP2 Lab – Security Privacy People
In SP2, we research how security and privacy affect people, and how people affect security and privacy. We use methods and tools from computer science and social science to investigate how and why people make security- and privacy-relevant decisions, and to give people tools to make better decisions.
Improving Developers’ Privacy and Security Decisionmaking
Well-intentioned software developers can put user data at risk when they make errors with complicated encryption protocols, abuse device identifiers and location data, or make other mistakes. In this work, we examine why these errors occur and how they can be prevented.
Keeping Kids Safe
(with Marshini Chetty at Princeton, Jessica Vitak, and Ph.d. student Priya Kumar)
NetCHI Lab
We study people and networks and create tools to help people with their networks. We are working to facilitate an efficient, secure, and affordable Internet for all by studying how users get online, creating broadband tools, and having fun in the process.
Understanding Security Advice Sources to Improve User Safety
Users selectively choose which security advice to accept and some (most) to reject; however, it is unclear whether they are effectively prioritizing what is most important or most useful. If we can understand from where and why users take security advice, we can develop more effective security interventions.
Social Media & Networks
Informal STEM Learning Through Alternate Reality Games
Our Alternate Reality Games (ARGs) represent an effort to bring underrepresented populations into STEM fields by engaging them in the uniquely immersive gameplay of ARGs. These games ask players to incorporate and imagine gameplay as it takes place in their day-to-day lives, encouraging them to co-construct the fictional storyline as they play along, and to collaborate with hundreds or thousands of other players dedicated to the same tasks. Our primary audience is teenagers (13-17 years old), including those currently underrepresented in STEM (e.g., females, blacks, and hispanics). Our ARGs inspire players to take on the roles of real life scientists, programmers, engineers, technicians, artists, writers, and explorers as they solve challenges and immerse themselves in the storyworld. This project represents a multidisciplinary partnership with the National Science Foundation (award number 1323306), as well as Brigham Young University, NASA, Tinder Transmedia, Intuitive Company, and the Computer History Museum. Our ARGs bring academic researchers, writers, artists, scientists, transmedia producers, and players together as part of a larger design team.
Older Activists and Ageism
Broad societal views, such as stereotypes based on age, become embedded in and perpetuated through the design of technologies. This project looks at the ways that older adults’ preferences for technology challenge some of the assumptions that designers and researchers make about older people, particularly in regard to conceptions around social isolation and physical and cognitive decline. The work also examines how older adults use technology platforms such as blogs and social media to respond to ageism.
Visualization
Main faculty contact: Niklas Elmqvist
Sample of projects:
FIGX Lab
Fumeng Yang joined the lab and the University of Maryland in 2024. Check out her projects as well.
Ubiquitous Analytics
Niklas Elmqvist project investigates how to support manipulating, thinking about, and analyzing data anytime, anywhere.
Event Analytics
Ben Shneiderman and Catherine Plaisant are now Emeriti members of HCIL. They last projects propelled the field of Event Analytics with a series of projects:
- EventFlow: a novel tool for event analytics to summarize and search temporal patterns.
- CoCo: A Visual Analytics Tool for Comparing Cohorts of Event Sequences
- EventAction: Visual Analytics for Temporal Event Sequence Recommendation
Keshif: Simplicity Driven Visual Faceted Browser
Keshif is a visual data browser that makes it easier to browse and understand your data. It presents visual summaries of your data properties, such as who, what, when and where, in its facets and timeline.
PAOHVIS: Dynamic Hypergraphs Visualization (in collaboration with INRIA France)
A novel technique to visualize dynamic hypergraphs. Easy to learn and well suited for medium size dynamic networks such as those commonly generated by digital humanities projects – our driving application domain.
Interactive data-intensive systems
Leilani Battle‘s current research is anchored in the field of databases, but utilizes research methodology and techniques from HCI and visualization to integrate data processing (databases) with interactive interfaces.
Leo Zhicheng Liu recently joined the lab and the University of Maryland. Check out his projects as well.
For all older projects see the Research Project Archive
