Research Areas

The HCIL has conducted a broad range of research over the years.
Below are RECENT projects.

For older projects see the RESEARCH ARCHIVE

Accessibility | Citizen Science | Communities | Design Process | Digital Humanities | Digital Libraries | Digital-Physical Interaction Design | Education | Innovation, Discovery, and Creativity | Open Data | Privacy | Public Access | Security | Social Media & Networks | Visualization

Accessibility

Clustering User Interactions in Outdoor Orientation Apps
We analyze the behavior of people who are blind, during their interaction with mobile technologies that support orientation and mobility. By representing a single blind user of a mobile navigation application as a stream of remotely collected interaction logs, we uncover underlying behavioral patterns and understand application usage. We propose a scalable solution for identifying naturally formed clusters from common interaction patterns of thousands of blind users and extracting semantic meanings for the clusters.

Intelligent Personal Assistants
For accessibility, voice-controlled, home-based intelligent personal assistants (IPAs) have the potential to greatly expand speech interaction beyond dictation and screenreader output. This research examines the accessibility of off-the-shelf IPAs (e.g., Amazon Echo) through two exploratory studies. Findings indicate preference of using IPAs over traditional computing devices for digital technology access. We identify design implications to better support use of IPAs. Both studies highlight challenges related to discoverability and the need for richer voice-based applications on IPAs.

Personal Object Recognizer
Many blind users benefit from technologies to support their movement through unfamiliar indoor or outdoor environments, including navigation to destinations and their awareness of the surroundings. We show that blind people can train a mobile application with only a few snapshots of their objects of interest (e.g. everyday items on their home). We use transfer learning to adapt a deep learning network trained for generic image recognition for user-specific recognition tasks given a few examples.

Project Sidewalk
Project Sidewalk links big data, machine learning, computer vision, and geographic information systems to help map and visualize D.C’s accessibility for people with mobile impairments. The tool allows volunteers to tag and label interactive images from GoogleStreet View for accessibility issues; the absence of curb ramps, narrow or blocked pathways and cracks in sidewalks and roads.

Technology Use and Dementia
Though a dominant view of dementia sees it as a condition that erases individuality and the ability to engage in activity, other ways of viewing dementia take into account the importance of the social context, abilities that persist long into dementia, and stigma that isolates and marginalizes people with this condition. This work explores the use of technology to challenge predominant conceptions of dementia and to support activities such as sharing online.

Citizen Science

NatureNet
NatureNet (NN) is an existing multi-touch technology platform that uses mobile devices and a website to enable participants to develop and carry out projects that support their own informal learning and foster positive scientific dispositions.

StreamBED VR
StreamBED VR is a a virtual reality training tool that teaches qualitative stream monitoring to citizen scientists. The goal of the system is to create an experience that guides novice water monitors to focus on key areas of the stream and so that they understand and evaluate streams relative to other similar streams spaces using visual, auditory and other sensory cues. This work contributes to research in the domains of citizen science, VR education, and multisensory design.

Communities

Crowd Discussions and Hidden Profile Paradigm
Online communities, from question and answer sites to general-purpose discussion forums, are increasingly working to solve hard problems together, often through a process of open sharing and discussion of ideas and information. “Wisdom of the crowd” arguments suggest that, in theory, crowds have the capacity to bring together diverse expertise, pooling distributed knowledge and thereby solving challenging and complex problems. In several ongoing projects we are testing this theory using the hidden profile paradigm, a set of problems used in small group research to evaluate whether groups pool distributed facts.

Science Everywhere
Science Everywhere is an NSF funded research study aimed at understanding how technology can engage entire communities in science learning. We utilize a design-based research approach in which we co-design innovative science learning technology with families, teachers, and leaders in a community, implement that technology in the community, and then redesign that technology in an iterative design process. Broadly, this study will contribute to theory on connected learning by developing an understanding of how to connect science learning at home, school, and community spaces with technology. This study also aims to contribute to our understanding of parent-child learning, interactive display design, and social media for learning.

Sensemaking technologies for collaboration at scale
In many collective creativity systems, the number of contributors can number in the thousands to tens of thousands, making true collaboration is a difficult challenge. Without effective mechanisms for surfacing key insights and sharing them among collaborators, many collective creativity efforts devolve into largely independent work. This often results in a preponderance of redundant, shallow, and bad ideas. We want to know: how might we build sensemaking technologies that support effective collaboration at scale? In collaboration with researchers in machine learning, we explore what kinds of sensemaking strategies are useful for creativity, what representational requirements are necessary for supporting those kinds of sensemaking (e.g., relational knowledge for analogy), and techniques from crowdsourcing that can support or work in combination with novel machine learning systems.

Design Process

Children as Design Partners (Intergenerational Design Teams)
Children, ages 7-11, work with HCIL faculty/staff/students after school and over the summer to create new technolgies for children.

Creatively reusing (old) knowledge
People build new ideas on what they know and have seen. Sometimes this a good thing; sometimes it kills creativity. We want to know: are there scientific principles that can guide creators’ interactions with sources of inspiration so that they are inspired and not hindered in their creativity? This is an old, fundamental question in the cognitive science of creativity, but has taken on a fresh significance in today’s information age, where creators can be exposed to many, many potential sources of inspiration online (e.g., Google Scholar, US Patent Database). Computational inspiration systems (such as search engines and recommender systems) can help by directing users’ attention to what is most inspirational. But what is most inspirational? (How) can we predict this beforehand?

Crowd civic systems
Classic work in design studies/research recognized policymaking as a prototypical kind of design: planning and taking actions to turn undesirable states into desirable ones. While much work has focused on how information technologies can augment the design process in other traditional domains like engineering, product, and service design, we are only just beginning to explore the potential of information technologies to augment the design of policy. Could there be computer-aided policymaking? What might that look like? Open government data, smart cities, and participatory budgeting have begun to scratch the surface of what designing policy could look like in the digital age, but much remains to be learned. We want to know: what new/better forms of governance and civic participation are possible in the digital age?atory budgeting have begun to scratch the surface of what designing policy could look like in the digital age, but much remains to be learned. We want to know: what new/better forms of governance and civic participation are possible in the digital age?

Digital Humanities

Informal STEM Learning Through Alternate Reality Games
Our Alternate Reality Games (ARGs) represent an effort to bring underrepresented populations into STEM fields by engaging them in the uniquely immersive gameplay of ARGs. These games ask players to incorporate and imagine gameplay as it takes place in their day-to-day lives, encouraging them to co-construct the fictional storyline as they play along, and to collaborate with hundreds or thousands of other players dedicated to the same tasks. Our primary audience is teenagers (13-17 years old), including those currently underrepresented in STEM (e.g., females, blacks, and hispanics). Our ARGs inspire players to take on the roles of real life scientists, programmers, engineers, technicians, artists, writers, and explorers as they solve challenges and immerse themselves in the storyworld. This project represents a multidisciplinary partnership with the National Science Foundation (award number 1323306), as well as Brigham Young University, NASA, Tinder Transmedia, Intuitive Company, and the Computer History Museum. Our ARGs bring academic researchers, writers, artists, scientists, transmedia producers, and players together as part of a larger design team.

Digital Libraries

International Children's Digital Library
A library that provides free access to children's books from around the world. By ensuring access to books from many cultures and in diverse languages, we foster a love of reading, a readiness to learn, and a response to the challenges of world illiteracy.

Keshif: Simplicity Driven Visual Faceted Browser
Keshif is a visual data browser that makes it easier to browse and understand your data. It presents visual summaries of your data properties, such as who, what, when and where, in its facets and timeline.

Sensemaking technologies for collaboration at scale
In many collective creativity systems, the number of contributors can number in the thousands to tens of thousands, making true collaboration is a difficult challenge. Without effective mechanisms for surfacing key insights and sharing them among collaborators, many collective creativity efforts devolve into largely independent work. This often results in a preponderance of redundant, shallow, and bad ideas. We want to know: how might we build sensemaking technologies that support effective collaboration at scale? In collaboration with researchers in machine learning, we explore what kinds of sensemaking strategies are useful for creativity, what representational requirements are necessary for supporting those kinds of sensemaking (e.g., relational knowledge for analogy), and techniques from crowdsourcing that can support or work in combination with novel machine learning systems.

Digital-Physical Interaction Design

BodyVis
A new wearable computing and electronic textiles (e-textiles) technology for learning about anatomy and physiology, and for supporting children’s scientific inquiry skills. (NSF-funded) (PI Jon Foehlich)

Technology Use and Dementia
Though a dominant view of dementia sees it as a condition that erases individuality and the ability to engage in activity, other ways of viewing dementia take into account the importance of the social context, abilities that persist long into dementia, and stigma that isolates and marginalizes people with this condition. This work explores the use of technology to challenge predominant conceptions of dementia and to support activities such as sharing online.

Education

BodyVis
A new wearable computing and electronic textiles (e-textiles) technology for learning about anatomy and physiology, and for supporting children’s scientific inquiry skills. (NSF-funded) (PI Jon Foehlich)

Children as Design Partners (Intergenerational Design Teams)
Children, ages 7-11, work with HCIL faculty/staff/students after school and over the summer to create new technolgies for children.

Enhancing Group Communication
Small groups rarely work together optimally. We developed a real-time language feedback system. It monitors communication patterns among students in discussion groups and provides real-time instructions to shape the way the group works together.

Informal STEM Learning Through Alternate Reality Games
Our Alternate Reality Games (ARGs) represent an effort to bring underrepresented populations into STEM fields by engaging them in the uniquely immersive gameplay of ARGs. These games ask players to incorporate and imagine gameplay as it takes place in their day-to-day lives, encouraging them to co-construct the fictional storyline as they play along, and to collaborate with hundreds or thousands of other players dedicated to the same tasks. Our primary audience is teenagers (13-17 years old), including those currently underrepresented in STEM (e.g., females, blacks, and hispanics). Our ARGs inspire players to take on the roles of real life scientists, programmers, engineers, technicians, artists, writers, and explorers as they solve challenges and immerse themselves in the storyworld. This project represents a multidisciplinary partnership with the National Science Foundation (award number 1323306), as well as Brigham Young University, NASA, Tinder Transmedia, Intuitive Company, and the Computer History Museum. Our ARGs bring academic researchers, writers, artists, scientists, transmedia producers, and players together as part of a larger design team.

International Children's Digital Library
A library that provides free access to children's books from around the world. By ensuring access to books from many cultures and in diverse languages, we foster a love of reading, a readiness to learn, and a response to the challenges of world illiteracy.

Science Everywhere
Science Everywhere is an NSF funded research study aimed at understanding how technology can engage entire communities in science learning. We utilize a design-based research approach in which we co-design innovative science learning technology with families, teachers, and leaders in a community, implement that technology in the community, and then redesign that technology in an iterative design process. Broadly, this study will contribute to theory on connected learning by developing an understanding of how to connect science learning at home, school, and community spaces with technology. This study also aims to contribute to our understanding of parent-child learning, interactive display design, and social media for learning.

Scratch Encore
This research collaboration with the University of Chicago and the Chicago Public School district seeks to answer the following question: Can we create advanced upper elementary Computer Science instructional materials that give equal value to improving equity and student learning outcomes? This research practitioner partnership is designing, developing, and evaluating advanced Scratch-based CS instructional materials for upper elementary students through a process that attends to practical barriers to equity. 

StreamBED VR
StreamBED VR is a a virtual reality training tool that teaches qualitative stream monitoring to citizen scientists. The goal of the system is to create an experience that guides novice water monitors to focus on key areas of the stream and so that they understand and evaluate streams relative to other similar streams spaces using visual, auditory and other sensory cues. This work contributes to research in the domains of citizen science, VR education, and multisensory design.

Understanding the Design of Introductory Programming Environments
Blocks-based programming environments are growing in popularity and are increasingly being used in formal introductory programming contexts. To date, much of the work evaluating such tools has focused on their effectiveness in out-of-school contexts and emphasized engagement and attitudinal measures over content mastery. Given their growing presence in classrooms, it is important to understand the benefits and drawbacks to the use of the blocks-based programming approach in formal learning contexts. This project seeks to understand the affordances and drawbacks of block-based programming relative to conventional text-based languages.

Innovation, Discovery, and Creativity

Colloquium on Revisiting Cybernetic Serendipity
Our ambition is to redirect the history of ideas, restoring the Leonardo-like close linkage between art/design and science/engineering/medicine.  We believe that internet-enabled collaborations can make more people more creative more of the time. This colloquium used the historical framework of Cybernetic Serendipity to look at how the context has changed, and how creativity and collaboration are impacting practice and research today. How should we re-envision research policy and educational approaches to maximize the impact of partnerships with design, art, and humanities? How can we productively engage business, government, and non-governmental organizations as research and educational partners?

Creatively reusing (old) knowledge
People build new ideas on what they know and have seen. Sometimes this a good thing; sometimes it kills creativity. We want to know: are there scientific principles that can guide creators’ interactions with sources of inspiration so that they are inspired and not hindered in their creativity? This is an old, fundamental question in the cognitive science of creativity, but has taken on a fresh significance in today’s information age, where creators can be exposed to many, many potential sources of inspiration online (e.g., Google Scholar, US Patent Database). Computational inspiration systems (such as search engines and recommender systems) can help by directing users’ attention to what is most inspirational. But what is most inspirational? (How) can we predict this beforehand?

Open Data

Open Collaboration and Data Science
The democratization of data science and open data initiatives have inspired groups from civic hackers to data journalists to use data to address social issues as well as open innovation contest platforms like Kaggle. Our ongoing work investigates ways people are collaborating to analyze open data using data science and big data techniques.

Privacy

Improving Developers' Privacy and Security Decisionmaking
Well-intentioned software developers can put user data at risk when they make errors with complicated encryption protocols, abuse device identifiers and location data, or make other mistakes. In this work, we examine why these errors occur and how they can be prevented.

NetCHI Lab
We study people and networks and create tools to help people with their networks. We are working to facilitate an efficient, secure, and affordable Internet for all by studying how users get online, creating broadband tools, and having fun in the process.

Public Access

International Children's Digital Library
A library that provides free access to children's books from around the world. By ensuring access to books from many cultures and in diverse languages, we foster a love of reading, a readiness to learn, and a response to the challenges of world illiteracy.

Security

Improving Developers' Privacy and Security Decisionmaking
Well-intentioned software developers can put user data at risk when they make errors with complicated encryption protocols, abuse device identifiers and location data, or make other mistakes. In this work, we examine why these errors occur and how they can be prevented.

Keeping Kids Safe
(with Marshini Chetty at Princeton, Jessica Vitak, and Ph.d. student Priya Kumar)

NetCHI Lab
We study people and networks and create tools to help people with their networks. We are working to facilitate an efficient, secure, and affordable Internet for all by studying how users get online, creating broadband tools, and having fun in the process.

Understanding Security Advice Sources to Improve User Safety
Users selectively choose which security advice to accept and some (most) to reject; however, it is unclear whether they are effectively prioritizing what is most important or most useful. If we can understand from where and why users take security advice, we can develop more effective security interventions.

Social Media & Networks

Informal STEM Learning Through Alternate Reality Games
Our Alternate Reality Games (ARGs) represent an effort to bring underrepresented populations into STEM fields by engaging them in the uniquely immersive gameplay of ARGs. These games ask players to incorporate and imagine gameplay as it takes place in their day-to-day lives, encouraging them to co-construct the fictional storyline as they play along, and to collaborate with hundreds or thousands of other players dedicated to the same tasks. Our primary audience is teenagers (13-17 years old), including those currently underrepresented in STEM (e.g., females, blacks, and hispanics). Our ARGs inspire players to take on the roles of real life scientists, programmers, engineers, technicians, artists, writers, and explorers as they solve challenges and immerse themselves in the storyworld. This project represents a multidisciplinary partnership with the National Science Foundation (award number 1323306), as well as Brigham Young University, NASA, Tinder Transmedia, Intuitive Company, and the Computer History Museum. Our ARGs bring academic researchers, writers, artists, scientists, transmedia producers, and players together as part of a larger design team.

Older Activists and Ageism
Broad societal views, such as stereotypes based on age, become embedded in and perpetuated through the design of technologies. This project looks at the ways that older adults' preferences for technology challenge some of the assumptions that designers and researchers make about older people, particularly in regard to conceptions around social isolation and physical and cognitive decline. The work also examines how older adults use technology platforms such as blogs and social media to respond to ageism.

Visualization

BodyVis
A new wearable computing and electronic textiles (e-textiles) technology for learning about anatomy and physiology, and for supporting children’s scientific inquiry skills. (NSF-funded) (PI Jon Foehlich)

CoCo: A Visual Analytics Tool for Comparing Cohorts of Event Sequences
CoCo is a visual analytics tool that enables users to compare two sets of temporal sequence data. It combines automated statistical tests with user-guidance to enable insights, hypothesis generation, and much more. Users see (1) statistics about their dataset, (2) event-level statistics, and (3) a menu of metrics. CoCo displays significance tests in a unified form for measures such as prevalence and duration of gaps.

EventAction: Visual Analytics for Temporal Event Sequence Recommendation
EventAction introduces prescriptive analytics with temporal event sequences. After identifying similar temporal records users can review recommended temporal event sequences that help them achieve their goals and define a personalized action plan.

EventFlow
The HCIL's ongoing work with temporal event records has produced powerful tools for analyzing patterns of point-based events (LifeLines2, LifeFlow). However, users found that point-based events limited their capacity to solve problems that had inherently interval attributes. To address this issue, EventFlow extends its predecessors to support both point-based and interval-based events. With EventFlow, we present novel solutions for displaying interval events, simplifying their visual impact, and incorporating them into meaningful queries.

Keshif: Simplicity Driven Visual Faceted Browser
Keshif is a visual data browser that makes it easier to browse and understand your data. It presents visual summaries of your data properties, such as who, what, when and where, in its facets and timeline.

PAOHVIS: Dynamic Hypergraphs Visualization (links to INRIA Webpage)
Parallel Aggregated Ordered Hypergraph (PAOH) is a novel technique to visualize dynamic hypergraphs. It is easy to learn and is well suited for medium size dynamic hypergraph networks such as those commonly generated by digital humanities projects - our driving application domain.