WORKSHOP: Temporal Event Sequence Analysis EventFlow and CoCo User Group Meeting 2016

WORKSHOP: Temporal Event Sequence Analysis
EventFlow and CoCo User Group Meeting

Thursday, May 26, 2016

A workshop of the
33^rd Human-Computer Interaction Lab Symposium
University of Maryland

Overview and Topics

The growing volume of temporal data available from sensors, social media sources, Web logs, and medical histories present remarkable opportunities for researchers and policy analysts. Research at HCIL has led to the development of a family of tools for the exploration of temporal event sequences, such as EventFlow to summarize and search event sequence patterns and CoCo – short for Cohort Comparison — for the comparison of groups of records and their event sequences.

This workshop will allow researchers who are using those tools to share their findings and strategies for successful event data analysis. We anticipate various application domains to be discussed, including the analysis of Electronic Health Record data, activity log analysis, learning analytics, and more. We also welcome discussions of other uses of visual approaches, statistical methods, machine learning, etc. to study temporal patterns in event sequences, where the goals may be to find common patterns, rare events, similar records etc. Topics might also include data cleaning, integration from multiple sources, coping with incomplete or conflicting information, and presentation of result sets.

This workshop will include talks (followed by ample time for discussions) from:

HCIL researchers (e.g. our latest work on EventFlow and our newer work on cohort comparison)
Users of HCIL tools such as EventFlow (who will be invited to report on their case studies), and
Other researchers working on similar topics will be selected to present their work (please contact us if you are interested).

To Present Your Work:

If you are interested in presenting please submit a short abstract (~ 300 words max). Summarize what you would like to present on the topic of Exploring Temporal Patterns in Electronic Health Record Data, and provide pointers to papers and screen shots as needed.
Submit to Catherine Plaisant at plaisant@cs.umd.edu (Deadline: May 1, 2016, Notification: May 8, 2016). If we receive too many requests, we may ask you to bring a poster for presentation during the breaks.

To Participate:

Come and join us! The workshop is opened to all. Space permitting, so plan to register early (mid-May at the latest is best).

Organizers

Catherine Plaisant — Senior Research Scientist, HCIL
Ben Shneiderman — Professor of Computer Science
Fan Du — Graduate Research Assistant, Dept. of Computer Science

Questions: Please contact Catherine Plaisant (plaisant@cs.umd.edu)

Participants

Agenda (PDF)

The workshop will take place at Room 3120. Other symposium activities are on the 1st floor (directions).

08:15am – Symposium Registration, Breakfast
09:00am – Symposium Plenary Talks (more information)

1:00pm-2:20pm – Workshop Session I
- Introduction (video)
  - Ben Shneiderman, HCIL, University of Maryland
- EventFlow Overview (slides | video)
  - Catherine Plaisant, HCIL, University of Maryland
- Giardiasis Diagnosis and Treatment in the United States: New Tools for an Old Disease (abstract | slides | video)
  
  Background: Giardiasis, the most common enteric parasitic infection in the United States, causes an estimated 1.2 million cases annually. Despite this burden, clinical index of suspicion remains low and case series data suggest delayed diagnosis and absent or ineffective treatment are common. We investigated diagnosis and treatment patterns using a “big data” approach and EventFlow, novel data visualization software for longitudinal data.
  
  Methods: We linked testing, diagnosis and prescription claims (“events”) to create a longitudinal cohort of giardiasis patients (ICD-9-CM 007.1), using the MarketScan insurance database (80 million persons). We used EventFlow to view and query event sequences occurring 90 days before and after diagnosis, and identify temporal patterns. We defined the “expected” sequence as: diagnostic test followed by antiparasitic prescription (drugs effective against Giardia), but no antibiotic prescription (ineffective drugs). We identified attributes of outpatients with “unexpected” sequences, and analyzed associations between sequence and age group, sex, and census region using chi-square tests.
  
  Results: Among 2,995 outpatients with 212,433 claims, 19% followed the “expected” event sequence. Among the 2,454 (81%) outpatients with an “unexpected” event sequence, almost half (47%) did not receive a diagnostic test; 33% received an antibiotic ineffective against Giardia before their diagnosis, and 44% never received an antiparasitic. Outpatients living in the southern US were less likely to receive the “expected” care sequence than those in other regions (P<.001).
  
  Conclusions: Few outpatients received the “expected” giardiasis care sequence, and sequences differed by region. Ineffective antibiotics and absent testing suggest opportunities for provider education. Using an iterative analytic process coupling visualization software with traditional statistics can quickly identify clinically relevant patterns in complex longitudinal data to prioritize scarce public health resources.
  - Karlyn D. Beer, Centers for Disease Control and Prevention
- Modeling Drug and Medical Device Innovation as Temporal Sequences Using EventFlow (abstract | slides | video)
  Science, Technology and Innovation (STI) processes may be characterized as sequences of activities ranging from basic research to invention, prototyping, development (commercialization), testing, approvals (if required), and culminating in the introduction of new products. These sequences are complex, often requiring multiple iterations. Each activity also requires specialized talent and funding. Our research uses EventFlow to model and measure the STI sequences for drugs and medical devices. Understanding these STI temporal patterns could inform policy changes to accelerate the processes that bring new drugs and medical devices to the marketplace. We show data from historical databases using EventFlow.
  - C. Scott Dempwolf, Urban Studies and Planning Program, University of Maryland

2:35pm-4:10pm – Workshop Session II
- Everyday Use of EventFlow in an Academic Teaching Hospital (abstract | video)
  Electronic medical record (EMR) systems’ digital format allows manipulation of data for such tailored purposes as creating meaningful displays and aiding clinical decision support. While EMR systems have demonstrated success in improving care processes, current evidence for improved patient outcomes is sparse. One major technical challenge in establishing outcome improvement using EMR systems is navigating temporal relationships. Clutter from the increasing volume and variety of sequential events distracts from meaningful insights, and an improved analytic focus-sharpening strategy is needed. We demonstrate various ways that EventFlow’s strengths in visualization and exploratory analysis aid us in mining temporal health-related event sequences. EventFlow streamlines various data mining tasks, which, if attempted programmatically, would be tedious and error-prone. It helps us manipulate the data and figure out the right questions to ask quickly by providing tools for iterative exploration of our data sets and giving instant feedback. We highlight several real-world examples that show how EventFlow helps us to extract insights from complex temporal medical data and enumerate various lessons we have learned along the way.
  - Gloria Lipori, Chief Data Officer, University of Florida Health and University of Florida Health Science Center
- High-Volume Hypothesis Testing: Systematic Exploration of Event Sequence Comparisons (abstract | slides | video)
  Cohort comparison studies have traditionally been hypothesis driven and conducted in carefully controlled environments (such as clinical trials). Given two groups of event sequence data, researchers test a single hypothesis (e.g., does the group taking Medication A exhibit more deaths than the group taking Medication B?). Recently, however, researchers have been moving toward more exploratory methods of retrospective analysis with existing data. In this article, we begin by showing that the task of cohort comparison is specific enough to support automatic computation against a bounded set of potential questions and objectives, a method that we refer to as High-Volume Hypothesis Testing (HVHT). From this starting point, we demonstrate that the diversity of these objectives, both across and within different domains, as well as the inherent complexities of real-world datasets, still requires human involvement to determine meaningful insights. We explore how visualization and interaction better support the task of exploratory data analysis and the understanding of HVHT results (how significant they are, why they are meaningful, and whether the entire dataset has been exhaustively explored). Through interviews and case studies with domain experts, we iteratively design and implement visualization and interaction techniques in a visual analytics tool, CoCo. As a result of our evaluation, we propose six design guidelines for enabling users to explore large result sets of HVHT systematically and flexibly in order to glean meaningful insights more quickly. Finally, we illustrate the utility of this method with three case studies in the medical domain.
  - Sana Malik¹, Ben Shneiderman¹, Fan Du¹, Catherine Plaisant¹, Margret Bjarnadottir²
    - ¹HCIL, University of Maryland; ²Robert H. Smith School of Business, University of Maryland
- Analyzing Medical Claim Data with CoCo (slides)
  - Margret Bjarnadottir, Robert H. Smith School of Business, University of Maryland
- Other Examples of CoCo Analysis (slides | video)
  - Sana Malik, HCIL, University of Maryland

4:25pm-5:00pm – Workshop Session III
- Visual Analytics for Temporal Event Sequence Recommendation (abstract | slides | poster | video)
  Recommender systems are being widely used to assist people in making decisions, for example, recommending films to watch or books to buy. Despite its ubiquity, the problem of presenting the recommendations of temporal event sequences has not been studied. We propose EventAction, which to our knowledge, is the first attempt at a prescriptive analytics interface designed to present and explain recommendations of temporal event sequences. EventAction provides a visual analytics approach to (1) identify similar records, (2) recommend temporal event sequences that might help achieve the users’ goals, (3) identify key steps in the recommended event sequences that are most important, and (4) interactively assist users as they define a personalized action plan associated with a probability of success. Roughly following the design study methodology, we designed and deployed EventAction in the context of student advising and report on the evaluation with a review manager of graduate students in computer science and students working on their own.
  - Fan Du¹, Catherine Plaisant¹, Neil Spring², Ben Shneiderman¹
    - ¹HCIL, University of Maryland; ²Department of Computer Science, University of Maryland
- Discussion & Other Examples of EventFlow Use
  - Catherine Plaisant, HCIL, University of Maryland

05:00pm – Symposium Demos, Posters, Reception
06:30pm – Symposium Ends

Event Logistics

Symposium Registration (now open).
Accommodations.
Directions, Maps, and Parking Information.
All activities will take place in CSIC (Computer Science Instruction Center), near the AV Williams bldg / Computer Science department (also near Route 1).
Plan enough time for parking.
Coffee and light breakfast will be available in the morning during the demos.
Lunch will include some vegetarian offerings.

Related HCIL Materials

Strategy Paper: Du, F., Shneiderman, B., Plaisant, C., Malik, S., Perer, A., Coping with Volume and Variety in Temporal Event Sequences: Strategies for Sharpening Analytic Focus, IEEE Transactions on Visualization and Computer Graphics, in press.
CoCo Paper: Malik, S., Shneiderman, B., Du, F., Plaisant, C., Bjarnadottir, M., High-Volume Hypothesis Testing: Systematic Exploration of Event Sequence Comparisons, ACM Transactions on Interactive Intelligent Systems, in press.
Review Paper: Rind, A., Wang, T., Aigner, W., Miksch, S., Wongsuphasawat, K., Plaisant, C., Shneiderman, B., Interactive Information Visualization for Exploring and Querying Electronic Health Records: A Systematic Review, Foundations and Trends in Human-Computer Interaction, Vol. 5, No. 3 (2013) 207-298.
EventFlow
CoCo for Cohort Comparison
LifeFlow (older)
LifeLines2 (older)
LifeLines (older)