This week we’ll hear from two researchers about their UIST papers!

Location: HBK 2105 and Zoom

 

Talk Title: EvalAssist: Insights on Task-Specific Evaluations and AI-Assisted Judgment Strategy Preferences

Speaker: Zahra Ashktorab, Senior Research Scientist, IBM Research

Abstract:Evaluating LLM outputs is now central to product reliability, alignment, and AI safety, yet existing workflows remain costly, ad hoc, and difficult to scale. LLM-as-a-judge methods offer scalable evaluation, but current systems provide limited transparency into judgment processes, lack support for evolving task-specific criteria, and give practitioners little control over how judgments are made. In this talk, I trace the human-centered design and development of EvalAssist, a framework and platform for authoring, validating, and scaling LLM-based evaluations. The system isolates generation from evaluation, supports multiple evaluator models, exposes bias indicators, and allows practitioners to select between direct assessment and pairwise comparison, the two dominant judgment strategies in LLM-as-a-judge workflows. Through a controlled study with practitioners, and deployment, I show how task framing, criteria iteration, and judgment strategy choices shape evaluation outcomes, human–AI alignment, trust, and perceived cognitive load. We demonstrate how practitioners refine criteria (modifying specificity, adding exclusion rules, redefining scales) and how design affordances influence strategy preferences.

Bio: Zahra Ashktorab is a Senior Research Scientist at IBM Research, where she investigates how to enable better outcomes in human–AI interaction. Her work focuses on how people understand and collaborate with AI systems, and on developing evaluation frameworks that make AI behavior more reliable and transparent. She led the 0→1 development of EvalAssist, a human-centered LLM-as-a-Judge platform that grew from early prototypes to a deployed tool used by practitioners. Her research sits at the intersection of HCI and AI and has helped shape emerging practices around LLM-as-a-Judge and evaluation-as-infrastructure. Zahra publishes across leading HCI and AI communities, including CHI, CSCW, UIST and IUI. She holds a Ph.D. in Human–Computer Interaction from the University of Maryland.

——————————————————————————————————————————————————————————————

Talk Title: From Velcro to Zipper: Enabling Functions and Forms Between the Dimensions of 3D Printing.

Speaker: Deying Pan, PhD candidate, Zhejiang University

Abstract: My research aims to bridge the gap between static form and dynamic function in FDM printed structures. By developing computational workflows for integral fastening and dimensional folding, we enable the rapid design and fabrication of large-scale, reversible assemblies that transcend the limitations of traditional build volumes and support structures. As a result, these methods enable accessible fabrication across diverse scales, from kinetic installation and wearable technology to modular furniture and large-scale architectural prototyping.

Bio: Deying Pan, PhD candidate at Zhejiang University, currently visiting at CMU.