Sensing and User Modeling
How can CPS robustly sense, interpret, and model users across diverse real-world contexts?
The convergence of wearables, robotics, extended reality, and smart environments is expanding the design space for cyber-physical systems (CPS) that support and augment human abilities in everyday life. By sensing real-world contexts, modeling users’ needs, and delivering situated assistance, these systems can enhance capabilities such as perception, memory, learning, and mobility. They also offer new ways to address long-standing accessibility barriers in areas such as independent living, navigation, and social participation.
This workshop brings together researchers and practitioners from HCI, AI, wearables, robotics, XR, smart environments, accessibility, and ability augmentation to examine shared strategies and challenges for designing accessibility- and ability-centered CPS. Through keynotes, panel discussions, interactive demos, and mixed-group design activities, participants will identify emerging design principles, surface shared challenges, and chart future directions for CPS that support and augment human abilities in everyday life. The organizing committee will synthesize key workshop outcomes into a public report and maintain a participant communication channel to support continued exchange beyond the event.

How can CPS robustly sense, interpret, and model users across diverse real-world contexts?
How can CPS provide situated assistance while preserving user agency during daily tasks?
How can CPS adapt to diverse user needs, abilities, preferences, and physical environments?
How should CPS be evaluated to assess real-world usability and long-term impact?
How can CPS protect privacy, safety, and bystander consent in everyday physical spaces?
9:00 - 9:30
9:30 - 10:15
We are nearing what I call the AI Realization Era — a time when many of the long-standing problems in AI are solved. I believe this will profoundly change how we interact with computers, and may finally allow us to solve long-standing problems in accessibility. At the same time, this will not come for free. In this talk, I’ll revisit three lines of work from my twenty years of working in accessibility, discuss how they have changed with the coming AI Realization Era, and what new problems we need to solve to ensure that people with disabilities fully benefit.
10:15 - 11:00
AI is leaving the chat window and entering the physical world. Through wearable cameras and mixed-reality glasses, it now sees what people see, reasons about their surroundings, and acts on their behalf. Whether this shift empowers or excludes people is a central question for human-computer interaction, and the problem is particularly sharp for people with disabilities (PwD). PwD have abilities and needs that defeat one-size-fits-all interaction paradigms, produce behaviors and data that AI has rarely seen, and face heightened risks to safety, agency, privacy, and trust when AI mediates their perception or acts for them. These challenges make disability the most demanding and revealing context for building human-centered AI that is useful, safe, trustworthy, and value-aligned.
Using low vision as a case, I will present my research on embodied, assistive AI through mixed reality that supports activities of daily living by building on users’ visual abilities and preferences rather than substituting for them. I first describe adaptive AR systems that perceive users’ context and behavior and augment only what matters: scene-aware systems that highlight the affordances of kitchen tools for safe interaction and distinguish objects by importance in cluttered scenes for at-a-glance perception, and a gaze-aware system that infers visual intent from eye movements to support line switching and difficult-word recognition in reading. I then turn from AI as proactive assistance to human-AI collaboration, introducing mixed-initiative agents in which the user and the AI each contribute what they perceive best. VisiAgent, our mixed-reality agent for low vision, lets users ground requests in their own gaze and pointing and critique the AI’s augmentations, retaining agency over what to augment and how, while the agent steps in proactively when safety demands. I will close with the open questions this work raises for collaborative assistive AI: who should take initiative and when, how to disambiguate references for users whose referencing abilities differ, how to allocate feedback across senses, and how to reduce the risks users face when relying on an AI assistant that can make errors.
11:00 - 12:00
12:00 - 13:45
13:45 - 14:45
14:45 - 15:30
Most talks about augmenting human abilities focus on what systems can add: more information, more alerts, more intelligence. This talk is about the opposite. Over the past decade, my lab has built systems that augment hearing, mostly with deaf and hard of hearing people: IoT displays that show sounds across a home, smartwatches that recognize them, mixed-reality headsets that describe a scene, and hearables that reshape the sounds a person hears. Looking across them, the most useful lesson is a counterintuitive one: they got better when they did less. They helped most when they stayed silent about sounds the person already knew, described what they heard without guessing what it meant, turned sounds down rather than cancelling them, and left the final judgment to the person whenever the person knew more than the microphone did. This restraint matters most when people, like many of those we work with, cannot double-check what a system tells them about a sense they don’t have full access to. I’ll share the systems and real-world deployments that taught us these lessons, and close with open questions for building cyber-physical systems that augment ability without taking it over.
15:30 - 16:15
16:15 - 17:00

Associate Professor at Carnegie Mellon University
Jeffrey P. Bigham is an Associate Professor in the Human-Computer Interaction and Language Technologies Institutes in the School of Computer Science at Carnegie Mellon University. He also leads a Human-Centered Machine Intelligence Group at Apple, which works on research and applied projects in Accessibility, AI Fairness, ML Design, Learning Sciences, InfoVis, and Computational Understanding of UIs.

Assistant Professor at University of Wisconsin-Madison
Yuhang Zhao is an Assistant Professor in Computer Sciences at the University of Wisconsin-Madison. Her research interests include Human-Computer Interaction (HCI), accessibility, augmented and virtual reality (AR/VR), and AI-powered interactive systems. She designs and builds embodied intelligence for accessibility: AI-powered XR systems that interpret user context and behavior, and adaptively enhance human abilities.

Assistant Professor at University of Michigan
Dhruv Jain is an Assistant Professor of Computer Science and Engineering at the University of Michigan, also affiliated with the School of Information and Michigan Medicine. He builds and studies human-AI systems for accessibility across three research pillars: (1) AI for Sound Accessibility; (2) Accessible AI for the Future of Work; and (3) Reliability in Real-World Accessible AI.

Assistant Professor, UNC-Chapel Hill
Assistive technology, ubiquitous computing, and HCI



Humboldt Research Fellow, University of Stuttgart
Artificial Intelligence and Cognitive Science

PhD Candidate, University of Illinois Urbana-Champaign
navigation technologies for blind and low-vision explorers
The attendees below have kindly consented to sharing a brief introduction. We hope these introductions help bridge diverse communities and foster meaningful connections.
My research sits at the intersection of human-AI interaction and AI for social good. I am broadly interested in how intelligent systems can be designed to preserve human agency and critical thinking—particularly in contexts where AI outputs carry uncertainty or where users are underrepresented in default interface design. This includes questions of how to surface model confidence in ways that build rather than undermine user trust, how adaptive AI systems can serve diverse user needs without requiring explicit configuration, and how the internal dynamics of large language models (such as reasoning depth and hidden-state convergence) relate to the transparency and reliability that end-users experience.
My research interests lie in augmented reality and accessibility. I design and develop AR technology to automatically recognize surrounding visual information that cannot be easily perceived by people with low vision, and generate suitable multi-modal feedback to enhance their perceptual abilities in various daily tasks.
I work on Assistance Systems for Skill Acquisition.
I work on accessibility technology for people with visual impairments in outdoor sports. My project SnowEcho is a blind skiing assistance system that conveys obstacles and course boundaries through spatial audio, allowing skiers to perceive traversable free space and choose their own line rather than following a prescribed route.
I challenge outcome-driven "access" and develop a process-oriented frame: Developmental Flourishing. It reframes assistive technologies not as instruments for reaching task outcomes, but as a means through which blind and low-vision individuals can explore, create, and construct meaning from their experiences. I share an analysis of sensing and AI systems, discussing the extent to which they align with and augment blind people’s perceptual and cognitive processes, while highlighting persistent challenges and future directions.
I am broadly interested in Artificial Intelligence and Human-Computer Interaction, particularly with applications in accessibility for blind users. Recently, I built two AI tools, CodeA11y and iTagPDF, that help automatically produce and preserve accessibility metadata for websites and PDFs, respectively.
Accessibility of interfaces through computational methods.
My research lies at the intersection of ubiquitous computing, mobile sensing, human-centered computing, and responsible AI. My work advances privacy-aware sensing systems for health and wellness.
Economics, Consumer Behavior, and Market Psychology.
I am a researcher on the MIND team at Apple. I received my CS Ph.D. student from Stony Brook University, working with Shubham Jain at the intersection of Ubiquitous Sensing and Health Systems in the PiCASSo lab. I have worked on around-the-ear devices. I spent the summer of 2025 with Microsoft Research, Redmond, in the Networking Research Group. My research has been awarded by Stony Brook University and covered by The Economic Times, Outlook, Financial Express, and The Hindu.
I study and design AI and AR interfaces to explore how computational tools can become more transparent, malleable, and dynamic, as well as how they relate to the broader environments they inhabit, primarily in the contexts of accessibility, creativity support, and sensemaking. I believe in empowering people to engage with and through technology in more embodied, pluralistic, and liberatory ways.
I am interested in how AI can enable new, accessible forms of spatial design and artistic expression. As a practitioner in film and design, I am particularly interested in creative agency, including the ability to shape an experience as well as participate in it.
My research explores how AI-enabled personal devices, such as smartphones and wearables, can provide individualized, situated accessibility support that strengthens disabled people’s agency and autonomy. This directly connects to the workshop’s focus on accessibility-centered cyber-physical systems that sense real-world contexts and support everyday activities. Recently, I have been thinking about these goals in terms of end-user programming (primarily natural language programming, but more recently also Programming by Example and Programming by Demonstration). This has led to a project, ProgramAT, which advances this agenda by enabling blind and low-vision people to create and refine bespoke, camera-based, assistive tools, shifting from systems that simply try to adapt to users toward systems users can actively shape.
My research lies at the intersection of AI and HCI, exploring how interactive systems can understand people, context, and the physical world to make interaction more natural and accessible. My recent work focuses on context-aware gesture understanding and vision-based tactile sensing, and I am interested in extending this toward wearable and cyber-physical systems that can adaptively guide, constrain, or augment human movement to support and extend human abilities.
I study accessibility technologies for deaf–hearing communication. We co-designed SignWarriors, a Japanese Sign Language learning game where hearing parents of deaf children summon warriors by signing to a camera. This use of camera sensing to support accessibility-oriented learning aligns with the workshop’s themes, which is why I am joining.
My research explores how XR and AI can augment human cognition. My recent work investigates diminished reality to help students with ADHD manage visual distractions as well as the diminishment and abstract representation of others in shared spaces to support attention while preserving social awareness. More broadly, I’m interested in making everyday environments more accessible and responsive to different cognitive needs.
My research focuses on building interactive audio AI systems to help people make sense of and shape their physical and digital environments as their needs evolve.
My research interests include accessibility-driven design and how we can build interactive systems to support people in their daily lives across a variety of tasks. I'm interested in making complex technologies and information more accessible and exploring how adaptive interfaces can accommodate diverse abilities and preferences, including through multimodal interaction and interdisciplinary approaches, to foster greater independence and agency.
My research interests lie at the intersection of HCI, human perception, and cognitive science. My previous work has involved XR, haptic interfaces, and behavioral experiments, exploring how interactive technologies influence human perception and decision-making. Through these experiences, I became increasingly interested in understanding why people perceive and interpret the same information differently, particularly under uncertainty. Recently, I have been exploring computational approaches to cognition, especially how behavioral experiments and probabilistic models can help explain human perception, judgment, and confidence. Ultimately, I hope to connect these insights with interactive technologies to develop systems that augment human perceptual and cognitive capabilities.
I’m broadly interested in human–AI interaction and accessibility. My research explores how personal AI can understand and expand people’s capabilities and help them develop new ones.
My research explores how sensing and embodied interaction can expand the ways people use everyday systems. In my UIST 2026 paper, I developed a software approach that uses microphone arrays to detect coarse airflow direction, motivated in part by hands-busy situations. I also work on movement-based interaction in XR. These projects connect with the workshop’s focus on context sensing and ability augmentation. I hope to learn from accessibility researchers and discuss how to design and evaluate such systems with diverse users before drawing conclusions about their real-world benefits.
7 more attendees (chose not to disclose their information)
* equal contributions

HKUST · MIT Media Lab

Carnegie Mellon University

UC Berkeley

University of Rochester

University of Rochester

MIT Media Lab

University of Michigan

University of Tokyo

HKUST

UC Berkeley

UNC-Chapel Hill

University of Rochester

Columbia University

MIT Media Lab
If you have any accessibility needs, please contact us. We will do our best to accommodate your needs.
This workshop builds on the Accessible Cyber-Physical Activities workshop at UIST 2025.