Sonic Stage: Auto-Generating Interactive Spatial Soundscapes to Facilitate Dialogue Video Comprehension for Blind Viewers
Shuchang Xu, Xiaofu Jin, Gaurav Jain, Wenshuo Zhang, Huamin Qu, Brian A. Smith, Yukang Yan
ACM Symposium on User Interface Software and Technology (UIST), 2026.
Abstract
Audio description (AD) makes film and television accessible to blind and low-vision (BLV) audiences by narrating characters’ actions. However, in scenes with lots of dialogue, AD often omits important actions because it is constrained not to overlap with speech. It is not yet known how to convey characters’ actions during dialogue. We present Sonic Stage, a system that transforms dialogue videos into interactive spatial soundscapes, enabling BLV audiences to intuitively understand characters’ actions and movements through immersive auditory cues. Sonic Stage conveys essential visual information during dialogue through three auditory techniques: (1) spatialized dialogue to represent spatial layout, (2) diegetic sound to convey character actions, and (3) interactive descriptions to provide context-specific visual details. Evaluation with 12 BLV viewers showed that Sonic Stage significantly improved video comprehension, spatial presence, and narrative engagement. We highlight opportunities for enhancing video accessibility across diverse genres through immersive, interactive audio representations.Links
BibTeX
@inproceedings{xu26_uist2,
title = {Sonic {{Stage}}: {{Auto-Generating Interactive Spatial Soundscapes}} to {{Facilitate Dialogue Video Comprehension}} for {{Blind Viewers}}},
shorttitle = {Sonic {{Stage}}},
author = {Xu, Shuchang and Jin, Xiaofu and Jain, Gaurav and Zhang, Wenshuo and Qu, Huamin and Smith, Brian A. and Yan, Yukang},
year = {2026},
doi = {10.48550/arXiv.2607.20835},
booktitle = {ACM Symposium on User Interface Software and Technology (UIST)}
}
