Multi-duration Saliency Prediction for Mobile User Interfaces

Description: Saliency maps are a common methodology to visualise and estimate the users’ visual attention on a number of visual stimuli (e.g. images, videos, user interfaces, etc.). Traditionally, such saliency maps are obtained either with special-purpose eye tracking equipment or with regular commodity cameras that track the users’ gaze. However, cameras may not always be available. Recent works have proposed computational models of attention, i.e. methods that estimate the users’ attention directly from the stimuli. Yet these methods do not capture the temporal dynamics of gaze behaviour.

In this work, we will investigate prior works that proposed multi-duration saliency maps, i.e. saliency maps that capture different viewing durations, for natural images (Fosco et al. 2020) and extend this concept to mobile user interfaces. For evaluations, we will rely on existing datasets (e.g. the Mobile UI Saliency dataset (Leiva et al. 2020)) Image source: Leiva et al., 2020.

Supervisor: Mihai Bâce

Distribution: 30% Literature, 10% Data Preparation, 30% Implementation, 30% Analysis and Evaluation

Requirements: Experience with Tensorflow/PyTorch

Literature: Fosco, Camilo Luciano, Anelise Newman, Pat Sukhum, Yun Bin Zhang, Nanxuan Zhao, Aude Oliva, and Zoya Bylinskii. 2020. How much time do you have? Modeling multi-duration saliency. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

Leiva, Luis A., Yunfei Xue, Avya Bansal, Hamed Tavakoli, Tuðçe Köroðlu, Jingzhou Du, Niraj Dayama and Antti Oulasvirta. 2020. Understanding visual saliency in mobile user interfaces. Proceedings of the 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI).