Two papers accepted at NeurIPS workshop Gaze Meets ML

We are pleased to announce that two papers was accepted for publication in NeurIPS workshop Gaze Meets ML, including one as an oral presentation. Congratulations to the authors!

1. 

   Federated Learning for Appearance-based Gaze Estimation in the Wild

   Mayar Elfares, Zhiming Hu, Pascal Reisert, Andreas Bulling, Ralf Küsters

   Proceedings of The 1st Gaze Meets ML workshop, PMLR, pp. 20–36, 2023.

   Abstract Links BibTeX Project

   Gaze estimation methods have significantly matured in recent years, but the large number of eye images required to train deep learning models poses significant privacy risks. In addition, the heterogeneous data distribution across different users can significantly hinder the training process. In this work, we propose the first federated learning approach for gaze estimation to preserve the privacy of gaze data. We further employ pseudo-gradient optimisation to adapt our federated learning approach to the divergent model updates to address the heterogeneous nature of in-the-wild gaze data in collaborative setups. We evaluate our approach on a real-world dataset (MPIIGaze) and show that our work enhances the privacy guarantees of conventional appearance-based gaze estimation methods, handles the convergence issues of gaze estimators, and significantly outperforms vanilla federated learning by 15.8% (from a mean error of 10.63 degrees to 8.95 degrees). As such, our work paves the way to develop privacy-aware collaborative learning setups for gaze estimation while maintaining the model’s performance.

   Paper: elfares23_gmml.pdf

   Paper Access: https://proceedings.mlr.press/v210/elfares23a.html

   @inproceedings{elfares23_gmml, title = {Federated Learning for Appearance-based Gaze Estimation in the Wild}, author = {Elfares, Mayar and Hu, Zhiming and Reisert, Pascal and Bulling, Andreas and K{\"u}sters, Ralf}, booktitle = {Proceedings of The 1st Gaze Meets ML workshop, PMLR}, pages = {20--36}, year = {2023}, editor = {Lourentzou, Ismini and Wu, Joy and Kashyap, Satyananda and Karargyris, Alexandros and Celi, Leo Anthony and Kawas, Ban and Talathi, Sachin}, volume = {210}, series = {Proceedings of Machine Learning Research}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v210/elfares23a/elfares23a.pdf}, url = {https://proceedings.mlr.press/v210/elfares23a.html} }
2. 

   Facial Composite Generation with Iterative Human Feedback

   Florian Strohm, Ekta Sood, Dominike Thomas, Mihai Bâce, Andreas Bulling

   Proc. The 1st Gaze Meets ML workshop, PMLR, pp. 165–183, 2023.

   Abstract Links BibTeX Project

   We propose the first method in which human and AI collaborate to iteratively reconstruct the human’s mental image of another person’s face only from their eye gaze. Current tools for generating digital human faces involve a tedious and time-consuming manual design process. While gaze-based mental image reconstruction represents a promising alternative, previous methods still assumed prior knowledge about the target face, thereby severely limiting their practical usefulness. The key novelty of our method is a collaborative, it- erative query engine: Based on the user’s gaze behaviour in each iteration, our method predicts which images to show to the user in the next iteration. Results from two human studies (N=12 and N=22) show that our method can visually reconstruct digital faces that are more similar to the mental image, and is more usable compared to other methods. As such, our findings point at the significant potential of human-AI collaboration for recon- structing mental images, potentially also beyond faces, and of human gaze as a rich source of information and a powerful mediator in said collaboration.

   Paper: strohm23_gmml.pdf

   Paper Access: https://proceedings.mlr.press/v210/strohm23a.html

   @inproceedings{strohm23_gmml, title = {Facial Composite Generation with Iterative Human Feedback}, author = {Strohm, Florian and Sood, Ekta and Thomas, Dominike and B{\^a}ce, Mihai and Bulling, Andreas}, booktitle = {Proc. The 1st Gaze Meets ML workshop, PMLR}, pages = {165--183}, year = {2023}, editor = {Lourentzou, Ismini and Wu, Joy and Kashyap, Satyananda and Karargyris, Alexandros and Celi, Leo Anthony and Kawas, Ban and Talathi, Sachin}, volume = {210}, series = {Proceedings of Machine Learning Research}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v210/strohm23a/strohm23a.pdf}, url = {https://proceedings.mlr.press/v210/strohm23a.html} }

Here are some related news you might like to read next:

• Paper accepted at GAZE
• Two papers accepted at NeurIPS and CoNLL