Mind Attribution in Code

Description: As AI systems are becoming more widely applied, more humans interact with them. The perception and evaluation of AI systems does not only depend on their bevaviour but also on the language used to describe AI systems [1].

The overall goal of this project is to analyse the language used in comments of AI system implementations. For this, it is necessary to find an adequate data set of code comments (for example [2]), parse the code to find all comments (using an abstract syntax tree parser like [3]), and analyse the language used in the comments (for examply by looking for mind attribution [4, 5]). The outcome of this project is an assessment of the language used by developers to describe their AI systems. Optionally, the effect of the language on the percepetion and evaluation of AI systems could be tested in a user study, similar to [1].

Supervisor: Susanne Hindennach

Distribution: 20% Literature, 30% Implementation, 50% Analysis

Requirements: information retrieval, NLP, interest in perception of AI systems (including philosophy/linguistics)

Literature: [1] Langer, Markus, Tim Hunsicker, Tina Feldkamp, Cornelius J. König, and Nina Grgić-Hlača. “‘Look! It’s a Computer Program! It’s an Algorithm! It’s AI!’: Does Terminology Affect Human Perceptions and Evaluations of Algorithmic Decision-Making Systems?” In CHI Conference on Human Factors in Computing Systems, 1–28, 2022. https://doi.org/10.1145/3491102.3517527.

[2] Gao, Leo, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, et al. “The Pile: An 800GB Dataset of Diverse Text for Language Modeling.” arXiv, December 31, 2020. https://doi.org/10.48550/arXiv.2101.00027.

[3] https://docs.python.org/3/library/ast.html

[4] Orr, Ram Isaac, and Michael Gilead. “Development and Validation of the Mental-Physical Verb Norms (MPVN): A Text Analysis Measure of Mental State Attribution.” Behavior Research Methods, July 25, 2022. https://doi.org/10.3758/s13428-022-01911-7.

[5] Schweitzer, Shane, and Adam Waytz. “Language as a Window into Mind Perception: How Mental State Language Differentiates Body and Mind, Human and Nonhuman, and the Self from Others.” Journal of Experimental Psychology: General 150 (2021): 1642–72. https://doi.org/10.1037/xge0001013.

[6] Picture credit: https://unsplash.com/de/fotos/DwDZ5mgwhsc