Ongoing projects

Part A: Meta-Projects

META 2.1: Non-Symbolic Representation in Situated Cognition. Describing Sensorimotor Representations via Embodied Similarity Spaces

Project Leader: Peter Brössel

PhD-Student: N.N.

Abstract: Symbolic representation and the computation with such representations take center stage in traditional approaches in cognitive science (Fodor 1975, 1983). In contrast, embodied theories of cognition deny such a privileged role for symbolic representation and instead maintain that sensorimotor information and by extension the continuous interaction with the environment play a more important role in our cognitive household. Anti-representationalists advocates of embodied cognition maintain that most of our cognitive capacities, and even our decisions and actions can be explained with the help of dynamical system theory or connectionist neural networks and without presupposing mental representation, let alone symbolic representation. Other advocates of embodied cognition hold that mental representations have an essential role to play.

The main aim of this project is to investigate whether conceptual spaces (Gärdenfors 2000, 2014) provide an adequate framework for integrating representational states into theories of embodied cognition. In particular, embodied similarity spaces are an alternative to both connectionism and symbolic mental representation. On the basis of such geometrically conceived similarity spaces we hope to understand both (i) how similarity spaces are shaped by sensorimotor information and constant interaction with the environment and (ii) how such embodied representation shape our thoughts and beliefs and decisions and actions without symbolic representation.



META 2.2: Situated Cognition is Scaffolded and Flexible. Comparing the Cognition of Humans and Animals

Project Leader: Albert Newen

PhD-Student: Maja Griem

Abstract: The standard way to characterize situated cognition consists in the 4Es: embodied, enacted, embedded, and extended (Newen et al. 2018). During the discussion of the concepts, the scientific community including our own contributions reached a status according to which embodiment has a twofold but clear meaning and enactment denotes the role of dispositions to act for all cognitive states and processes. Both concepts are thus rather clear and have in the meantime received a lot of detailed support including highlighting some limitations. The notions of extendedness and embeddedness are less clear and remain underspecified since it is not indicating any borderline between standard background conditions and specific enabling conditions for a cognitive ability or process. To change this, we want to argue that these two concepts should be substituted by the concept of „scaffolding“ (Sterelny 2003; Varga 2018, Stephan & Walter forthcoming) (resulting in the trias of embodiment, enactment, and scaffolding) and the general observation of flexible use of learned cognitive abilities. The working hypothesis is that human cognition is especially scaffolded by physical and social conditions and it is integrated into an early development of flexible use. This will be investigated with the philosophical project including a comparison of the realization condition two core abilities, e.g. tool use and nonlinguistic communication, in humans and some species of nonhuman animals.



META 2.3: 4E Cognition, Biologism and Functionalism

Project Leader: Tobias Schlicht

PhD-Student: Bartosz Radomski

Abstract: Functionalism has been the orthodox background framework driving cognitive science since the 1960s (e.g. Putnam 1965). But ever since Varela’s pioneering work launching the enactive movement in cognitive science (Varela et al. 1991; Chemero 2009), the notion of autopoiesis (self-organization) has played a major role. According to autopoietic enactivism, the self-organizational structure of organisms constitutes the biological roots of intentionality, traditionally conceived as the directedness or aboutness of cognitive processes and states (Thompson 2007). Exemplified even by a single cell, the self-organizational structure of living beings is taken to be characteristic of cognitive systems, yielding the mind-life-continuity thesis, which is accepted by many leading enactivists (see also Noë 2009).

But taking this starting point raises new and challenging questions concerning our conceptualization of cognition and the possibility of artificial intelligence:

  1. How, given this biologistic starting point, is cognition in artificial systems possible? This has always been a project at the heart of cognitive science and we should provide room for this possibility even though we may learn about the nature of biological cognition by appealing to the enactive framework. But allowing for cognition in artificial systems requires a more flexible, functionalist framework of cognition rather than a biologistic one. This challenge is even more surprising considering the fact that Brooks’ work in robotics, using subsumption architectures, has been a driving force in establishing embodied enactive cognitive science. The task of this project is to investigate to what extend a unique version of functionalism can pay respect to both horns of this dilemma.
  2. What is the contribution of neural processing and brains in general to cognition if cognition is ultimately rooted in all living systems, even those without brains and nervous systems? In this context it is necessary to ask to what extend are the two equally popular explanatory frameworks of 4E cognition and predictive processing compatible? (Hohwy 2018; Schlicht 2018)



META 2.4: Analyzing the Concept of Affective Scaffolding and its Application: Extending the Boundaries of Emotion Regulation

Project Leader: Sabrina Coninx

Abstract: The environment plays a decisive ongoing role in our affective life by shaping what we feel and how we feel in certain situations (Krueger & Colombetti 2018). This is true for healthy people as well as people suffering from mental disorders (Varga 2019). However, we are not at mercy of these contextual factors but we can actively influence and use them in order to stabilize our emotions: we can partly offload processes of emotional self-regulation on the environment (Krueger 2018; Stephan & Walter forthcoming). For example, we may remove all items from our house that remind us of a lost love in order to avoid negative feelings of sorrow. In contrast, after the death of a loved one we may create specific locations, such as graves or shrines which provide a defined space for our grief. These forms of affective scaffolding concerning the manipulation of the material environment constitute an essential aspect of our everyday life. The project addresses two central questions in the subject area of affective scaffolding. First, the project aims to investigate such concept from a theoretical perspective. Hereby, the question is in the focus of how we can conceptualize affective scaffolding particularly in its relation to other related concepts, such as the embeddedness and extendedness of emotions. Second, the project aims to apply the concept of affective scaffolding to an innovative field of research which has so far not received much attention: virtual realities. Computer and video games seem to constitute driving mechanisms of emotional regulation in modern societies as they allow gamers to actively engage in a multiplicity of virtual environments. It appears of utter important to understand (a) how games shape our emotions, (b) how gamers actively choose games to regulate their emotions, (c) how games are designed in order to provide such opportunity and (d) how we can make use of such knowledge to employ games in therapeutic measures, for example.



Part B: Situated Agency and Its Evaluation

AGEN 2.1: Embodied Agency in Pigeons: The Integration of a New Arm into a Body Schema

Project Leader: Onur Güntürkün

PhD-Student: Guillermo Hidalgo Gadea

Abstract: Tool use and also meta-tool use is often observed in primates (Proffitt et al. 2016) and parrots (Goodman et al. 2018; Taylor et al. 2007). There is evidence from monkeys that tools can be embodied and can alter the neural representations (Iriki et al. 1996; Maravita et al. 2004; Quallo et al. 2009). However, in all of these examples the tools are mere elongations of existing body parts like beaks or arms.

Pigeons also occasionally use tools (Epstein & Medalie 1983) but usually as an extension of their beak. The body schema of birds does not foresee the possession of arms. What happens when a pigeon starts to wear an artificial robotic arm? Will it habituate to it as a mere nuisance? Or will it use it like an external tool but only under conditions for which it had gathered experience? Or will it embody it such that it starts to anticipate how to spontaneously alter its own body movements in novel situations in order to optimally exploit the options of the robotic arm for gathering food? In addition to detailed behavioral pattern analyses using machine learning, we will also employ molecular imaging to correlate our behavioral data with plasticity-related brain activity patterns. This study has the potential for a detailed analysis of the concept of embodied agency since, a) it uses an evolutionarily unforeseen but highly functional body extension in a non-human animal; b) it allows for detailed behavioral analysis of the process of learning and transfer to novel conditions, and; c) bridges action patterns to neural plasticity.



AGEN 2.2: Action Control – Is the Automatic Approach Bias Embodied?

Project Leaders: Sven Walter and Peter König

PhD-Student: Johannes Solzbacher

Abstract: Addicts exhibit an automatic approach bias to drug-related cues. In Approach Avoidance Tasks (AAT) in which they have to use a joystick to move cues either away from them or to-wards them, they approach rather than avoid drug-related cues, and they approach them faster than they avoid them. This automatic bias can be modified through a joystick-based AAT: During cognitive bias modification subjects are asked to push drug-related cues away, while pulling neutral cues towards them. Even after short training periods, this has a profound effect on their automatic approach bias (e.g. Wiers et al. 2011). However, in part due to the lack of a conceptually sound philosophical framework, the mechanisms behind this phenomenon are not yet understood (e.g. Fridland & Wiers 2018). In particular, a general appraisal of approach- and avoidance-movements as positive and negative can arguably not be the full explanation, contrary to what is typically assumed: For while stretching out one’s arm is negatively valenced when a harmful object is pushed away, it is positively valenced when one reaches for nourishing food. This not only calls for further conceptual work, but also for variations in the experimental approach. The project fills this lacuna using the virtual reality (VR) labs in Osnabrück. Unlike real life joystick-based AATs, in VR participants can react with movements whose valence is unambiguous (say ‘smashing’ negative cues with the fist while ‘hugging’ positive ones). Based on conceptual work regarding philosophical accounts of embodiment and the available empirical literature on AATs, the project develops a corresponding VR-environment for further research.



AGEN 2.3: Situated Evaluation of Actions: Acceptability of Decisions by Autonomous Agents

Project Leaders: Peter König and Achim Stephan

PhD-Student: Shadi Derakhshan

Abstract: In previous research we investigated human decisions in dilemma situations and the acceptability of ethical decisions of autonomous agents in such situations. Building on this work we here investigate human behavior in joint task involving human avatars as well as artificial autonomous agents. We investigate the moral evaluation of decisions made by an autonomous car in traffic situations involving dilemmata: does the traffic situation with autonomous cars lead to specific criteria guiding our moral evaluations? Is there a general framework in place or do we rely on a situated framework especially for moral decisions with autonomous cars? The experimental setup uses a large, open source, scriptable and modular VR city (Westdrive). Specifically, we will use gaze behavior (eye and head direction) towards relevant objects in symmetrical (multiple object tracking) as well as asymmetrical tasks (driver, passenger). Pupil dilation is used to evaluate the cognitive load in these situations. Further, by means of EEG we will investigate attention processes (alpha activity, fixation onset ERP, ERSP of ACC and TPJ). Exploratory analysis will cover areas of the mirror system. This study allows scrutinizing subject’s self-assessments and subject’s interaction with autonomous agents with objective behavioral data. Further, it tests whether identical neuronal circuits are involved in human-AI interactions as in human-human interactions.



AGEN 2.4: Enacted Cognition in Pretending the Future: Possible Implications for Proactive Problem Solving in Preschool-Aged Children

Project Leaders: Babett Voigt and Silvia Schneider

PhD-Student: Felix Schreiber

Abstract: Children during the preschool years become more and more able to prepare for a problem that will only occur during a future event (Munakata et al. 2012). The transition from reactive to proactive problem solving is seen as a milestone of child development (Pauen et al. 2016). Hence, research on how to foster proactive problem solving is highly warranted. Pretend play may be one promising candidate. During pretend play children imagine what another person or they themselves may think and feel at another place and time and enact their ideas accordingly (Lillard et al. 2013, for a review). Enacting imaginations about a possible upcoming problem may thus enhance proactive problem solving (Suddendorf et al. 2010, 2013; Hudson et al. 2011). Research on this question is still lacking, however.

The planned project aims to elucidate the conditions under which pretend play may facilitate proactive problem solving. Specifically, we will address the following questions: Is spontaneous pretend play more helpful than prompted pretend play? Is role play from a first-person view more helpful than symbolic play from a third-person view? What phenomenological features and which content of pretend play relate to proactive problem solving? Is it more helpful to enact the positive consequences of solving the problem or to enact the negative consequences of failure? In all three experiments, preschool-aged children´s problem solving behaviour is observed in a standardized laboratory setting after (a) a pretend play phase, or (b) after an interview about the upcoming problem or (c) after having the opportunity to play with some unrelated toys.



Part C: Situated Emotions and Social Cognition

SESC 2.1: Embodied Empathy: Hormonal and Olfaction Basis of Empathy

Project Leader: Ursula Stockhorst

PhD-Student: Katrin Heyers

Abstract: In Part 2 of the RTG, we will further elucidate the role of steroidal sex hormones for cognitive and affective empathy of complex emotions and widen this perspective by (a) investigating the role of olfactory perception and (b) addressing the effects of psychological stress on both empathy components.

Emotion recognition, and/or empathy are key aspects of perceiving, understanding and feeling the emotional states of others and constitute a basic ability for social interaction. Empathy typically entails two components: cognitive empathy (ability to infer another person’s mental state) and affective empathy (responding with a suitable affect to others’ emotional state) (Drimalla et al. 2019). From comparing women in menstrual-cycle stages and/or using oral contraceptives, we know that basic-emotion recognition varies with situational factors (valence, emotion type, gender of the protagonist) while interacting with sex-hormone status. Importantly, we here extend the scope to analyzing empathy for complex emotions which are only seldom addressed thus far.

Neurobiological studies in rodents reveal that genomic and mainly non-genomic actions of estrogen at brain estrogen receptors (ERs) modify social recognition (identification of conspecifics), social preferences and social learning (Ervin et al. 2015). Interestingly, rodent’s social cognition (e. g. recognition of conspecifics) also relies on chemosensory information in an ER-related way. Examining the role of olfactory performance for empathy in humans is rare and correlational. First data reveal e.g., a positive correlation between odor identification and affective empathy, or an association between odor-/pheromone-based identification of roommates and odor identification with cognitive empathy in females. Interestingly, social-network size was positively correlated with olfactory performance (Zou et al. 2016). Based on this background, we here examine the role of olfactory performance for cognitive and affective empathy in humans while addressing sex-hormone levels and social-network parameters. Additionally, we investigate the effects of experimental stress (via noradrenergic and cortisol inducing stressors) on cognitive and emotional empathy (Wolf et al. 2015) while considering the so-far neglected role of steroidal sex-hormones in this field and the role of olfactory performance.



SESC 2.2: Affective Scaffolding by Extremist Groups

Project Leaders: Achim Stephan

PhD-Student: Hina Haq


Terrorism in its various forms is one of the big challenges for democratic societies in our times. To develop strategies to prevent terrorist attacks it is of utmost importance to first comprehend how particularly young people get radicalized and drive radicalization into violent behavior; second, to consider and counter the measures that radical groups use to recruit new members, to retain individuals or to avoid that these individuals abandon the group; and third, to provide strategies to both detect young adults and adolescents that could be recruited and to detect recruiters.

There is no simple explanation to why and how a person is radicalized. Radicalization is a multi-factor, multi-pathway and complex process (e.g., McCauley and Moskalenko, 2008; Hafez and Mullins, 2015). Thus far, many models and metaphors describing the process of radicalization have been presented by various researchers in (social) psychology, mostly interpreting it as a progression over a period of time and involving different factors and dynamics in individual and collective settings (for a deeper introduction, see Haq et al. 2020).

The research group of Stephan has a strong background in the philosophy of cognitive and affective science with a stress on recent developments in so-called situated affectivity (Stephan 2018, Stephan & Walter 2020) and started over the last years to apply this perspective to features of radicalization (Haq et al. 2020, Valentini et al 2020). For this project, the group provides—in a novel way—philosophical analyses of the environmental scaffoldings of emotional processes and self-stimulated affective loops in radicalization processes. Many radical organizations provide structures for affective bonding and the development of affective loops between the recruits and other members of the organization. Among these scaffolds are opportunities to fulfill emotionally laden desires such as power, status, revenge, and identity based on relevant trainings and rituals which make the recruits to create deep affective bonding to the group. The PhD candidate should analyze first-hand interviews, narratives and profiles from de-radicalization centers and from internet databases to investigate the impact of affective patterns in different face-to face and in online environments. We will investigate specific emotions, different ideological and material approaches radical organizations use to form and maintain the emotional bond with the recruits. Furthermore, she will consider the impact both other people (such as leaders, influencers, family members, peers, and enlisters) and the psychological situation of the recruit have on their radicalization process. In addition, we will study how affective patterns evolve during the process of radicalization. One member of the team (Hina Haq) provides first-hand experience of working in a de-radicalization center in Pakistan and is able to screen materials published in Urdu, Pashto, and English.



Part D: Situated Linguistic Understanding and Communication

LING 2.1: Effects of Enacted Communication on Meaning: Co-speech Gestures and Non-Verbal Signaling

Project Leader: Markus Werning

PhD-Student: Ludmila Reimer

Abstract: According to the received view, the meaning of a sentence is compositionally determined by its syntactic structure and the lexical meanings of the words that figure therein (Werning 2004, 2005; Werning, Hinzen & Machery 2012). In face-to-face communication, however, meaning is often enacted. How the listener interprets the speaker’s communicative intentions also depends on co-speech gestures and non-verbal signaling (McNeill 1992). Their contributions to meaning may regard different linguistic levels: (i) the lexicon: e.g., uttering “John is sitting on an article” + wiggling up and down with fingers lets us interpret the expression sit on as typewriting (Dick et al. 2009); (ii) syntax/logical form: e.g., headshaking accompanying a sentence is interpreted as negation (Fricke 2012); (iii) information structure/focus: eyebrow raising or pitch accent on a word raises logical alternatives (Ebert, Evert & Wilmes 2011). Enactment directly feeds into sentence meaning composition and immediately affects truth conditions (Cosentino, Baggio, Kontinen & Werning 2017). Using an established correlation between surprisal and the N400-component in the event-related potential (Kuperberg & Jaeger 2016; Spychalska, Kontinen, Noveck, Reimer & Werning 2018; Spychalska, Kontinen & Werning 2016), we will develop predictions of a Bayesian model of linguistic enrichment (Werning & Cosentino 2017; Werning, Unterhuber & Wiedemann 2019), test them in a series of EEG experiments and adjust our theory accordingly.



LING 2.2: The Evolutionary Roots of Situated Communication: The Impact of the Social Environment on Communicative Interactions of Chimpanzees (Pan Troglodytes)

Project Leader: Simone Pika

PhD-Student: Kayla Kolff

Abstract: Situated approaches on cognition postulate that cognition is primarily based on reciprocal real-time interactions of embodied agents with their environments (Stephan, Walter & Wilutzky 2014). Several scholars have used communication as a window onto how cognition is extended by aspects of the social environment (e.g. Eibl-Eibesfeldt 1988; Cutting & Dunn 2006). However, relatively little is known about the evolutionary roots of situated communication and the existing studies have mainly been focusing on vocalizations of nonhuman animals (Seyfarth, Cheney & Marler 1980; Pepperberg 1981; Slocombe & Zuberbühler 2005). To enable a more holistic understanding of the impact of the social environment on communicative interactions, the planned project will investigate turn-taking interactions of one of humans’ closest living relative, the chimpanzees (Pan troglodytes schweinfurthii), living in the Kibale National Park in Uganda. We will apply a unique combination of computational modelling and behavioral observations, enabling to test the computational model outcome against real-world data on chimpanzee communication.

The following research questions will be addressed:

  1. Are turn-taking interactions of chimpanzees influenced by the social context? To address this question, we will focus on communicative (bi-modal, gestural, and vocal) turn-taking events and turn-taking actions (e.g. play). We will investigate whether turn-taking interactions are distributed equally across social contexts or whether they mainly occur in contexts characterized by cooperation (such as for instance food sharing, grooming, leaving together). The context will be operationalized by activity of interacting individuals before, during and after the exchange, the desired outcome on behalf of the signaler (operationalized by examining whether a recipient’s reaction satisfied the signaler, shown by cessation of communication on the part of the signaler (Hobaiter & Byrne 2014), and assessments of visibility and distance between individuals.
  2. Are turn-taking interactions of chimpanzees influenced by social matrices? To address this question, we will examine whether turn-taking interactions occur in specific dyads only (e.g. closely bonded individuals, allies), and whether the elements involved in turn-taking events differ with regards to individual matrices (e.g. age, relatedness, personality, interaction experience, etc.). We will adopt seminal measures by Seyfarth and colleagues (Seyfarth, Silk & Cheney 2012) developed to determine personalities in wild-living baboons (Papio hamadryas ursinus) and adapt them to chimpanzees.



LING 2.3: Language as a Cognitive Scaffold – On the Cognitive Functions of Inner Speech

Project Leader: Nikola Kompa

PhD-Student: Francesco Fanti Rovetta

Abstract: Recent scholarship emphasizes the scaffolding role of language for cognition. Language is conceived of as a cognitive niche (Clark 2006a), a programming tool for cognition (Lupyan & Bergen 2016), a material artifact (Clark 2006b), or a neuroenhancement (Dove 2019). Yet how exactly does language serve this function? The aim of the project is to spell out, with a certain degree of detail, the idea that language scaffolds impressive cognitive achievements.

One way of approaching the topic is by recourse to the work of Vygotsky and his followers on private and inner speech. Vygotsky and neo-Vygotskian scholars emphasize the beneficial role of language for cognitive development, in particular its role in organizing, directing, and controlling behavior (Vygotsky 1986; Vygotsky & Luria 1994). Children first acquire signs in social interaction, then learn to direct speech at themselves and to thereby control their behavior and organize their thought. Thus, language is claimed to be first acquired as an external symbol system before then becoming internalized and re-used as a cognitive tool. To date, there is a growing body of evidence that private or inner speech increases children’s performance in various problem-solving tasks and enhances cognitive control (Winsler et al. 2009; Diaz & Berg 1992).

The aim of the project is to carefully conceptualize key notions and to then develop a theoretically coherent as well as empirically informed account of private and inner speech which also allows for the generation of hypotheses about the cognitive functions they may serve. To that end, it will be necessary to first distinguish inner speech from related phenomena (Hurlburt et al. 2013; Fernyhough 2016; Gauker 2019), such as unsymbolized thinking, inner hearing or the auditory imagery of speech, motivational self-talk, etc. In a second step, the Vygotskian approach as well as more recent psychological and philosophical literature on inner and private speech needs to be critically reviewed. This will put us in a position to develop, in a third, step, a model of the cognitive functions inner and private speech might serve (cf. Alderson-Day & Fernyhough 2015 for a sketch) and to generate, in a fourth step, hypotheses that could then be put to empirical test.