Projects of the Second Cohort (2020–2023)
Part A: Meta-Projects
META 2.1: Non-Symbolic Representation in Situated Cognition. Describing Sensorimotor Representations via Embodied Similarity Spaces
Project Leader: Peter Brössel
PhD-Student: Eline Kuipers
Abstract: The computation with symbolic representations takes center stage in traditional approaches in cognitive science (Fodor 1975, 1983). Embodied theories of cognition deny such a privileged role for symbolic representation. They hold that sensorimotor information and, by extension, the continuous interaction with the environment plays a more important role in understanding cognition. Anti-representationalists hold that most of our cognitive capacities can be explained with the help of dynamical systems theory or connectionist neural networks without presupposing any mental representations. Some advocates of embodied cognition hold that mental representations have an essential role to play, but they do not offer an account of such representations. This project investigates whether conceptual spaces (Brössel 2017; Gärdenfors 2000, 2014; Poth & Brössel 2020) provide an adequate framework for integrating representational states into theories of embodied cognition. The project aims at establishing embodied similarity spaces as an alternative to both connectionism and symbolic mental representation. It will study (i) how embodied similarity spaces are shaped by sensorimotor information and constant interaction with the environment and (ii) how they shape our thoughts, beliefs, decisions and actions without symbolic representation.
META 2.2: Situated Cognition is Scaffolded and Flexible. Comparing the Cognition of Humans and Animals
Project Leader: Albert Newen
PhD-Student: Maja Griem
Abstract: During the discussion of the standard 4E concepts (Newen et al. 2018), the scientific community reached a status according to which embodiment has a twofold but clear meaning and enactment denotes the role of dispositions to act for all cognitive processes. Both concepts have in the meantime received a lot of detailed support including highlighting some limitations. Since the distinction between being constitutive and being merely dependent on cannot be empirically grounded (Baumgartner & Wilutzky 2017), we give up the distinction between extendedness and embeddedness. As a substitute for describing the role of extrabodily features for cognition we suggest to use the concept of „scaffolding“ (Sterelny 2003, 2010; Varga 2019; Stephan & Walter 2020) (This leads to the trias of embodiment, enactment and environmental scaffolding). We aim to develop a multidimensional concept of scaffolding which can be used as theoretical basis for the whole RTG. This is relevant since human cognition is especially scaffolded by physical and social conditions and scaffolds are especially integrated into an early development of flexible cognition. The role of scaffolds will be investigated in this philosophical project with a particular focus on a comparison of the realization conditions of two abilities, i.e. tool use and nonlinguistic communication, in humans and animals.
META 2.3: 4E Cognition, Biologism and Functionalism: Adaptivity as a Prerequisite for Cognition
Project Leader: Tobias Schlicht
PhD-Student: Bartosz Radomski
Abstract: “There can be no cognition without adaptivity”. This bold claim is a cornerstone of two major philosophical schools: autopoietic enactivism (henceforth AE) (Maturana & Varela, 1980) and the free-energy principle (henceforth FEP) (Friston, 2010). Both AE and FEP promote a view that the capacity to adapt is a root of further properties usually associated with cognition, such as normativity, intentionality, and learning.
Adaptivity refers to systemic processes of responding to emerging circumstances in the external environment or internal rhythms within the system. Both AE and FEP use the concept of adaptivity but focus on its different aspects. According to AE, adaptivity is an ability to sustain and realize autopoiesis, while according to FEP, it is an ability to minimize the long-term average of free-energy.
The aim of this project is to elucidate the notion of “adaptivity”, enabling the development of an argument for adaptivity’s inherent connection with cognition. The working assumption is that it can be achieved by integrating AE and FEP. By combining the structural account (AE) and the functional account (FEP), the goal is to obtain a unified understanding of adaptivity that can (a) be simultaneously applied across a broad range of philosophical topics in philosophy of mind, and (b) put to use in fields like situated robotics, considering cognition in biological as well as artificial systems. This project is closely connected to AGEN 2.1, which inspects the adaptivity of pigeons regarding the integration of new limbs into their body schemas.
META 2.4: Analyzing the Concept of Affective Scaffolding and its Application: Extending the Boundaries of Emotion Regulation
Project Leader: Sabrina Coninx
Abstract: The environment plays a decisive role in our affective life by shaping what we feel and how we feel (Krueger & Colombetti 2018). This is true for healthy people as well as people suffering from mental disorders (Varga 2019). At the same time, we can actively use contextual factors in order to stabilize our emotions: we can partly offload processes of emotional self-regulation onto the environment (Stephan & Walter 2020). These forms of affective scaffolding constitute an essential aspect of our everyday life and the project addresses two central issues in this subject area. First, the project aims to investigate such a concept from a purely theoretical perspective. Hereby, the focal question is of how we can conceptualize affective scaffolding particularly in its relation to similar concepts, such as affordances and the social embedding of emotions. The second aim is to discuss the concept of affective scaffolding as fruitful for the innovative field of virtual realities. It is utterly important for our modern society to understand (a) how computer and video games shape our emotions, (b) how gamers actively choose games to regulate their emotions, (c) how games are designed in order to provide such opportunity, and (d) how we can make use of such knowledge to employ games in therapeutic measures.
Part B: Situated Agency and Its Evaluation
AGEN 2.1: Embodied Agency in Pigeons: The Integration of a New Arm into a Body Schema
Project Leader: Onur Güntürkün
PhD-Student: Guillermo Hidalgo Gadea
Abstract: Tool use and meta-tool use is often observed in primates (Proffitt et al. 2016) and parrots (Goodman et al. 2018; Taylor et al. 2007). There is evidence from monkeys that tools can be embodied and can alter neural representations (Iriki et al. 1996; Maravita et al. 2004; Quallo et al. 2009). However, in all of these examples, the tools are mere elongations of existing body parts like beaks or arms. Pigeons also occasionally use tools (Epstein & Medalie 1983) but usually as an extension of their beak. The body schema of birds does not foresee the possession of arms. What happens when a pigeon starts to wear an artificial robotic arm? Will it habituate to it as a mere nuisance? Or will it use it like an external tool – under conditions for which it had gathered experience? Or will it embody it such that it starts to anticipate how to spontaneously alter its own body movements in novel situations in order to optimally exploit the options of the robotic arm for gathering food? In addition to detailed behavioral pattern analyses using machine learning, we will also employ molecular imaging to correlate our behavioral data with plasticity-related brain activity patterns. This study has the potential for a detailed analysis of the concept of embodied agency since, a) it uses an evolutionarily unforeseen but highly functional body extension in a non-human animal; b) it allows for detailed behavioral analysis of the process of learning and transfer to novel conditions, and; c) bridges action patterns to neural plasticity.
AGEN 2.2: Action Control – Is the Automatic Approach Bias Embodied?
Project Leaders: Sven Walter and Peter König
PhD-Student: Johannes Solzbacher
Abstract: The valence of a stimulus is tightly coupled to the (re)actions by subjects. This applies to the influence of stimulus valence and mood on eye movements (Kaspar et al. 2014) as well as to observations in the Approach Avoidance Task. Here, addicts move cues either away from them or towards them. They approach rather than avoid these drug-related cues, and they approach them faster than they avoid them. Even after short training periods, this has a profound effect on their automatic approach bias (e.g. Wiers et al. 2011), which opens the prospect of therapeutic applications. However, in part due to the lack of a conceptually sound philosophical account and the variation in technical implementations that often use rather abstract actions, the data base is far from clear and the mechanisms behind this phenomenon are not yet understood (Fridland & Wiers 2018). In recent submitted work we could demonstrate that congruent bodily interaction with images in the priming phase fosters visual interaction in the subsequent exploration phase. The project takes the next step and uses a virtual reality lab where participants react with movements whose valence is natural and unambiguous. Further, this is paired with conceptual work regarding philosophical accounts of embodied agency.
AGEN 2.3: Situated Evaluation of Actions: Acceptability of Decisions by Autonomous Agents
Project Leaders: Peter König and Achim Stephan
PhD-Student: Shadi Derakhshan
Abstract: In previous research we investigated human decisions in dilemma situations and the acceptability of ethical decisions of autonomous agents in such situations (Faulhaber et al. 2019; Kallioinen et al. 2019; Sütfeld et al. 2017; Sütfeld et al. 2019). Here, building on this work, we investigate human behavior in a joint task involving human avatars as well as artificial autonomous agents. We investigate the moral evaluation of decisions made by an autonomous car in traffic situations involving dilemmata: does the traffic situation with autonomous cars lead to specific criteria guiding our moral evaluations? Is there a general framework in place or do we rely on a situated framework especially for moral decisions with autonomous cars? The experimental setup uses a large, open source, scriptable and modular VR city (Nezami et al. 2020). Specifically, we will use gaze behavior towards relevant objects in symmetrical as well as asymmetrical tasks. Pupil dilation is used to evaluate the cognitive load in these situations. Further, by means of EEG we will investigate attention processes. Exploratory analysis will cover areas of the mirror system. This study allows scrutinizing subject’s self-assessments and subject’s interaction with autonomous agents with objective behavioral data.
AGEN 2.4: Enacted Cognition in Pretending the Future – Implications for Proactive Behavior in Preschool Children
Project Leaders: Babett Voigt and Silvia Schneider
PhD-Student: Felix Schreiber
Abstract: As adults we take control of our live and often act in anticipation of upcoming problems or opportunities. For example, we practice our musical skills to succeed in a competition or look for new job opportunities long before the current position ends. In contrast, children usually tend to live in the here and now. We still know little about what helps them to become more proactive and to take control of their own future (Pauen et al. 2016; Munakata et al. 2012)? The present project aims to gain novel insights into this question. Two points seem to be interesting in this respect.
First, acting in advance requires to imagine how a specific event unfolds and what consequences it may have (episodic foresight; e.g., Suddendorf & Corballis, 2007, Suddendorf et al., 2011). Drawing on adult research, the present project examines two ways in which episodic foresight may foster self-initiated proactivity in children: (a) Children may be more motivated to prepare if they anticipate possible affective consequences of the event that are more or less desirable. (b) Thinking of the spatial-temporal features of the event may help children to become aware of an upcoming problem and the possible means to solve it.
Second, simple forms of episodic foresight are already in place in preschool-age (Hudson et al., 2011), but children still seem to struggle to vividly pre-feel the hedonic consequences of an event and to imagine the spatial-temporal context in sufficient detail (Gott & Lah, 2014; Redshaw & Suddendorf, 2013). This may be one reason why they rarely act in advance on their own in everyday life. Enactment may help them to overcome these shortcomings. Different lines of research support this idea. For example, bodily expressing emotional states has been shown to affect the way that one thinks about emotional states (Niedenthal, 2007) and so embodying anticipated emotions may help to pre-feel and to think about emotional consequences of a future event (Gilbert & Wilson, 2007). Further, taking the perspective of somebody else has been shown to benefit from role play in adults and pretend play in children (Furumi & Koyasu, 2014; Gillepsie, 2012; Lillard et al., 2013) and may thus also facilitate to take the perspective of the future self. In addition, acting during thinking elicits representational change (Boncoddo, Dixon & Kelley, 2010), fosters an episodic-like structure of narratives (Koc & Ilgaz, 2005; Hostetter & Alibali, 2019) and thus may also enhance a detailed, episodic imagination of the future that goes beyond the immediate present.
Bringing together those diverse research fields, the present project aims to investigate whether and how (enacted) episodic foresight helps preschool-aged children to behave proactively. In general, this project has the potential to improve our understanding of how enactment fosters autonomous and adaptive behavior in children by supporting developing complex cognition.
Part C: Situated Emotions and Social Cognition
SESC 2.1: Effects of Hormone-Related Embodiment on Emotional and Cognitive Systems: Brain Network Activity Under Acute Stress
Project Leader: Ursula Stockhorst and Onur Güntürkün
PhD-Student: Katrin Heyers
Abstract: We will further elucidate the role of steroid hormones for cognition and emotion. We will widen this perspective by (a) focusing on neural networks assessed with resting-state fMRI and (b) investigating the role of cortisol increase after psychosocial stress. Acute stress is a key modulator of negative emotions and has been shown to lead to dynamic adaptations in large-scale brain networks: this is an embodiment of cognitive processes by the hormonal system. Specifically, acute stress leads to a re-allocation of cognitive resources from the executive control network to the so-called salience network that stimulates vigilance and fear (Hermans et al. 2014). Given this background, we will examine the effect of acute stress on resting state network activity and the relation of these networks to cognitive and affective parameters, while addressing stress-hormone levels (Ocklenburg et al. 2016). To this end, we will use the ScanSTRESS paradigm (Lederbogen et al. 2011) to induce acute social-evaluative stress in the MRI-scanner and investigate its effects (via noradrenergic and steroid hormonal pathways) on brain network activity, emotion and cognition. In addition, we will assess the effect of stress on neural networks related to cognitive and emotional empathy but also on the behavioral correlates, as they indicate that acute stress could enhance emotional empathy (Wolf et al. 2015).
SESC 2.2: Affective Scaffolding by Extremist Groups
Project Leaders: Achim Stephan
PhD-Student: Hina Haq
Abstract: Terrorism is one of the big challenges in our times. To develop strategies to prevent terrorist attacks it is important to comprehend how people get radicalized and to consider radical groups’ measures of recruitment. Many models in (social) psychology describe radicalization as a complex process (e.g. Hafez & Mullins 2015; McCauley & Moskaleno 2008) involving different factors and dynamics in individual and collective settings. The group of Stephan—having a strong background in situated affectivity (Stephan 2018; Stephan & Walter 2020)—started to apply this perspective to features of radicalization (Haq et al. 2020; Valentini et al. 2020). In a novel way, the project aims at studying the environmental scaffoldings of emotional processes in radicalization. Radical organizations provide structures for affective bonding and the development of affective loops between the recruits and the group. Among these are opportunities to fulfill emotionally laden desires such as power, revenge, status, and identity based on relevant trainings and rituals. We will analyze first-hand interviews, narratives and profiles from de-radicalization centers and from internet databases to investigate how affective patterns evolve during the process of radicalization in face-to face and online environments.
Part D: Situated Linguistic Understanding and Communication
LING 2.1: Effects of Enacted Communication on Meaning: Co-speech Gestures and Non-Verbal Signaling
Project Leader: Markus Werning
PhD-Student: Ludmila Reimer
Abstract: According to the received view, the meaning of a sentence is compositionally determined by its syntactic structure and the lexical meanings of the words that figure therein (Hinzen et al. 2012; Werning 2004, 2005). In face-to-face communication, however, meaning is often enacted. How the listener interprets the speaker’s communicative intentions also depends on co-speech gestures and non-verbal signaling (McNeill et al. 1994). Their contributions to meaning may regard different linguistic levels: (i) the lexicon: e.g., uttering “John is sitting on an article” + wiggling up and down with fingers lets us interpret the expression sit on as typewriting (Dick et al. 2009); (ii) syntax/logical form: e.g., headshaking accompanying a sentence is interpreted as negation in many cultures (Fricke 2012); (iii) information structure/focus: eyebrow raising or pitch accent on a word raises logical alternatives (Ebert et al. 2011). Enactment directly feeds into sentence meaning composition and immediately affects truth conditions (Cosentino et al. 2017). Using an established correlation between surprisal and the N400-component in the event-related potential (Kuperberg & Jaeger 2016; Spychalska et al. 2016, 2019) we will develop predictions of a Bayesian model of linguistic enrichment (Werning & Cosentino 2017; Werning et al. 2019), test them in a series of EEG studies and adjust our theory.
LING 2.2: The Evolutionary Roots of Situated Communication: The Impact of the Social Environment on Communicative Interactions of Chimpanzees (Pan Troglodytes)
Project Leader: Simone Pika
PhD-Student: Kayla Kolff
Abstract: Situated approaches on cognition postulate that cognition is primarily based on reciprocal real-time interactions of embodied agents with their environments (Stephan et al. 2014). Several scholars have used communication as a window into how cognition is extended by aspects of the social environment (e.g. Cutting & Dunn 2006; Eibl-Eibesfeldt 1988). However, relatively little is known about the evolutionary roots of situated communication and the existing studies have mainly been focusing on vocalizations of nonhuman animals (Pepperberg 1981; Seyfarth et al. 1980; Slocombe & Zuberbühler 2005); To enable a more holistic understanding of the impact of the social environment on communicative interactions, the planned project will investigate turn-taking interactions of one of humans’ closest living relative, the chimpanzee (Pan troglodytes schweinfurthii), living in the Kibale National Park in Uganda, with a special focus on the impact of social context and social matrices. We will apply a unique combination of computational modelling and behavioral observations, enabling to test the computational model outcome against real-world data on chimpanzee communication.
LING 2.3: Inner Speech as a Cognitive Scaffold – Exploring the Role of Inner Speech for Social Cognition
Project Leader: Nikola Kompa
PhD-Student: Francesco Fanti Rovetta
Abstract: Recent scholarship emphasizes the scaffolding role of language for cognition. Language is conceived of as a cognitive niche (Clark 2006a), a programming tool for cognition (Lupyan & Bergen 2016), a material artefact (Clark 2006b) or a neuroenhancement (Dove 2019). Yet how exactly does language serve this function? The aim of the project is to spell out, with a certain degree of detail, the idea that language scaffolds impressive cognitive achievements.
One way of approaching the topic is by recourse to the work of Vygotsky and his followers on private and inner speech. Vygotsky and neo-Vygotskian scholars emphasize the beneficial role of language for cognitive development, for organizing, directing, and controlling behavior (Vygotsky 1986; Vygotsky & Luria 1994) as well as for self-reflection (Morin 2019) and social understanding (Fernyhough 2009). Children first acquire signs in social interaction, then learn to direct speech at themselves and to thereby control their behavior and organize their thought. Thus, language is claimed to be first acquired as an external symbol system before then becomes internalized and re-used as a cognitive tool. To date, there is a growing body of evidence that private or inner speech increases children’s performance in problem-solving tasks and enhances cognitive control (Winsler et al. 2009; Diaz & Berk 1992;).
The aim of the project is to carefully conceptualize key notions, and to then develop a theoretically coherent as well as empirically informed account of private and inner speech which also allows for the generation of hypotheses about the cognitive functions they may serve. Particular attention will be devoted to the hypothesis that inner speech can be used to enhance social understanding, either by facilitating perspective-taking or by scaffolding the narrative interpretation of experience, or both. To that end, it will be necessary to first distinguish inner speech from related phenomena (Hurlburt et al. 2013; Fernyhough 2016; Gauker 2019such as unsymbolized thinking, inner hearing or the auditory imagery of inner speech, motivational self-talk, etc. In a second step, the Vygotskian approach as well as more recent psychological and philosophical literature on inner and private speech needs to be critically reviewed. This will put us in a position to develop, in a third, step, a model of the cognitive functions inner and private speech might serve (cf. Alderson-Day & Fernyhough 2015 for a sketch) and to generate, in a fourth step, hypotheses that could then be put to empirical test.