Category Archives: Cognitive Sciences

Adapting inverse reinforcement learning for including the risk-aversion of the agent

May 28, 2018 08:11 ,

Sumeet Singh, Jonathan Lacotte, Anirudha Majumdar, and Marco Pavone, Risk-sensitive inverse reinforcement learning via semi- and non-parametric methods , The International Journal of Robotics Research First Published May 22, 2018 DOI: 10.1177/0278364918772017.

The literature on inverse reinforcement learning (IRL) typically assumes that humans take actions to minimize the expected value of a cost function, i.e., that humans are risk neutral. Yet, in practice, humans are often far from being risk neutral. To fill this gap, the objective of this paper is to devise a framework for risk-sensitive (RS) IRL to explicitly account for a human’s risk sensitivity. To this end, we propose a flexible class of models based on coherent risk measures, which allow us to capture an entire spectrum of risk preferences from risk neutral to worst case. We propose efficient non-parametric algorithms based on linear programming and semi-parametric algorithms based on maximum likelihood for inferring a human’s underlying risk measure and cost function for a rich class of static and dynamic decision-making settings. The resulting approach is demonstrated on a simulated driving game with 10 human participants. Our method is able to infer and mimic a wide range of qualitatively different driving styles from highly risk averse to risk neutral in a data-efficient manner. Moreover, comparisons of the RS-IRL approach with a risk-neutral model show that the RS-IRL framework more accurately captures observed participant behavior both qualitatively and quantitatively, especially in scenarios where catastrophic outcomes such as collisions can occur.

Posted in: Psycho-physiological bases of engineering, Reinforcement learning in AI , Tagged:

All the information about our cognitive processes that can be deduced from our mouse movements

May 16, 2018 08:35 ,

Paul E. Stillman, Xi Shen, Melissa J. Ferguson, How Mouse-tracking Can Advance Social Cognitive Theory, Trends in Cognitive Sciences, Volume 22, Issue 6, 2018, Pages 531-543 DOI: 10.1016/j.tics.2018.03.012.

Mouse-tracking – measuring computer-mouse movements made by participants while they choose between response options – is an emerging tool that offers an accessible, data-rich, and real-time window into how people categorize and make decisions. In the present article we review recent research in social cognition that uses mouse-tracking to test models and advance theory. In particular, mouse-tracking allows examination of nuanced predictions about both the nature of conflict (e.g., its antecedents and consequences) as well as how this conflict is resolved (e.g., how decisions evolve). We demonstrate how mouse-tracking can further our theoretical understanding by highlighting research in two domains − social categorization and self-control. We conclude with future directions and a discussion of the limitations of mouse-tracking as a method.

Posted in: Cognitive sciences , Tagged:

On how sleep improves our problem-solving capabilities

May 16, 2018 08:33 ,

Penelope A. Lewis, Günther Knoblich, Gina Poe, How Memory Replay in Sleep Boosts Creative Problem-Solving, Trends in Cognitive Sciences, Volume 22, Issue 6, 2018, Pages 491-503 DOI: 10.1016/j.tics.2018.03.009.

Creative thought relies on the reorganisation of existing knowledge. Sleep is known to be important for creative thinking, but there is a debate about which sleep stage is most relevant, and why. We address this issue by proposing that rapid eye movement sleep, or ‘REM’, and non-REM sleep facilitate creativity in different ways. Memory replay mechanisms in non-REM can abstract rules from corpuses of learned information, while replay in REM may promote novel associations. We propose that the iterative interleaving of REM and non-REM across a night boosts the formation of complex knowledge frameworks, and allows these frameworks to be restructured, thus facilitating creative thought. We outline a hypothetical computational model which will allow explicit testing of these hypotheses.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

How a robot can learn to recognize itself on a mirror

April 20, 2018 07:50 ,

Zeng, Y., Zhao, Y., Bai, J. et al., Toward Robot Self-Consciousness (II): Brain-Inspired Robot Bodily Self Model for Self-Recognition, Cogn Comput (2018) 10: 307, DOI: 10.1007/s12559-017-9505-1.

The neural correlates and nature of self-consciousness is an advanced topic in Cognitive Neuroscience. Only a few animal species have been testified to be with this cognitive ability. From artificial intelligence and robotics point of view, few efforts are deeply rooted in the neural correlates and brain mechanisms of biological self-consciousness. Despite the fact that the scientific understanding of biological self-consciousness is still in preliminary stage, we make our efforts to integrate and adopt known biological findings of self-consciousness to build a brain-inspired model for robot self-consciousness. In this paper, we propose a brain-inspired robot bodily self model based on extensions to primate mirror neuron system and apply it to humanoid robot for self recognition. In this model, the robot firstly learns the correlations between self-generated actions and visual feedbacks in motion by learning with spike timing dependent plasticity (STDP), and then learns the appearance of body part with the expectation that the visual feedback is consistent with its motion. Based on this model, the robot uses multisensory integration to learn its own body in real world and in mirror. Then it can distinguish itself from others. In a mirror test setting with three robots with the same appearance, with the proposed brain-inspired robot bodily self model, each of them can recognize itself in the mirror after these robots make random movements at the same time. The theoretic modeling and experimental validations indicate that the brain-inspired robot bodily self model is biologically inspired, and computationally feasible as a foundation for robot self recognition.

Posted in: Developmental robotics, Psycho-physiological bases of engineering , Tagged:

What is Cognitive Computational Neuroscience

April 18, 2018 07:29 ,

Thomas Naselaris, Danielle S. Bassett, Alyson K. Fletcher, Konrad Kording, Nikolaus Kriegeskorte, Hendrikje Nienborg, Russell A. Poldrack, Daphna Shohamy, Kendrick Kay, Cognitive Computational Neuroscience: A New Conference for an Emerging Discipline, Trends in Cognitive Sciences, Volume 22, Issue 5, 2018, Pages 365-367, DOI: 10.1016/j.tics.2018.02.008.

Understanding the computational principles that underlie complex behavior is a central goal in cognitive science, artificial intelligence, and neuroscience. In an attempt to unify these disconnected communities, we created a new conference called Cognitive Computational Neuroscience (CCN). The inaugural meeting revealed considerable enthusiasm but significant obstacles remain.

Posted in: Cognitive sciences, Psycho-physiological bases of engineering , Tagged:

On how seeking for the lowest-cost action is not always what happens in reality

March 21, 2018 08:51 ,

Michael Inzlicht, Amitai Shenhav, Christopher Y. Olivola, The Effort Paradox: Effort Is Both Costly and Valued, Trends in Cognitive Sciences, Volume 22, Issue 4, 2018, Pages 337-349, DOI: 10.1016/j.tics.2018.01.007.

According to prominent models in cognitive psychology, neuroscience, and economics, effort (be it physical or mental) is costly: when given a choice, humans and non-human animals alike tend to avoid effort. Here, we suggest that the opposite is also true and review extensive evidence that effort can also add value. Not only can the same outcomes be more rewarding if we apply more (not less) effort, sometimes we select options precisely because they require effort. Given the increasing recognition of effort’s role in motivation, cognitive control, and value-based decision-making, considering this neglected side of effort will not only improve formal computational models, but also provide clues about how to promote sustained mental effort across time.

Posted in: Psycho-physiological bases of engineering , Tagged:

On how children learn with progressive environment changes aimed at improving their learning statistically

March 21, 2018 08:49 ,

Linda B. Smith, Swapnaa Jayaraman, Elizabeth Clerkin, Chen Yu, The Developing Infant Creates a Curriculum for Statistical Learning, Trends in Cognitive Sciences, Volume 22, Issue 4, 2018, Pages 325-336, DOI: 10.1016/j.tics.2018.02.004.

New efforts are using head cameras and eye-trackers worn by infants to capture everyday visual environments from the point of view of the infant learner. From this vantage point, the training sets for statistical learning develop as the sensorimotor abilities of the infant develop, yielding a series of ordered datasets for visual learning that differ in content and structure between timepoints but are highly selective at each timepoint. These changing environments may constitute a developmentally ordered curriculum that optimizes learning across many domains. Future advances in computational models will be necessary to connect the developmentally changing content and statistics of infant experience to the internal machinery that does the learning.

Posted in: Psycho-physiological bases of engineering , Tagged:

A theory that integrates motivation and control

March 21, 2018 08:46 ,

Giovanni Pezzulo, Francesco Rigoli, Karl J. Friston, Hierarchical Active Inference: A Theory of Motivated Control, Trends in Cognitive Sciences, Volume 22, Issue 4, 2018, Pages 294-306, DOI: 10.1016/j.tics.2018.01.009.

Motivated control refers to the coordination of behaviour to achieve affectively valenced outcomes or goals. The study of motivated control traditionally assumes a distinction between control and motivational processes, which map to distinct (dorsolateral versus ventromedial) brain systems. However, the respective roles and interactions between these processes remain controversial. We offer a novel perspective that casts control and motivational processes as complementary aspects − goal propagation and prioritization, respectively − of active inference and hierarchical goal processing under deep generative models. We propose that the control hierarchy propagates prior preferences or goals, but their precision is informed by the motivational context, inferred at different levels of the motivational hierarchy. The ensuing integration of control and motivational processes underwrites action and policy selection and, ultimately, motivated behaviour, by enabling deep inference to prioritize goals in a context-sensitive way.

Posted in: Psycho-physiological bases of engineering , Tagged:

A critic of the “two types of thinking” myth (deliberative, slow, rational, optimal vs. reactive, quick, emotional, suboptimal)

March 21, 2018 08:43 ,

David E. Melnikoff, John A. Bargh, The Mythical Number Two, Trends in Cognitive Sciences, Volume 22, Issue 4, 2018, Pages 280-293, DOI: 10.1016/j.tics.2018.02.001.

It is often said that there are two types of psychological processes: one that is intentional, controllable, conscious, and inefficient, and another that is unintentional, uncontrollable, unconscious, and efficient. Yet, there have been persistent and increasing objections to this widely influential dual-process typology. Critics point out that the ‘two types’ framework lacks empirical support, contradicts well-established findings, and is internally incoherent. Moreover, the untested and untenable assumption that psychological phenomena can be partitioned into two types, we argue, has the consequence of systematically thwarting scientific progress. It is time that we as a field come to terms with these issues. In short, the dual-process typology is a convenient and seductive myth, and we think cognitive science can do better.

Posted in: Psycho-physiological bases of engineering , Tagged:

Survey on the concept of affordance and its use in robotics (the rest of this issue of the journal also deals with affordances in robotics)

March 15, 2018 09:54 ,

L. Jamone et al, Affordances in Psychology, Neuroscience, and Robotics: A Survey,, IEEE Transactions on Cognitive and Developmental Systems, vol. 10, no. 1, pp. 4-25, March 2018, DOI: 10.1109/TCDS.2016.2594134.

The concept of affordances appeared in psychology during the late 60s as an alternative perspective on the visual perception of the environment. It was revolutionary in the intuition that the way living beings perceive the world is deeply influenced by the actions they are able to perform. Then, across the last 40 years, it has influenced many applied fields, e.g., design, human-computer interaction, computer vision, and robotics. In this paper, we offer a multidisciplinary perspective on the notion of affordances. We first discuss the main definitions and formalizations of the affordance theory, then we report the most significant evidence in psychology and neuroscience that support it, and finally we review the most relevant applications of this concept in robotics.

Posted in: Psycho-physiological bases of engineering , Tagged: ,