Category Archives: Psycho-physiological Bases Of Engineering

On rewards and values when the RL theory is applied to human brain

September 16, 2019 16:04 ,

Keno Juechems, Christopher Summerfield, Where Does Value Come From?. Trends in Cognitive Sciences, Volume 23, Issue 10, 2019, Pages 836-850, DOI: 10.1016/j.tics.2019.07.012.

The computational framework of reinforcement learning (RL) has allowed us to both understand biological brains and build successful artificial agents. However, in this opinion, we highlight open challenges for RL as a model of animal behaviour in natural environments. We ask how the external reward function is designed for biological systems, and how we can account for the context sensitivity of valuation. We summarise both old and new theories proposing that animals track current and desired internal states and seek to minimise the distance to a goal across multiple value dimensions. We suggest that this framework readily accounts for canonical phenomena observed in the fields of psychology, behavioural ecology, and economics, and recent findings from brain-imaging studies of value-guided decision-making.

Posted in: Psycho-physiological bases of engineering , Tagged:

On the integer numbers in the brain

September 16, 2019 16:01 ,

Susan Carey, David Barner, Ontogenetic Origins of Human Integer Representations. Trends in Cognitive Sciences, Volume 23, Issue 10, 2019, Pages 823-835, DOI: 10.1016/j.tics.2019.07.004.

Do children learn number words by associating them with perceptual magnitudes? Recent studies argue that approximate numerical magnitudes play a foundational role in the development of integer concepts. Against this, we argue that approximate number representations fail both empirically and in principle to provide the content required of integer concepts. Instead, we suggest that children\u2019s understanding of integer concepts proceeds in two phases. In the first phase, children learn small exact number word meanings by associating words with small sets. In the second phase, children learn the meanings of larger number words by mastering the logic of exact counting algorithms, which implement the successor function and Hume\u2019s principle (that one-to-one correspondence guarantees exact equality). In neither phase do approximate number representations play a foundational role.

Posted in: Psycho-physiological bases of engineering , Tagged:

On the role and limitations of motor internal simulation as a way of predicting the effects of a future action in the brain

September 16, 2019 15:46 ,

Myrthel Dogge, Ruud Custers, Henk Aarts, Moving Forward: On the Limits of Motor-Based Forward Models. Trends in Cognitive Sciences, Volume 23, Issue 9, 2019, Pages 743-753, DOI: 10.1016/j.tics.2019.06.008.

The human ability to anticipate the consequences that result from action is an essential building block for cognitive, emotional, and social functioning. A dominant view is that this faculty is based on motor predictions, in which a forward model uses a copy of the motor command to predict imminent sensory action-consequences. Although this account was originally conceived to explain the processing of action-outcomes that are tightly coupled to bodily movements, it has been increasingly extrapolated to effects beyond the body. Here, we critically evaluate this generalization and argue that, although there is ample evidence for the role of predictions in the processing of environment-related action-outcomes, there is hitherto little reason to assume that these predictions result from motor-based forward models.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

Numerosity in animals (insects)

September 16, 2019 15:43 ,

Martin Giurfa, An Insect\u2019s Sense of Number. Trends in Cognitive Sciences, Volume 23, Issue 9, 2019, Pages 720-722, DOI: 10.1016/j.tics.2019.06.010.

Recent studies revealed numerosity judgments in bees, which include the concept of zero, subtraction and addition, and matching symbols to numbers. Despite their distant origins, bees and vertebrates share similarities in their numeric competences, thus suggesting that numerosity is evolutionary conserved and can be implemented in miniature brains without neocortex.

Posted in: Psycho-physiological bases of engineering , Tagged:

On theories of human decision making and the role of affects

June 24, 2019 11:07 ,

Ian D. Roberts, Cendri A. Hutcherson, Affect and Decision Making: Insights and Predictions from Computational Models, Trends in Cognitive Sciences,
Volume 23, Issue 7, 2019, Pages 602-614 DOI: 10.1016/j.tics.2019.04.005.

In recent years interest in integrating the affective and decision sciences has skyrocketed. Immense progress has been made, but the complexities of each field, which can multiply when combined, present a significant obstacle. A carefully defined framework for integration is needed. The shift towards computational modeling in decision science provides a powerful basis and a path forward, but one whose synergistic potential will only be fully realized by drawing on the theoretical richness of the affective sciences. Reviewing research using a popular computational model of choice (the drift diffusion model), we discuss how mapping concepts to parameters reduces conceptual ambiguity and reveals novel hypotheses.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

A brief (and relatively shallow) account of computer programming as a cognitive ability

June 24, 2019 10:57 ,

Evelina Fedorenko, Anna Ivanova, Riva Dhamala, Marina Umaschi Bers, The Language of Programming: A Cognitive Perspective, Trends in Cognitive Sciences,
Volume 23, Issue 7, 2019, Pages 525-528 DOI: 10.1016/j.tics.2019.04.010.

Computer programming is becoming essential across fields. Traditionally grouped with science, technology, engineering, and mathematics (STEM) disciplines, programming also bears parallels to natural languages. These parallels may translate into overlapping processing mechanisms. Investigating the cognitive basis of programming is important for understanding the human mind and could transform education practices.

Posted in: Computer science, Psycho-physiological bases of engineering

The concepts of agency and ownership in cognitive maps, and a nice survey of cognitive maps

May 24, 2019 11:00 ,

Shahar Arzy, Daniel L. Schacter, Self-Agency and Self-Ownership in Cognitive Mapping, Trends in Cognitive Sciences, Volume 23, Issue 6, 2019, Pages 476-487, DOI: 10.1016/j.tics.2019.04.003.

The concepts of agency of one’s actions and ownership of one’s experience have proved useful in relating body representations to bodily consciousness. Here we apply these concepts to cognitive maps. Agency is defined as ‘the sense that I am the one who is generating the experience represented on a cognitive map’, while ownership is defined as ‘the sense that I am the one who is undergoing an experience, represented on a cognitive map’. The roles of agency and ownership are examined with respect to the transformation between egocentric and allocentric representations and the underlying neurocognitive and computational mechanisms; and within the neuropsychiatric domain, including Alzheimer’s disease (AD) and other memory-related disorders, in which the senses of agency and ownership may be disrupted.

Posted in: Psycho-physiological bases of engineering , Tagged: , ,

A nice (short) survey of deep RL

May 10, 2019 11:43 ,

Matthew Botvinick, Sam Ritter, Jane X. Wang, Zeb Kurth-Nelson, Charles Blundell, Demis Hassabis, Reinforcement Learning, Fast and Slow, Trends in Cognitive Sciences, Volume 23, Issue 5, 2019, Pages 408-422 DOI: 10.1016/j.tics.2019.02.006.

Deep reinforcement learning (RL) methods have driven impressive advances in artificial intelligence in recent years, exceeding human performance in domains ranging from Atari to Go to no-limit poker. This progress has drawn the attention of cognitive scientists interested in understanding human learning. However, the concern has been raised that deep RL may be too sample-inefficient – that is, it may simply be too slow – to provide a plausible model of how humans learn. In the present review, we counter this critique by describing recently developed techniques that allow deep RL to operate more nimbly, solving problems much more quickly than previous methods. Although these techniques were developed in an AI context, we propose that they may have rich implications for psychology and neuroscience. A key insight, arising from these AI methods, concerns the fundamental connection between fast RL and slower, more incremental forms of learning.

Posted in: Psycho-physiological bases of engineering, Reinforcement learning theory , Tagged: ,

Measuring unconscious behaviour

May 10, 2019 11:39 ,

David Soto, Usman Ayub Sheikh, Clive R. Rosenthal, A Novel Framework for Unconscious Processing, Trends in Cognitive Sciences, Volume 23, Issue 5, 2019, Pages 372-376 DOI: 10.1016/j.tics.2019.03.002.

Understanding the distinction between conscious and unconscious cognition remains a priority in psychology and neuroscience. A comprehensive neurocognitive account of conscious awareness will not be possible without a sound framework to isolate and understand unconscious information processing. Here, we provide a brain-based framework that allows the identification of unconscious processes, even with null effects on behaviour.

Posted in: Cognitive sciences, Psycho-physiological bases of engineering

On how attention, modelled by bayesian inference (for category learning), can structure the way reinforcement learning works

March 20, 2019 10:18 ,

Angela Radulescu, Yael Niv, Ian Ballard, Holistic Reinforcement Learning: The Role of Structure and Attention, Trends in Cognitive Sciences, Volume 23, Issue 4, 2019, Pages 278-292, DOI: 10.1016/j.tics.2019.01.010.

Compact representations of the environment allow humans to behave efficiently in a complex world. Reinforcement learning models capture many behavioral and neural effects but do not explain recent findings showing that structure in the environment influences learning. In parallel, Bayesian cognitive models predict how humans learn structured knowledge but do not have a clear neurobiological implementation. We propose an integration of these two model classes in which structured knowledge learned via approximate Bayesian inference acts as a source of selective attention. In turn, selective attention biases reinforcement learning towards relevant dimensions of the environment. An understanding of structure learning will help to resolve the fundamental challenge in decision science: explaining why people make the decisions they do.

Posted in: Psycho-physiological bases of engineering , Tagged: ,