Category Archives: Psycho-physiological Bases Of Engineering

Interesting hypothesis about how cognitive abilities can be modelled with closed control loops that run in parallel -using hierarchies of abstraction and prediction-, traditionally used just for low-level behaviours

June 8, 2017 08:48 ,

Giovanni Pezzulo, Paul Cisek, Navigating the Affordance Landscape: Feedback Control as a Process Model of Behavior and Cognition, Trends in Cognitive Sciences, Volume 20, Issue 6, June 2016, Pages 414-424, ISSN 1364-6613, DOI: 10.1016/j.tics.2016.03.013.

We discuss how cybernetic principles of feedback control, used to explain sensorimotor behavior, can be extended to provide a foundation for understanding cognition. In particular, we describe behavior as parallel processes of competition and selection among potential action opportunities (‘affordances’) expressed at multiple levels of abstraction. Adaptive selection among currently available affordances is biased not only by predictions of their immediate outcomes and payoffs but also by predictions of what new affordances they will make available. This allows animals to purposively create new affordances that they can later exploit to achieve high-level goals, resulting in intentional action that links across multiple levels of control. Finally, we discuss how such a ‘hierarchical affordance competition’ process can be mapped to brain structure.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

Physiological evidences that visual attention is based on predictions

June 8, 2017 08:37 ,

Martin Rolfs, Martin Szinte, Remapping Attention Pointers: Linking Physiology and Behavior, Trends in Cognitive Sciences, Volume 20, Issue 6, 2016, Pages 399-401, ISSN 1364-6613, DOI: 10.1016/j.tics.2016.04.003.

Our eyes rapidly scan visual scenes, displacing the projection on the retina with every move. Yet these frequent retinal image shifts do not appear to hamper vision. Two recent physiological studies shed new light on the role of attention in visual processing across saccadic eye movements.

Posted in: Psycho-physiological bases of engineering , Tagged:

“Nexting” (predicting events that occur next, possibly at different time scales) implemented in a robot through temporal difference learning and with a large number of learners

June 7, 2017 11:27 ,

Joseph Modayil, Adam White, Richard S. Sutton (2011), Multi-timescale Nexting in a Reinforcement Learning Robot, arXiv:1112.1133 [cs.LG]. ARXIV, (this version to appear in the Proceedings of the Conference on the Simulation of Adaptive Behavior, 2012).

The term “nexting” has been used by psychologists to refer to the propensity of people and many other animals to continually predict what will happen next in an immediate, local, and personal sense. The ability to “next” constitutes a basic kind of awareness and knowledge of one’s environment. In this paper we present results with a robot that learns to next in real time, predicting thousands of features of the world’s state, including all sensory inputs, at timescales from 0.1 to 8 seconds. This was achieved by treating each state feature as a reward-like target and applying temporal-difference methods to learn a corresponding value function with a discount rate corresponding to the timescale. We show that two thousand predictions, each dependent on six thousand state features, can be learned and updated online at better than 10Hz on a laptop computer, using the standard TD(lambda) algorithm with linear function approximation. We show that this approach is efficient enough to be practical, with most of the learning complete within 30 minutes. We also show that a single tile-coded feature representation suffices to accurately predict many different signals at a significant range of timescales. Finally, we show that the accuracy of our learned predictions compares favorably with the optimal off-line solution.

Posted in: Applications of reinforcement learning to robots, Psycho-physiological bases of engineering , Tagged: ,

Theoretical models for explaining the human (quick) decicion-making process

April 20, 2016 16:42 ,

Roger Ratcliff, Philip L. Smith, Scott D. Brown, Gail McKoon, Diffusion Decision Model: Current Issues and History, Trends in Cognitive Sciences, Volume 20, Issue 4, April 2016, Pages 260-281, ISSN 1364-6613, DOI: 10.1016/j.tics.2016.01.007.

There is growing interest in diffusion models to represent the cognitive and neural processes of speeded decision making. Sequential-sampling models like the diffusion model have a long history in psychology. They view decision making as a process of noisy accumulation of evidence from a stimulus. The standard model assumes that evidence accumulates at a constant rate during the second or two it takes to make a decision. This process can be linked to the behaviors of populations of neurons and to theories of optimality. Diffusion models have been used successfully in a range of cognitive tasks and as psychometric tools in clinical research to examine individual differences. In this review, we relate the models to both earlier and more recent research in psychology.

Posted in: Psycho-physiological bases of engineering , Tagged:

The diverse roles of the hippocampus

April 20, 2016 16:16 ,

Daniel Bendor, Hugo J. Spiers, Does the Hippocampus Map Out the Future?, Trends in Cognitive Sciences, Volume 20, Issue 3, March 2016, Pages 167-169, ISSN 1364-6613, DOI: 10.1016/j.tics.2016.01.003.

Decades of research have established two central roles of the hippocampus – memory consolidation and spatial navigation. Recently, a third function of the hippocampus has been proposed: simulating future events. However, claims that the neural patterns underlying simulation occur without prior experience have come under fire in light of newly published data.

Posted in: Psycho-physiological bases of engineering , Tagged:

It seems that the human motor cortex not only contains a map of motions but a map of basic behaviors (compositions of motions)

February 24, 2016 19:04 ,

Michael S.A. Graziano, Ethological Action Maps: A Paradigm Shift for the Motor Cortex, Trends in Cognitive Sciences, Volume 20, Issue 2, February 2016, Pages 121-132, ISSN 1364-6613, DOI: 10.1016/j.tics.2015.10.008.

The map of the body in the motor cortex is one of the most iconic images in neuroscience. The map, however, is not perfect. It contains overlaps, reversals, and fractures. The complex pattern suggests that a body plan is not the only organizing principle. Recently a second organizing principle was discovered: an action map. The motor cortex appears to contain functional zones, each of which emphasizes an ethologically relevant category of behavior. Some of these complex actions can be evoked by cortical stimulation. Although the findings were initially controversial, interest in the ethological action map has grown. Experiments on primates, mice, and rats have now confirmed and extended the earlier findings with a range of new methods.

Posted in: Psycho-physiological bases of engineering , Tagged:

How mood influcences learning, concretely perception of rewards in the context of reinforcement learning

February 4, 2016 16:34 ,

Eran Eldar, Robb B. Rutledge, Raymond J. Dolan, Yael Niv, Mood as Representation of Momentum, Trends in Cognitive Sciences, Volume 20, Issue 1, January 2016, Pages 15-24, ISSN 1364-6613, DOI: j.tics.2015.07.010.

Experiences affect mood, which in turn affects subsequent experiences. Recent studies suggest two specific principles. First, mood depends on how recent reward outcomes differ from expectations. Second, mood biases the way we perceive outcomes (e.g., rewards), and this bias affects learning about those outcomes. We propose that this two-way interaction serves to mitigate inefficiencies in the application of reinforcement learning to real-world problems. Specifically, we propose that mood represents the overall momentum of recent outcomes, and its biasing influence on the perception of outcomes ‘corrects’ learning to account for environmental dependencies. We describe potential dysfunctions of this adaptive mechanism that might contribute to the symptoms of mood disorders.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

Modelling emotions in adaptive agents through the action selection part of reinforcement learning, plus some references on the neurophysiological bases of RL and a good review of literature on emotions

October 5, 2015 09:08 ,

Joost Broekens , Elmer Jacobs , Catholijn M. Jonker, A reinforcement learning model of joy, distress, hope and fear, Connection Science, Vol. 27, Iss. 3, 2015, DOI: 10.1080/09540091.2015.1031081.

In this paper we computationally study the relation between adaptive behaviour and emotion. Using the reinforcement learning framework, we propose that learned state utility, V(s), models fear (negative) and hope (positive) based on the fact that both signals are about anticipation of loss or gain. Further, we propose that joy/distress is a signal similar to the error signal. We present agent-based simulation experiments that show that this model replicates psychological and behavioural dynamics of emotion. This work distinguishes itself by assessing the dynamics of emotion in an adaptive agent framework – coupling it to the literature on habituation, development, extinction and hope theory. Our results support the idea that the function of emotion is to provide a complex feedback signal for an organism to adapt its behaviour. Our work is relevant for understanding the relation between emotion and adaptation in animals, as well as for human–robot interaction, in particular how emotional signals can be used to communicate between adaptive agents and humans.

Posted in: Psycho-physiological bases of engineering, Reinforcement learning in AI , Tagged: , , , , ,

On how the human cognition detects regularities in noisy sensory data (“Statistical learning” in psychology terms)

October 1, 2015 18:01 ,

Annabelle Goujon, André Didierjean, Simon Thorpe, Investigating implicit statistical learning mechanisms through contextual cueing, Trends in Cognitive Sciences, Volume 19, Issue 9, September 2015, Pages 524-533, ISSN 1364-6613, DOI: 10.1016/j.tics.2015.07.009.

Since its inception, the contextual cueing (CC) paradigm has generated considerable interest in various fields of cognitive sciences because it constitutes an elegant approach to understanding how statistical learning (SL) mechanisms can detect contextual regularities during a visual search. In this article we review and discuss five aspects of CC: (i) the implicit nature of learning, (ii) the mechanisms involved in CC, (iii) the mediating factors affecting CC, (iv) the generalization of CC phenomena, and (v) the dissociation between implicit and explicit CC phenomena. The findings suggest that implicit SL is an inherent component of ongoing processing which operates through clustering, associative, and reinforcement processes at various levels of sensory-motor processing, and might result from simple spike-timing-dependent plasticity.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

Quantum probability theory as an alternative to classical (Kolgomorov) probability theory for modelling human decision making processes, and a curious description of the effect of a particular ordering of decisions in the complete result

July 21, 2015 11:45 ,

Peter D. Bruza, Zheng Wang, Jerome R. Busemeyer, Quantum cognition: a new theoretical approach to psychology, Trends in Cognitive Sciences, Volume 19, Issue 7, July 2015, Pages 383-393, ISSN 1364-6613, DOI: 10.1016/j.tics.2015.05.001.

What type of probability theory best describes the way humans make judgments under uncertainty and decisions under conflict? Although rational models of cognition have become prominent and have achieved much success, they adhere to the laws of classical probability theory despite the fact that human reasoning does not always conform to these laws. For this reason we have seen the recent emergence of models based on an alternative probabilistic framework drawn from quantum theory. These quantum models show promise in addressing cognitive phenomena that have proven recalcitrant to modeling by means of classical probability theory. This review compares and contrasts probabilistic models based on Bayesian or classical versus quantum principles, and highlights the advantages and disadvantages of each approach.

Posted in: Probability theories and interpretations, Psycho-physiological bases of engineering , Tagged: , , ,