Category Archives: Cognitive Sciences

On the roots in the ability to control outcomes of human motivation

September 20, 2017 08:35 ,

Justin M. Moscarello, Catherine A. Hartley, Agency and the Calibration of Motivated Behavior, Trends in Cognitive Sciences, Volume 21, Issue 10, 2017, Pages 725-735, DOI: 10.1016/j.tics.2017.06.008.

The controllability of positive or negative environmental events has long been recognized as a critical factor determining their impact on an organism. In studies across species, controllable and uncontrollable reinforcement have been found to yield divergent effects on subsequent behavior. Here we present a model of the organizing influence of control, or a lack thereof, on the behavioral repertoire. We propose that individuals derive a generalizable estimate of agency from controllable and uncontrollable outcomes, which serves to calibrate their behavioral strategies in a manner that is most likely to be adaptive given their prior experience.

Posted in: Psycho-physiological bases of engineering , Tagged:

Evidence of the dicotomy reactive/predictive control in the brain

August 26, 2017 11:45 ,

Mattie Tops, Markus Quirin, Maarten A.S. Boksem, Sander L. Koole, Large-scale neural networks and the lateralization of motivation and emotion, International Journal of Psychophysiology, Volume 119, 2017, Pages 41-49, DOI: 10.1016/j.ijpsycho.2017.02.004.

Several lines of research in animals and humans converge on the distinction between two basic large-scale brain networks of self-regulation, giving rise to predictive and reactive control systems (PARCS). Predictive (internally-driven) and reactive (externally-guided) control are supported by dorsal versus ventral corticolimbic systems, respectively. Based on extant empirical evidence, we demonstrate how the PARCS produce frontal laterality effects in emotion and motivation. In addition, we explain how this framework gives rise to individual differences in appraising and coping with challenges. PARCS theory integrates separate fields of research, such as research on the motivational correlates of affect, EEG frontal alpha power asymmetry and implicit affective priming effects on cardiovascular indicators of effort during cognitive task performance. Across these different paradigms, converging evidence points to a qualitative motivational division between, on the one hand, angry and happy emotions, and, on the other hand, sad and fearful emotions. PARCS suggests that those two pairs of emotions are associated with predictive and reactive control, respectively. PARCS theory may thus generate important new insights on the motivational and emotional dynamics that drive autonomic and homeostatic control processes.

Posted in: Psycho-physiological bases of engineering , Tagged: , ,

On how the simplification on physics made in computer games for real-time execution can explain the simplification on physics made by infants when understanding the world

August 17, 2017 11:39 ,

Tomer D. Ullman, Elizabeth Spelke, Peter Battaglia, Joshua B. Tenenbaum, Mind Games: Game Engines as an Architecture for Intuitive Physics, Trends in Cognitive Sciences, Volume 21, Issue 9, 2017, Pages 649-665, DOI: 10.1016/j.tics.2017.05.012.

We explore the hypothesis that many intuitive physical inferences are based on a mental physics engine that is analogous in many ways to the machine physics engines used in building interactive video games. We describe the key features of game physics engines and their parallels in human mental representation, focusing especially on the intuitive physics of young infants where the hypothesis helps to unify many classic and otherwise puzzling phenomena, and may provide the basis for a computational account of how the physical knowledge of infants develops. This hypothesis also explains several ‘physics illusions’, and helps to inform the development of artificial intelligence (AI) systems with more human-like common sense.

Posted in: Probability and statistics, Psycho-physiological bases of engineering , Tagged:

An interesting soft-partition method based on hierarchical graphs (trees, actually) applied to topic detection in documents

August 9, 2017 09:37 ,

Peixian Chen, Nevin L. Zhang, Tengfei Liu, Leonard K.M. Poon, Zhourong Chen, Farhan Khawar, Latent tree models for hierarchical topic detection, Artificial Intelligence, Volume 250, 2017, Pages 105-124, DOI: 10.1016/j.artint.2017.06.004.

We present a novel method for hierarchical topic detection where topics are obtained by clustering documents in multiple ways. Specifically, we model document collections using a class of graphical models called hierarchical latent tree models (HLTMs). The variables at the bottom level of an HLTM are observed binary variables that represent the presence/absence of words in a document. The variables at other levels are binary latent variables that represent word co-occurrence patterns or co-occurrences of such patterns. Each latent variable gives a soft partition of the documents, and document clusters in the partitions are interpreted as topics. Latent variables at high levels of the hierarchy capture long-range word co-occurrence patterns and hence give thematically more general topics, while those at low levels of the hierarchy capture short-range word co-occurrence patterns and give thematically more specific topics. In comparison with LDA-based methods, a key advantage of the new method is that it represents co-occurrence patterns explicitly using model structures. Extensive empirical results show that the new method significantly outperforms the LDA-based methods in term of model quality and meaningfulness of topics and topic hierarchies.

Posted in: Artificial Intelligence , Tagged: , ,

POMDPs with multicriteria in the cost to optimize – a hierarchical approach

August 7, 2017 09:25 ,

Seyedshams Feyzabadi, Stefano Carpin, Planning using hierarchical constrained Markov decision processes, Autonomous Robots, Volume 41, Issue 8, pp 1589–1607, DOI: 10.1007/s10514-017-9630-4.

Constrained Markov decision processes offer a principled method to determine policies for sequential stochastic decision problems where multiple costs are concurrently considered. Although they could be very valuable in numerous robotic applications, to date their use has been quite limited. Among the reasons for their limited adoption is their computational complexity, since policy computation requires the solution of constrained linear programs with an extremely large number of variables. To overcome this limitation, we propose a hierarchical method to solve large problem instances. States are clustered into macro states and the parameters defining the dynamic behavior and the costs of the clustered model are determined using a Monte Carlo approach. We show that the algorithm we propose to create clustered states maintains valuable properties of the original model, like the existence of a solution for the problem. Our algorithm is validated in various planning problems in simulation and on a mobile robot platform, and we experimentally show that the clustered approach significantly outperforms the non-hierarchical solution while experiencing only moderate losses in terms of objective functions.

Posted in: Artificial Intelligence, Probability and statistics, Robot task planning , Tagged: , ,

Reinterpretation of evolutionary processes as algorithms for Bayesian inference

June 30, 2017 11:56 ,

Jordan W. Suchow, David D. Bourgin, Thomas L. Griffiths, Evolution in Mind: Evolutionary Dynamics, Cognitive Processes, and Bayesian Inference, Trends in Cognitive Sciences, Volume 21, Issue 7, July 2017, Pages 522-530, ISSN 1364-6613, DOI: 10.1016/j.tics.2017.04.005.

Evolutionary theory describes the dynamics of population change in settings affected by reproduction, selection, mutation, and drift. In the context of human cognition, evolutionary theory is most often invoked to explain the origins of capacities such as language, metacognition, and spatial reasoning, framing them as functional adaptations to an ancestral environment. However, evolutionary theory is useful for understanding the mind in a second way: as a mathematical framework for describing evolving populations of thoughts, ideas, and memories within a single mind. In fact, deep correspondences exist between the mathematics of evolution and of learning, with perhaps the deepest being an equivalence between certain evolutionary dynamics and Bayesian inference. This equivalence permits reinterpretation of evolutionary processes as algorithms for Bayesian inference and has relevance for understanding diverse cognitive capacities, including memory and creativity.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

A good intro about actor-critic and decision making without model on MDPs

June 30, 2017 11:31 ,

J. Wang and I. C. Paschalidis, “An Actor-Critic Algorithm With Second-Order Actor and Critic,” in IEEE Transactions on Automatic Control, vol. 62, no. 6, pp. 2689-2703, June 2017.DOI: 10.1109/TAC.2016.2616384.

Actor-critic algorithms solve dynamic decision making problems by optimizing a performance metric of interest over a user-specified parametric class of policies. They employ a combination of an actor, making policy improvement steps, and a critic, computing policy improvement directions. Many existing algorithms use a steepest ascent method to improve the policy, which is known to suffer from slow convergence for ill-conditioned problems. In this paper, we first develop an estimate of the (Hessian) matrix containing the second derivatives of the performance metric with respect to policy parameters. Using this estimate, we introduce a new second-order policy improvement method and couple it with a critic using a second-order learning method. We establish almost sure convergence of the new method to a neighborhood of a policy parameter stationary point. We compare the new algorithm with some existing algorithms in two applications and demonstrate that it leads to significantly faster convergence.

Posted in: Artificial Intelligence , Tagged: , ,

Evidences that the human brain has quantifying properties -i.e., ability to discriminate between sets of different sizes- as a result of evolution, but that numerical cognition is a result of culture

June 30, 2017 11:05 ,

Rafael E. Núñez, Is There Really an Evolved Capacity for Number?, Trends in Cognitive Sciences, Volume 21, Issue 6, June 2017, Pages 409-424, ISSN 1364-6613, DOI: 10.1016/j.tics.2017.03.005.

Humans and other species have biologically endowed abilities for discriminating quantities. A widely accepted view sees such abilities as an evolved capacity specific for number and arithmetic. This view, however, is based on an implicit teleological rationale, builds on inaccurate conceptions of biological evolution, downplays human data from non-industrialized cultures, overinterprets results from trained animals, and is enabled by loose terminology that facilitates teleological argumentation. A distinction between quantical (e.g., quantity discrimination) and numerical (exact, symbolic) cognition is needed: quantical cognition provides biologically evolved preconditions for numerical cognition but it does not scale up to number and arithmetic, which require cultural mediation. The argument has implications for debates about the origins of other special capacities – geometry, music, art, and language.

Posted in: Psycho-physiological bases of engineering , Tagged:

Reinforcement learning to learn the model of the world intrinsically motivated

June 27, 2017 12:10 ,

Todd Hester, Peter Stone, Intrinsically motivated model learning for developing curious robots, Artificial Intelligence, Volume 247, June 2017, Pages 170-186, ISSN 0004-3702, DOI: 10.1016/j.artint.2015.05.002.

Reinforcement Learning (RL) agents are typically deployed to learn a specific, concrete task based on a pre-defined reward function. However, in some cases an agent may be able to gain experience in the domain prior to being given a task. In such cases, intrinsic motivation can be used to enable the agent to learn a useful model of the environment that is likely to help it learn its eventual tasks more efficiently. This paradigm fits robots particularly well, as they need to learn about their own dynamics and affordances which can be applied to many different tasks. This article presents the texplore with Variance-And-Novelty-Intrinsic-Rewards algorithm (texplore-vanir), an intrinsically motivated model-based RL algorithm. The algorithm learns models of the transition dynamics of a domain using random forests. It calculates two different intrinsic motivations from this model: one to explore where the model is uncertain, and one to acquire novel experiences that the model has not yet been trained on. This article presents experiments demonstrating that the combination of these two intrinsic rewards enables the algorithm to learn an accurate model of a domain with no external rewards and that the learned model can be used afterward to perform tasks in the domain. While learning the model, the agent explores the domain in a developing and curious way, progressively learning more complex skills. In addition, the experiments show that combining the agent’s intrinsic rewards with external task rewards enables the agent to learn faster than using external rewards alone. We also present results demonstrating the applicability of this approach to learning on robots.

Posted in: Applications of reinforcement learning to robots, Artificial Intelligence , Tagged: ,

State of the art and historical background of the classical divergence between AI and robotics

June 27, 2017 12:03 ,

Kanna Rajan, Alessandro Saffiotti, Towards a science of integrated AI and Robotics, Artificial Intelligence, Volume 247, June 2017, Pages 1-9, ISSN 0004-3702, DOI: 10.1016/j.artint.2017.03.003.

The early promise of the impact of machine intelligence did not involve the partitioning of the nascent field of Artificial Intelligence. The founders of AI envisioned the notion of embedded intelligence as being conjoined between perception, reasoning and actuation. Yet over the years the fields of AI and Robotics drifted apart. Practitioners of AI focused on problems and algorithms abstracted from the real world. Roboticists, generally with a background in mechanical and electrical engineering, concentrated on sensori-motor functions. That divergence is slowly being bridged with the maturity of both fields and with the growing interest in autonomous systems. This special issue brings together the state of the art and practice of the emergent field of integrated AI and Robotics, and highlights the key areas along which this current evolution of machine intelligence is heading.

Posted in: Artificial Intelligence, Robotics , Tagged: