Category Archives: Cognitive Sciences

The quick-intuition vs. slow-deliberation dilemma from a decision-making perspective

December 22, 2015 17:36 ,

Y-Lan Boureau, Peter Sokol-Hessner, Nathaniel D. Daw, Deciding How To Decide: Self-Control and Meta-Decision Making, Trends in Cognitive Sciences, Volume 19, Issue 11, November 2015, Pages 700-710, ISSN 1364-6613, DOI: 10.1016/j.tics.2015.08.013.

Many different situations related to self control involve competition between two routes to decisions: default and frugal versus more resource-intensive. Examples include habits versus deliberative decisions, fatigue versus cognitive effort, and Pavlovian versus instrumental decision making. We propose that these situations are linked by a strikingly similar core dilemma, pitting the opportunity costs of monopolizing shared resources such as executive functions for some time, against the possibility of obtaining a better outcome. We offer a unifying normative perspective on this underlying rational meta-optimization, review how this may tie together recent advances in many separate areas, and connect several independent models. Finally, we suggest that the crucial mechanisms and meta-decision variables may be shared across domains.

Posted in: Cognitive sciences , Tagged: ,

A possible framework for the relationship between culture, behavior and the brain

December 22, 2015 17:33 ,

Shihui Han, Yina Ma, A Culture–Behavior–Brain Loop Model of Human Development, Trends in Cognitive Sciences, Volume 19, Issue 11, November 2015, Pages 666-676, ISSN 1364-6613, DOI: 10.1016/j.tics.2015.08.010.

Increasing evidence suggests that cultural influences on brain activity are associated with multiple cognitive and affective processes. These findings prompt an integrative framework to account for dynamic interactions between culture, behavior, and the brain. We put forward a culture–behavior–brain (CBB) loop model of human development that proposes that culture shapes the brain by contextualizing behavior, and the brain fits and modifies culture via behavioral influences. Genes provide a fundamental basis for, and interact with, the CBB loop at both individual and population levels. The CBB loop model advances our understanding of the dynamic relationships between culture, behavior, and the brain, which are crucial for human phylogeny and ontogeny. Future brain changes due to cultural influences are discussed based on the CBB loop model.

Posted in: Cognitive sciences , Tagged:

On how moral can shape perception

December 22, 2015 17:30 ,

Ana P. Gantman, Jay J. Van Bavel,Moral Perception, Trends in Cognitive Sciences, Volume 19, Issue 11, November 2015, Pages 631-633, ISSN 1364-6613, DOI: 10.1016/j.tics.2015.08.004.

Based on emerging research, we propose that human perception is preferentially attuned to moral content. We describe how moral concerns enhance detection of morally relevant stimuli, and both command and direct attention. These perceptual processes, in turn, have important consequences for moral judgment and behavior.

Posted in: Cognitive sciences , Tagged:

Using MDPs when the transition probability matrix is just partially specified, therefore getting closer to a model-free approach

November 12, 2015 19:42 ,

Karina V. Delgado, Leliane N. de Barros, Daniel B. Dias, Scott Sanner, Real-time dynamic programming for Markov decision processes with imprecise probabilities, Artificial Intelligence, Volume 230, January 2016, Pages 192-223, ISSN 0004-3702, DOI: 10.1016/j.artint.2015.09.005.

Markov Decision Processes have become the standard model for probabilistic planning. However, when applied to many practical problems, the estimates of transition probabilities are inaccurate. This may be due to conflicting elicitations from experts or insufficient state transition information. The Markov Decision Process with Imprecise Transition Probabilities (MDP-IPs) was introduced to obtain a robust policy where there is uncertainty in the transition. Although it has been proposed a symbolic dynamic programming algorithm for MDP-IPs (called SPUDD-IP) that can solve problems up to 22 state variables, in practice, solving MDP-IP problems is time-consuming. In this paper we propose efficient algorithms for a more general class of MDP-IPs, called Stochastic Shortest Path MDP-IPs (SSP MDP-IPs) that use initial state information to solve complex problems by focusing on reachable states. The (L)RTDP-IP algorithm, a (Labeled) Real Time Dynamic Programming algorithm for SSP MDP-IPs, is proposed together with three different methods for sampling the next state. It is shown here that the convergence of (L)RTDP-IP can be obtained by using any of these three methods, although the Bellman backups for this class of problems prescribe a minimax optimization. As far as we are aware, this is the first asynchronous algorithm for SSP MDP-IPs given in terms of a general set of probability constraints that requires non-linear optimization over imprecise probabilities in the Bellman backup. Our results show up to three orders of magnitude speedup for (L)RTDP-IP when compared with the SPUDD-IP algorithm.

See also:

  • Karina Valdivia Delgado, Scott Sanner, Leliane Nunes de Barros, Efficient solutions to factored MDPs with imprecise transition probabilities, Artif. Intell. 175 (9–10) (2011) 1498–1527.
  • Satia, J. K., and Lave Jr., R. E. 1970. MDPs with uncertain transition probabilities. Operations Research 21:728–740
  • White III, C. C., and El-Deib, H. K. 1994. MDPs with Imprecise Transition Probabilities. Operations Research 42(4):739–749
Posted in: Artificial Intelligence , Tagged: ,

Modelling emotions in adaptive agents through the action selection part of reinforcement learning, plus some references on the neurophysiological bases of RL and a good review of literature on emotions

October 5, 2015 09:08 ,

Joost Broekens , Elmer Jacobs , Catholijn M. Jonker, A reinforcement learning model of joy, distress, hope and fear, Connection Science, Vol. 27, Iss. 3, 2015, DOI: 10.1080/09540091.2015.1031081.

In this paper we computationally study the relation between adaptive behaviour and emotion. Using the reinforcement learning framework, we propose that learned state utility, V(s), models fear (negative) and hope (positive) based on the fact that both signals are about anticipation of loss or gain. Further, we propose that joy/distress is a signal similar to the error signal. We present agent-based simulation experiments that show that this model replicates psychological and behavioural dynamics of emotion. This work distinguishes itself by assessing the dynamics of emotion in an adaptive agent framework – coupling it to the literature on habituation, development, extinction and hope theory. Our results support the idea that the function of emotion is to provide a complex feedback signal for an organism to adapt its behaviour. Our work is relevant for understanding the relation between emotion and adaptation in animals, as well as for human–robot interaction, in particular how emotional signals can be used to communicate between adaptive agents and humans.

Posted in: Psycho-physiological bases of engineering, Reinforcement learning in AI , Tagged: , , , , ,

On how the human cognition detects regularities in noisy sensory data (“Statistical learning” in psychology terms)

October 1, 2015 18:01 ,

Annabelle Goujon, André Didierjean, Simon Thorpe, Investigating implicit statistical learning mechanisms through contextual cueing, Trends in Cognitive Sciences, Volume 19, Issue 9, September 2015, Pages 524-533, ISSN 1364-6613, DOI: 10.1016/j.tics.2015.07.009.

Since its inception, the contextual cueing (CC) paradigm has generated considerable interest in various fields of cognitive sciences because it constitutes an elegant approach to understanding how statistical learning (SL) mechanisms can detect contextual regularities during a visual search. In this article we review and discuss five aspects of CC: (i) the implicit nature of learning, (ii) the mechanisms involved in CC, (iii) the mediating factors affecting CC, (iv) the generalization of CC phenomena, and (v) the dissociation between implicit and explicit CC phenomena. The findings suggest that implicit SL is an inherent component of ongoing processing which operates through clustering, associative, and reinforcement processes at various levels of sensory-motor processing, and might result from simple spike-timing-dependent plasticity.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

Good review of similarity measures between elements with semantics

September 30, 2015 16:49 ,

Mohammad Taher Pilehvar, Roberto Navigli, From senses to texts: An all-in-one graph-based approach for measuring semantic similarity, Artificial Intelligence, Volume 228, November 2015, Pages 95-128, ISSN 0004-3702, DOI: 10.1016/j.artint.2015.07.005.

Quantifying semantic similarity between linguistic items lies at the core of many applications in Natural Language Processing and Artificial Intelligence. It has therefore received a considerable amount of research interest, which in its turn has led to a wide range of approaches for measuring semantic similarity. However, these measures are usually limited to handling specific types of linguistic item, e.g., single word senses or entire sentences. Hence, for a downstream application to handle various types of input, multiple measures of semantic similarity are needed, measures that often use different internal representations or have different output scales. In this article we present a unified graph-based approach for measuring semantic similarity which enables effective comparison of linguistic items at multiple levels, from word senses to full texts. Our method first leverages the structural properties of a semantic network in order to model arbitrary linguistic items through a unified probabilistic representation, and then compares the linguistic items in terms of their representations. We report state-of-the-art performance on multiple datasets pertaining to three different levels: senses, words, and texts.

Posted in: Artificial Intelligence, Graph theory, Probability and statistics , Tagged:

Extending probabilistic logic programming with continuous r.v.s, and a nice and brief introduction to programming logic and probabilistic inference

September 30, 2015 16:39 ,

Steffen Michels, Arjen Hommersom, Peter J.F. Lucas, Marina Velikova, A new probabilistic constraint logic programming language based on a generalised distribution semantics, Artificial Intelligence, Volume 228, November 2015, Pages 1-44, ISSN 0004-3702, DOI: 10.1016/j.artint.2015.06.008.

Probabilistic logics combine the expressive power of logic with the ability to reason with uncertainty. Several probabilistic logic languages have been proposed in the past, each of them with their own features. We focus on a class of probabilistic logic based on Sato’s distribution semantics, which extends logic programming with probability distributions on binary random variables and guarantees a unique probability distribution. For many applications binary random variables are, however, not sufficient and one requires random variables with arbitrary ranges, e.g. real numbers. We tackle this problem by developing a generalised distribution semantics for a new probabilistic constraint logic programming language. In order to perform exact inference, imprecise probabilities are taken as a starting point, i.e. we deal with sets of probability distributions rather than a single one. It is shown that given any continuous distribution, conditional probabilities of events can be approximated arbitrarily close to the true probability. Furthermore, for this setting an inference algorithm that is a generalisation of weighted model counting is developed, making use of SMT solvers. We show that inference has similar complexity properties as precise probabilistic inference, unlike most imprecise methods for which inference is more complex. We also experimentally confirm that our algorithm is able to exploit local structure, such as determinism, which further reduces the computational complexity.

Posted in: Artificial Intelligence, Mathematical logic, Probability and statistics , Tagged: , ,

Quantum probability theory as an alternative to classical (Kolgomorov) probability theory for modelling human decision making processes, and a curious description of the effect of a particular ordering of decisions in the complete result

July 21, 2015 11:45 ,

Peter D. Bruza, Zheng Wang, Jerome R. Busemeyer, Quantum cognition: a new theoretical approach to psychology, Trends in Cognitive Sciences, Volume 19, Issue 7, July 2015, Pages 383-393, ISSN 1364-6613, DOI: 10.1016/j.tics.2015.05.001.

What type of probability theory best describes the way humans make judgments under uncertainty and decisions under conflict? Although rational models of cognition have become prominent and have achieved much success, they adhere to the laws of classical probability theory despite the fact that human reasoning does not always conform to these laws. For this reason we have seen the recent emergence of models based on an alternative probabilistic framework drawn from quantum theory. These quantum models show promise in addressing cognitive phenomena that have proven recalcitrant to modeling by means of classical probability theory. This review compares and contrasts probabilistic models based on Bayesian or classical versus quantum principles, and highlights the advantages and disadvantages of each approach.

Posted in: Probability theories and interpretations, Psycho-physiological bases of engineering , Tagged: , , ,

Transfer learning in reinforcement learning through case-based and the use of heuristics for selecting actions

July 21, 2015 11:24 ,

Reinaldo A.C. Bianchi, Luiz A. Celiberto Jr., Paulo E. Santos, Jackson P. Matsuura, Ramon Lopez de Mantaras, Transferring knowledge as heuristics in reinforcement learning: A case-based approach, Artificial Intelligence, Volume 226, September 2015, Pages 102-121, ISSN 0004-3702, DOI: 10.1016/j.artint.2015.05.008.

The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). This meta-algorithm works in three stages: first, it uses a Reinforcement Learning step to learn a task on the source domain, storing the knowledge thus obtained in a case base; second, it does an unsupervised mapping of the source-domain actions to the target-domain actions; and, third, the case base obtained in the first stage is used as heuristics to speed up the learning process in the target domain.
A set of empirical evaluations were conducted in two target domains: the 3D mountain car (using a learned case base from a 2D simulation) and stability learning for a humanoid robot in the Robocup 3D Soccer Simulator (that uses knowledge learned from the Acrobot domain). The results attest that our transfer learning algorithm outperforms recent heuristically-accelerated reinforcement learning and transfer learning algorithms.

Posted in: Reinforcement learning in AI, Reinforcement learning theory , Tagged: , , , ,