Category Archives: Cognitive Sciences

Using interactive reinforcement learning with the advisor being another reinforcement learning agent

Francisco Cruz, Sven Magg, Yukie Nagai & Stefan Wermter, Improving interactive reinforcement learning: What makes a good teacher?, Connection Science, DOI: 10.1080/09540091.2018.1443318.

Interactive reinforcement learning (IRL) has become an important apprenticeship approach to speed up convergence in classic reinforcement learning (RL) problems. In this regard, a variant of IRL is policy shaping which uses a parent-like trainer to propose the next action to be performed and by doing so reduces the search space by advice. On some occasions, the trainer may be another artificial agent which in turn was trained using RL methods to afterward becoming an advisor for other learner-agents. In this work, we analyse internal representations and characteristics of artificial agents to determine which agent may outperform others to become a better trainer-agent. Using a polymath agent, as compared to a specialist agent, an advisor leads to a larger reward and faster convergence of the reward signal and also to a more stable behaviour in terms of the state visit frequency of the learner-agents. Moreover, we analyse system interaction parameters in order to determine how influential they are in the apprenticeship process, where the consistency of feedback is much more relevant when dealing with different learner obedience parameters.

Using memory of past input data to improve the convergence of NN when faced with small samples

Zhang, S., Huang, K., Zhang, R. et al., Learning from Few Samples with Memory Network, Cogn Comput (2018) 10: 15, DOI: 10.1007/s12559-017-9507-z.

Neural networks (NN) have achieved great successes in pattern recognition and machine learning. However, the success of a NN usually relies on the provision of a sufficiently large number of data samples as training data. When fed with a limited data set, a NN’s performance may be degraded significantly. In this paper, a novel NN structure is proposed called a memory network. It is inspired by the cognitive mechanism of human beings, which can learn effectively, even from limited data. Taking advantage of the memory from previous samples, the new model achieves a remarkable improvement in performance when trained using limited data. The memory network is demonstrated here using the multi-layer perceptron (MLP) as a base model. However, it would be straightforward to extend the idea to other neural networks, e.g., convolutional neural networks (CNN). In this paper, the memory network structure is detailed, the training algorithm is presented, and a series of experiments are conducted to validate the proposed framework. Experimental results show that the proposed model outperforms traditional MLP-based models as well as other competitive algorithms in response to two real benchmark data sets.

A model of others’ emotions that predicts very well experimental results

Rebecca Saxe, Seeing Other Minds in 3D, Trends in Cognitive Sciences, Volume 22, Issue 3, 2018, Pages 193-195, DOI: 10.1016/j.tics.2018.01.003.

Tamir and Thornton [1] have identified three key dimensions that organize our understanding of other minds. These dimensions (glossed as valence, social impact, and rationality) can capture the similarities and differences between concepts of internal experiences (anger, loneliness, gratitude), and also between concepts of personalities (aggressive, introverted, agreeable). Most impressively, the three dimensions explain the patterns of hemodynamic activity in our brains as we consider these experiences [2] (Box 1). States such as anger and gratitude are invisible, but the patterns evoked in our brain as we think about them are as predictable by the model of Tamir and Thornton as the patterns evoked in our visual cortex when we look at chairs, bicycles, or pineapples are predictable by models of high-level vision [3]. Human social prediction follows the same dimensions: observers predict that transitions are more likely between states that are ‘nearby’ in this abstract 3D space [4]. Thus, we expect that a friend now feeling ‘anxious’ will be more likely to feel ‘sluggish’ than ‘energetic’ later.

A model of the interdependence of previous sensorimotor experiences in the following decision making

Evelina Dineva & Gregor Schöner, How infants’ reaches reveal principles of sensorimotor decision making, Connection Science vol. 30 iss. 1, p. 53-80, DOI: 10.1080/09540091.2017.1405382.

In Piaget’s classical A-not-B-task, infants repeatedly make a sensorimotor decision to reach to one of two cued targets. Perseverative errors are induced by switching the cue from A to B, while spontaneous errors are unsolicited reaches to B when only A is cued. We argue that theoretical accounts of sensorimotor decision-making fail to address how motor decisions leave a memory trace that may impact future sensorimotor decisions. Instead, in extant neural models, perseveration is caused solely by the history of stimulation. We present a neural dynamic model of sensorimotor decision-making within the framework of Dynamic Field Theory, in which a dynamic instability amplifies fluctuations in neural activation into macroscopic, stable neural activation states that leave memory traces. The model predicts perseveration, but also a tendency to repeat spontaneous errors. To test the account, we pool data from several A-not-B experiments. A conditional probabilities analysis accounts quantitatively how motor decisions depend on the history of reaching. The results provide evidence for the interdependence among subsequent reaching decisions that is explained by the model, showing that by amplifying small differences in activation and affecting learning, decisions have consequences beyond the individual behavioural act.

A survey in interactive perception in robots: interacting with the environment to improve perception and using internal models and prediction too

J. Bohg et al, Interactive Perception: Leveraging Action in Perception and Perception in Action, IEEE Transactions on Robotics, vol. 33, no. 6, pp. 1273-1291, DOI: 10.1109/TRO.2017.2721939.

Recent approaches in robot perception follow the insight that perception is facilitated by interaction with the environment. These approaches are subsumed under the term Interactive Perception (IP). This view of perception provides the following benefits. First, interaction with the environment creates a rich sensory signal that would otherwise not be present. Second, knowledge of the regularity in the combined space of sensory data and action parameters facilitates the prediction and interpretation of the sensory signal. In this survey, we postulate this as a principle for robot perception and collect evidence in its support by analyzing and categorizing existing work in this area. We also provide an overview of the most important applications of IP. We close this survey by discussing remaining open questions. With this survey, we hope to help define the field of Interactive Perception and to provide a valuable resource for future research.

On how psychologists realize that the brain, after all, may be creating symbols (concepts), like deep neural networks show

Jeffrey S. Bowers, Parallel Distributed Processing Theory in the Age of Deep Networks, Trends in Cognitive Sciences, Volume 21, Issue 12, 2017, Pages 950-961, DOI: 10.1016/j.tics.2017.09.013.

Parallel distributed processing (PDP) models in psychology are the precursors of deep networks used in computer science. However, only PDP models are associated with two core psychological claims, namely that all knowledge is coded in a distributed format and cognition is mediated by non-symbolic computations. These claims have long been debated in cognitive science, and recent work with deep networks speaks to this debate. Specifically, single-unit recordings show that deep networks learn units that respond selectively to meaningful categories, and researchers are finding that deep networks need to be supplemented with symbolic systems to perform some tasks. Given the close links between PDP and deep networks, it is surprising that research with deep networks is challenging PDP theory.

Towards taking into account the complexity of finding the best option in decision-making systems

Peter Bossaerts, Carsten Murawski, Computational Complexity and Human Decision-Making, Trends in Cognitive Sciences, Volume 21, Issue 12, 2017, Pages 917-929, DOI: 10.1016/j.tics.2017.09.005.

The rationality principle postulates that decision-makers always choose the best action available to them. It underlies most modern theories of decision-making. The principle does not take into account the difficulty of finding the best option. Here, we propose that computational complexity theory (CCT) provides a framework for defining and quantifying the difficulty of decisions. We review evidence showing that human decision-making is affected by computational complexity. Building on this evidence, we argue that most models of decision-making, and metacognition, are intractable from a computational perspective. To be plausible, future theories of decision-making will need to take into account both the resources required for implementing the computations implied by the theory, and the resource constraints imposed on the decision-maker by biology.

On numerical cognition and the inexistence of an innate concept of number but the existence of an innate concept of quantity

Tom Verguts, Qi Chen, Numerical Cognition: Learning Binds Biology to Culture, Trends in Cognitive Sciences, Volume 21, Issue 12, 2017, Pages 913-914, DOI: 10.1016/j.tics.2017.09.004.

First, we address the issue of which quantity representations are innate. Second, we consider the role of the number list, whose characteristics are no doubt highly culturally dependent.

On the need of integrating emotions in robotic architectures

Luiz Pessoa, Do Intelligent Robots Need Emotion?,Trends in Cognitive Sciences, Volume 21, Issue 11, 2017, Pages 817-819, DOI: 10.1016/j.tics.2017.06.010.

What is the place of emotion in intelligent robots? Researchers have advocated the inclusion of some emotion-related components in the information-processing architecture of autonomous agents. It is argued here that emotion needs to be merged with all aspects of the architecture: cognitive–emotional integration should be a key design principle.

Cognitive informatics: simulation of cognition through direct simulation of neurons

Shivhare, R., Cherukuri, A.K. & Li, Establishment of Cognitive Relations Based on Cognitive Informatics, J. Cogn Comput (2017) 9: 721, DOI: 10.1007/s12559-017-9498-9.

Cognitive informatics (CI) is an interdisciplinary study on modelling of the brain in terms of knowledge and information processing. In CI, objects/attributes are considered as neurons connected to each other via synapse. The relation represents the synapse in CI. In order to represent new information the brain generates new synapse or relation between the existing neurons. Therefore, the establishment of cognitive relations is essential to represent new information. In order to represent new information, we propose an algorithm which creates cognitive relation between the pair of objects and attributes by using the relational attribute and object method. Further, the cognitive relations between the pair of objects or attributes within the context could be checked with newly defined conditions, i.e. the necessary and sufficient condition. These conditions will evaluate whether the relational object and attribute is adequate to have relations between the pair of objects and attributes. The new information is obtained without increasing the number of neurons in brain. It is achieved by creating cognitive relations between the pair of objects and attributes. The obtained results are beneficial to simulate the intelligence behaviour of brain such as learning and memorizing. Integrating the idea of CI into cognitive relations is a promising and challenging research direction. In this paper, we have discussed it from the aspects of cognitive mechanism, cognitive computing and cognitive process.