Category Archives: Cognitive Sciences

Extracting video summaries from RL processes to explain and understand them

Pedro Sequeira, Melinda Gervasio, Interestingness elements for explainable reinforcement learning: Understanding agents’ capabilities and limitations. Artificial Intelligence, Volume 288, 2020 DOI: 10.1016/j.artint.2020.103367.

We propose an explainable reinforcement learning (XRL) framework that analyzes an agent’s history of interaction with the environment to extract interestingness elements that help explain its behavior. The framework relies on data readily available from standard RL algorithms, augmented with data that can easily be collected by the agent while learning. We describe how to create visual summaries of an agent’s behavior in the form of short video-clips highlighting key interaction moments, based on the proposed elements. We also report on a user study where we evaluated the ability of humans to correctly perceive the aptitude of agents with different characteristics, including their capabilities and limitations, given visual summaries automatically generated by our framework. The results show that the diversity of aspects captured by the different interestingness elements is crucial to help humans correctly understand an agent’s strengths and limitations in performing a task, and determine when it might need adjustments to improve its performance.

Expressing POMDPs policies through Knowledge-Based programs

Bruno Zanuttini, Jérôme Lang, Abdallah Saffidine, François Schwarzentruber Knowledge-based programs as succinct policies for partially observable domains. Artificial Intelligence, Volume 288, 2020 DOI: 10.1016/j.artint.2020.103365.

We suggest to express policies for contingent planning by knowledge-based programs (KBPs). KBPs, introduced by Fagin et al. (1995) [32], are high-level protocols describing the actions that the agent should perform as a function of their current knowledge: branching conditions are epistemic formulas that are interpretable by the agent. The main aim of our paper is to show that KBPs can be seen as a succinct language for expressing policies in single-agent contingent planning. KBP are conceptually very close to languages used for expressing policies in the partially observable planning literature: like them, they have conditional and looping structures, with actions as atomic programs and Boolean formulas on beliefs for choosing the execution path. Now, the specificity of KBPs is that branching conditions refer to the belief state and not to the observations. Because of their structural proximity, KBPs and standard languages for representing policies have the same power of expressivity: every standard policy can be expressed as a KBP, and every KBP can be “unfolded” into a standard policy. However, KBPs are more succinct, more readable, and more explainable than standard policies. On the other hand, they require more online computation time, but we show that this is an unavoidable tradeoff. We study knowledge-based programs along four criteria: expressivity, succinctness, complexity of online execution, and complexity of verification.

It seems that consciousness is not an analog uni-dimensional line, but multi-dimensional

Jonathan Birch, Alexandra K. Schnell, Nicola S. Clayton, Dimensions of Animal Consciousness. Trends in Cognitive Sciences, Volume 24, Issue 10, 2020, Pages 789-801 DOI: 10.1016/j.tics.2020.07.007.

How does consciousness vary across the animal kingdom? Are some animals ‘more conscious’ than others? This article presents a multidimensional framework for understanding interspecies variation in states of consciousness. The framework distinguishes five key dimensions of variation: perceptual richness, evaluative richness, integration at a time, integration across time, and self-consciousness. For each dimension, existing experiments that bear on it are reviewed and future experiments are suggested. By assessing a given species against each dimension, we can construct a consciousness profile for that species. On this framework, there is no single scale along which species can be ranked as more or less conscious. Rather, each species has its own distinctive consciousness profile.

It seems that our brain predicts semantic features of sensory stimuli to come

Friedemann Pulvermüller, Luigi Grisoni, Semantic Prediction in Brain and Mind. Trends in Cognitive Sciences, Volume 24, Issue 10, 2020, Pages 781-784 DOI: 10.1016/j.tics.2020.07.002.

We highlight a novel brain correlate of prediction, the prediction potential (or PP), a slow negative-going potential shift preceding visual, acoustic, and spoken or written verbal stimuli that can be predicted from their context. The cortical sources underlying the prediction potential reflect perceptual and semantic features of anticipated stimuli before these appear.

“Early exit” deep neural networks (i.e., CNN that provide outputs at intermediate points)

Scardapane, S., Scarpiniti, M., Baccarelli, E. et al. , Why Should We Add Early Exits to Neural Networks? . Cogn Comput 12, 954–966 (2020) DOI: 10.1007/s12559-020-09734-4.

Deep neural networks are generally designed as a stack of differentiable layers, in which a prediction is obtained only after running the full stack. Recently, some contributions have proposed techniques to endow the networks with early exits, allowing to obtain predictions at intermediate points of the stack. These multi-output networks have a number of advantages, including (i) significant reductions of the inference time, (ii) reduced tendency to overfitting and vanishing gradients, and (iii) capability of being distributed over multi-tier computation platforms. In addition, they connect to the wider themes of biological plausibility and layered cognitive reasoning. In this paper, we provide a comprehensive introduction to this family of neural networks, by describing in a unified fashion the way these architectures can be designed, trained, and actually deployed in time-constrained scenarios. We also describe in-depth their application scenarios in 5G and Fog computing environments, as long as some of the open research questions connected to them.

A new theory: we are curious about tasks that increase our ability to solve as many future tasks as possible

Franziska Brändle, Charley M. Wu, Eric Schulz, What Are We Curious about?, . Trends in Cognitive Sciences, Volume 24, Issue 9, 2020 DOI: 10.1016/j.tics.2020.05.010.

(no abstract).

Predicting optimistically seems to lead to better response of the agent to achieve the best goals

Zekun Sun, Chaz Firestone, Optimism and Pessimism in the Predictive Brain, . Trends in Cognitive Sciences, Volume 24, Issue 9, 2020 DOI: 10.1016/j.tics.2020.06.001.

(no abstract).

Interesting review of pshycological motivation and the role of RL in studying it

Randall C. O’Reilly, Unraveling the Mysteries of Motivation, Trends in Cognitive Sciences, Volume 24, Issue 6, 2020, Pages 425-434, DOI: 10.1016/j.tics.2020.03.001.

Motivation plays a central role in human behavior and cognition but is not well captured by widely used artificial intelligence (AI) and computational modeling frameworks. This Opinion article addresses two central questions regarding the nature of motivation: what are the nature and dynamics of the internal goals that drive our motivational system and how can this system be sufficiently flexible to support our ability to rapidly adapt to novel situations, tasks, etc.? In reviewing existing systems and neuroscience research and theorizing on these questions, a wealth of insights to constrain the development of computational models of motivation can be found.

Consciousness as a learning framework

Axel Cleeremans, Dalila Achoui, Arnaud Beauny, Lars Keuninckx, Jean-Remy Martin, Santiago Muñoz-Moldes, Laurène Vuillaume, Adélaïde de Heering, Learning to Be Conscious, Trends in Cognitive Sciences, Volume 24, Issue 2, 2020, Pages 112-123 DOI: 10.1016/j.tics.2019.11.011.

Consciousness remains a formidable challenge. Different theories of consciousness have proposed vastly different mechanisms to account for phenomenal experience. Here, appealing to aspects of global workspace theory, higher-order theories, social theories, and predictive processing, we introduce a novel framework: the self-organizing metarerpresentational account (SOMA), in which consciousness is viewed as something that the brain learns to do. By this account, the brain continuously and unconsciously learns to redescribe its own activity to itself, so developing systems of metarepresentations that qualify target first-order representations. Thus, experiences only occur in experiencers that have learned to know they possess certain first-order states and that have learned to care more about certain states than about others. In this sense, consciousness is the brain’s (unconscious, embodied, enactive, nonconceptual) theory about itself.

A model of the psychomotor behaviour of humans intended to be useful for integration with robots

Stephen Fox, Adrian Kotelba, Ilari Marstio, Jari Montonen, Aligning human psychomotor characteristics with robots, exoskeletons and augmented reality, Robotics and Computer-Integrated Manufacturing, Volume 63, 2020, DOI: 10.1016/j.rcim.2019.101922.

In previous production literature, the uncertainty of human behaviour has been recognized as a source of productivity, quality, and safety problems. However, fundamental reasons for the uncertainty of human behavior have received little analysis in the production literature. Furthermore, potential for these fundamental reasons to be aligned with production technologies in order to improve production performance has not been addressed. By contrast, in this paper, fundamental reasons for the uncertainty of human behaviour are explained through a model of psychomotor characteristics that encompasses physiology, past experiences, personality, gender, culture, emotion, reasoning, and biocybernetics. Through reference to 10 action research cases, the formal model is applied to provide guidelines for planning production work that includes robots, exoskeletons, and augmented reality.