Category Archives: Cognitive Sciences

On the existence of prior knowledge, “pre-wired” in animal brains, that guides further learning

October 17, 2018 09:12 ,

Elisabetta Versace, Antone Martinho-Truswell, Alex Kacelnik, Giorgio Vallortigara, Priors in Animal and Artificial Intelligence: Where Does Learning Begin?, Trends in Cognitive Sciences, Volume 22, Issue 11, 2018, Pages 963-965, DOI: 10.1016/j.tics.2018.07.005.

A major goal for the next generation of artificial intelligence (AI) is to build machines that are able to reason and cope with novel tasks, environments, and situations in a manner that approaches the abilities of animals. Evidence from precocial species suggests that driving learning through suitable priors can help to successfully face this challenge.

Posted in: Psycho-physiological bases of engineering

A new model of reinforcement learning based on the human brain that copes with continuous spaces through continuous rewards, with a short but nice state-of-the-art of RL applied to large, continuous spaces

October 17, 2018 08:58 ,

Feifei Zhao, Yi Zeng, Guixiang Wang, Jun Bai, Bo Xu, A Brain-Inspired Decision Making Model Based on Top-Down Biasing of Prefrontal Cortex to Basal Ganglia and Its Application in Autonomous UAV Explorations, Cognitive Computation, Volume 10, Issue 2, pp 296–306, DOI: 10.1007/s12559-017-9511-3.

Decision making is a fundamental ability for intelligent agents (e.g., humanoid robots and unmanned aerial vehicles). During decision making process, agents can improve the strategy for interacting with the dynamic environment through reinforcement learning. Many state-of-the-art reinforcement learning models deal with relatively smaller number of state-action pairs, and the states are preferably discrete, such as Q-learning and Actor-Critic algorithms. While in practice, in many scenario, the states are continuous and hard to be properly discretized. Better autonomous decision making methods need to be proposed to handle these problems. Inspired by the mechanism of decision making in human brain, we propose a general computational model, named as prefrontal cortex-basal ganglia (PFC-BG) algorithm. The proposed model is inspired by the biological reinforcement learning pathway and mechanisms from the following perspectives: (1) Dopamine signals continuously update reward-relevant information for both basal ganglia and working memory in prefrontal cortex. (2) We maintain the contextual reward information in working memory. This has a top-down biasing effect on reinforcement learning in basal ganglia. The proposed model separates the continuous states into smaller distinguishable states, and introduces continuous reward function for each state to obtain reward information at different time. To verify the performance of our model, we apply it to many UAV decision making experiments, such as avoiding obstacles and flying through window and door, and the experiments support the effectiveness of the model. Compared with traditional Q-learning and Actor-Critic algorithms, the proposed model is more biologically inspired, and more accurate and faster to make decision.

Posted in: Applications of reinforcement learning to robots, Psycho-physiological bases of engineering , Tagged: ,

Z-numbers: an extension of fuzzy variables for cognitive decision making, and the concept of cognitive information

October 17, 2018 08:54 ,

Hong-gang Peng, Jian-qiang Wang, Outranking Decision-Making Method with Z-Number Cognitive Information, Cognitive Computation, Volume 10, Issue 5, pp 752–768, DOI: 10.1007/s12559-018-9556-y.

The Z-number provides an adequate and reliable description of cognitive information. The nature of Z-numbers is complex, however, and important issues in Z-number computation remain to be addressed. This study focuses on developing a computationally simple method with Z-numbers to address multicriteria decision-making (MCDM) problems. Processing Z-numbers requires the direct computation of fuzzy and probabilistic uncertainties. We used an effective method to analyze the Z-number construct. Next, we proposed some outranking relations of Z-numbers and defined the dominance degree of discrete Z-numbers. Also, after analyzing the characteristics of elimination and choice translating reality III (ELECTRE III) and qualitative flexible multiple criteria method (QUALIFLEX), we developed an improved outranking method. To demonstrate this method, we provided an illustrative example concerning job-satisfaction evaluation. We further verified the validity of the method by a criteria test and comparative analysis. The results demonstrate that the method can be successfully applied to real-world decision-making problems, and it can identify more reasonable outcomes than previous methods. This study overcomes the high computational complexity in existing Z-number computation frameworks by exploring the pairwise comparison of Z-numbers. The method inherits the merits of the classical outranking method and considers the non-compensability of criteria. Therefore, it has remarkable potential to address practical decision-making problems involving Z-information.

Posted in: Cognitive sciences, Psycho-physiological bases of engineering , Tagged: ,

On how psychological time emerges from execution of actions in the environment

September 26, 2018 09:24 ,

Jennifer T. Coull, Sylvie Droit-Volet, Explicit Understanding of Duration Develops Implicitly through Action, Trends in Cognitive Sciences, Volume 22, Issue 10, 2018, Pages 923-937, DOI: 10.1016/j.tics.2018.07.011.

Time is relative. Changes in cognitive state or sensory context make it appear to speed up or slow down. Our perception of time is a rather fragile mental construct derived from the way events in the world are processed and integrated in memory. Nevertheless, the slippery concept of time can be structured by draping it over more concrete functional scaffolding. Converging evidence from developmental studies of children and neuroimaging in adults indicates that we can represent time in spatial or motor terms. We hypothesise that explicit processing of time is mediated by motor structures of the brain in adulthood because we implicitly learn about time through action during childhood. Future challenges will be to harness motor or spatial representations of time to optimise behaviour, potentially for therapeutic gain.

Posted in: Psycho-physiological bases of engineering , Tagged:

A very interesting analysis on how reinforcement learning depends on time, both for MDPs and for the psychological basis of RL in the human brain

September 26, 2018 09:17 ,

Elijah A. Petter, Samuel J. Gershman, Warren H. Meck, Integrating Models of Interval Timing and Reinforcement Learning, Trends in Cognitive Sciences, Volume 22, Issue 10, 2018, Pages 911-922 DOI: 10.1016/j.tics.2018.08.004.

We present an integrated view of interval timing and reinforcement learning (RL) in the brain. The computational goal of RL is to maximize future rewards, and this depends crucially on a representation of time. Different RL systems in the brain process time in distinct ways. A model-based system learns ‘what happens when’, employing this internal model to generate action plans, while a model-free system learns to predict reward directly from a set of temporal basis functions. We describe how these systems are subserved by a computational division of labor between several brain regions, with a focus on the basal ganglia and the hippocampus, as well as how these regions are influenced by the neuromodulator dopamine.

Some quotes beyond the abstract:

The Markov assumption also makes explicit the requirements for temporal representation. All temporal dynamics must be captured by the state-transition function, which means that the state representation must encode the time-invariant structure of the environment.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

A nice introduction to psychological time

September 26, 2018 09:13 ,

Lindsey Drayton, Moran Furman, Thy Mind, Thy Brain and Time, Trends in Cognitive Sciences, olume 22, Issue 10, 2018, Pages 841-843 DOI: 10.1016/j.tics.2018.08.007.

The passage of time has fascinated the human mind for millennia. Tools for measuring time emerged early in civilization: lunar calendars appear in the archeological record as far back as 10 000 years ago and water clocks some 6000 years ago. Later technological innovations such as mechanical clocks, and more recently atomic clocks, have allowed the tracking of time with ever-increasing precision. And yet, arguably, the most sophisticated ‘time piece’ is the brain. Our brains can not only track the duration and succession of events, but they can also coordinate complex motor movements at striking levels of precision; communicate effectively by generating and interpreting sounds and speech; determine how to maximize rewards over time in the face of uncertainty; reflect upon the past; plan for the future; respond to temporal regularities and irregularities in the environment; and adapt to change in temporal scales that range from millisecond resolution up to evolutionary processes spanning millions of years.

Posted in: Cognitive sciences , Tagged: ,

A new variant of A* that is more computationally efficient

September 19, 2018 08:37 ,

Adam Niewola, Leszek Podsedkowski, L* Algorithm—A Linear Computational Complexity Graph Searching Algorithm for Path Planning, Journal of Intelligent & Robotic Systems, September 2018, Volume 91, Issue 3–4, pp 425–444, DOI: 10.1007/s10846-017-0748-6.

The state-of-the-art graph searching algorithm applied to the optimal global path planning problem for mobile robots is the A* algorithm with the heap structured open list. In this paper, we present a novel algorithm, called the L* algorithm, which can be applied to global path planning and is faster than the A* algorithm. The structure of the open list with the use of bidirectional sublists (buckets) ensures the linear computational complexity of the L* algorithm because the nodes in the current bucket can be processed in any sequence and it is not necessary to sort the bucket. Our approach can maintain the optimality and linear computational complexity with the use of the cost expressed by floating-point numbers. The paper presents the requirements of the L* algorithm use and the proof of the admissibility of this algorithm. The experiments confirmed that the L* algorithm is faster than the A* algorithm in various path planning scenarios. We also introduced a method of estimating the execution time of the A* and the L* algorithm. The method was compared with the experimental results.

Posted in: Artificial Intelligence , Tagged: , ,

A summary on reward processing in psychophysiology

September 17, 2018 09:52 ,

Dan Foti, Anna Weinberg, Reward and feedback processing: State of the field, best practices, and future directions, International Journal of Psychophysiology, Volume 132, Part B, 2018, Pages 171-174, DOI: 10.1016/j.ijpsycho.2018.08.006.

There is a long history of studies using event-related potentials (ERPs) to examine how the brain monitors performance. Many initial studies focused on error processing, both internal (i.e., neural activity elicited by error commission) (Falkenstein et al., 1991; Gehring et al., 1993) and external (i.e. neural activity elicited by feedback indicating an unfavorable outcome) (Gehring and Willoughby, 2002; Miltner et al., 1997). A frequent assumption in this line of research has been that correct performance and favorable outcomes served as reference conditions, and that any effects on ERP amplitudes specifically reflected error processing. This starting premise is at odds with the large human and animal neuroscience literatures on reward processing, which focus on the motivated pursuit of said favorable outcomes. In fact, reward and error processing are intrinsically linked, and both undergird effective task performance: the brain is highly sensitive to events that are better or worse than expected in order to continuously modulate behavior in line with task goals (Holroyd and Coles, 2002). In recent years, the ERP literature on feedback processing has broadened to explicitly incorporate reward processing, thereby enriching traditional studies focused on error processing. Specific developments in this regard include an expanded focus on multiple stages of reward processing (e.g., anticipation versus outcome), charting the development of reward processing across the lifespan, and the examination of aberrant sensitivity to reward in psychiatric illnesses. While these advances are highly promising, the general ERP literature on feedback processing continues to be fragmented with regard to terminology, analytic techniques, task designs, and interpretation of findings, ultimately limiting progress in the field.

The overarching goal of this special issue was to carefully examine the state of the art in our current understanding of feedback processing. The aim was to provide an integrative overview that covers multiple theoretical perspectives and methodological approaches. Consideration has been given in this collection of articles to both basic and applied research topics, and throughout the special issue there is an emphasis on providing specific recommendations for study design and the identification of important future research directions. In the remainder of this introductory editorial, we set the stage for these articles by highlighting complementary results and points of intersection across four themes: integrating perspectives on reward and error processing; experimental manipulations, psychometrics, and individual differences.

Posted in: Psycho-physiological bases of engineering , Tagged:

A survey on decision making for multiagent systems, including multirobot systems

September 13, 2018 09:59 ,

Y. Rizk, M. Awad and E. W. Tunstel, Decision Making in Multiagent Systems: A Survey, IEEE Transactions on Cognitive and Developmental Systems, vol. 10, no. 3, pp. 514-529, DOI: 10.1109/TCDS.2018.2840971.

Intelligent transport systems, efficient electric grids, and sensor networks for data collection and analysis are some examples of the multiagent systems (MAS) that cooperate to achieve common goals. Decision making is an integral part of intelligent agents and MAS that will allow such systems to accomplish increasingly complex tasks. In this survey, we investigate state-of-the-art work within the past five years on cooperative MAS decision making models, including Markov decision processes, game theory, swarm intelligence, and graph theoretic models. We survey algorithms that result in optimal and suboptimal policies such as reinforcement learning, dynamic programming, evolutionary computing, and neural networks. We also discuss the application of these models to robotics, wireless sensor networks, cognitive radio networks, intelligent transport systems, and smart electric grids. In addition, we define key terms in the area and discuss remaining challenges that include incorporating big data advancements to decision making, developing autonomous, scalable and computationally efficient algorithms, tackling more complex tasks, and developing standardized evaluation metrics. While recent surveys have been published on this topic, we present a broader discussion of related models and applications.Note to Practitioners:Future smart cities will rely on cooperative MAS that make decisions about what actions to perform that will lead to the completion of their tasks. Decision making models and algorithms have been developed and reported in the literature to generate such sequences of actions. These models are based on a wide variety of principles including human decision making and social animal behavior. In this paper, we survey existing decision making models and algorithms that generate optimal and suboptimal sequences of actions. We also discuss some of the remaining challenges faced by the research community before more effective MAS deployment can be achieved in this age of Internet of Things, robotics, and mobile devices. These challenges include developing more scalable and efficient algorithms, utilizing the abundant sensory data available, tackling more complex tasks, and developing evaluation standards for decision making.

Posted in: Artificial Intelligence , Tagged: , , ,

Interpreting time series patterns through reasoning

September 3, 2018 16:31 ,

T. Teijeiro, P. Félix, On the adoption of abductive reasoning for time series interpretation, Artificial Intelligence, Volume 262, 2018, Pages 163-188, DOI: 10.1016/j.artint.2018.06.005.

Time series interpretation aims to provide an explanation of what is observed in terms of its underlying processes. The present work is based on the assumption that the common classification-based approaches to time series interpretation suffer from a set of inherent weaknesses, whose ultimate cause lies in the monotonic nature of the deductive reasoning paradigm. In this document we propose a new approach to this problem, based on the initial hypothesis that abductive reasoning properly accounts for the human ability to identify and characterize the patterns appearing in a time series. The result of this interpretation is a set of conjectures in the form of observations, organized into an abstraction hierarchy and explaining what has been observed. A knowledge-based framework and a set of algorithms for the interpretation task are provided, implementing a hypothesize-and-test cycle guided by an attentional mechanism. As a representative application domain, interpretation of the electrocardiogram allows us to highlight the strengths of the proposed approach in comparison with traditional classification-based approaches.

Posted in: Artificial Intelligence , Tagged: ,