On how psychological time emerges from execution of actions in the environment

Jennifer T. Coull, Sylvie Droit-Volet, Explicit Understanding of Duration Develops Implicitly through Action, Trends in Cognitive Sciences, Volume 22, Issue 10, 2018, Pages 923-937, DOI: 10.1016/j.tics.2018.07.011.

Time is relative. Changes in cognitive state or sensory context make it appear to speed up or slow down. Our perception of time is a rather fragile mental construct derived from the way events in the world are processed and integrated in memory. Nevertheless, the slippery concept of time can be structured by draping it over more concrete functional scaffolding. Converging evidence from developmental studies of children and neuroimaging in adults indicates that we can represent time in spatial or motor terms. We hypothesise that explicit processing of time is mediated by motor structures of the brain in adulthood because we implicitly learn about time through action during childhood. Future challenges will be to harness motor or spatial representations of time to optimise behaviour, potentially for therapeutic gain.

A very interesting analysis on how reinforcement learning depends on time, both for MDPs and for the psychological basis of RL in the human brain

Elijah A. Petter, Samuel J. Gershman, Warren H. Meck, Integrating Models of Interval Timing and Reinforcement Learning, Trends in Cognitive Sciences, Volume 22, Issue 10, 2018, Pages 911-922 DOI: 10.1016/j.tics.2018.08.004.

We present an integrated view of interval timing and reinforcement learning (RL) in the brain. The computational goal of RL is to maximize future rewards, and this depends crucially on a representation of time. Different RL systems in the brain process time in distinct ways. A model-based system learns ‘what happens when’, employing this internal model to generate action plans, while a model-free system learns to predict reward directly from a set of temporal basis functions. We describe how these systems are subserved by a computational division of labor between several brain regions, with a focus on the basal ganglia and the hippocampus, as well as how these regions are influenced by the neuromodulator dopamine.

Some quotes beyond the abstract:

The Markov assumption also makes explicit the requirements for temporal representation. All temporal dynamics must be captured by the state-transition function, which means that the state representation must encode the time-invariant structure of the environment.

A nice introduction to psychological time

Lindsey Drayton, Moran Furman, Thy Mind, Thy Brain and Time, Trends in Cognitive Sciences, olume 22, Issue 10, 2018, Pages 841-843 DOI: 10.1016/j.tics.2018.08.007.

The passage of time has fascinated the human mind for millennia. Tools for measuring time emerged early in civilization: lunar calendars appear in the archeological record as far back as 10 000 years ago and water clocks some 6000 years ago. Later technological innovations such as mechanical clocks, and more recently atomic clocks, have allowed the tracking of time with ever-increasing precision. And yet, arguably, the most sophisticated ‘time piece’ is the brain. Our brains can not only track the duration and succession of events, but they can also coordinate complex motor movements at striking levels of precision; communicate effectively by generating and interpreting sounds and speech; determine how to maximize rewards over time in the face of uncertainty; reflect upon the past; plan for the future; respond to temporal regularities and irregularities in the environment; and adapt to change in temporal scales that range from millisecond resolution up to evolutionary processes spanning millions of years.

A performance metric for evaluating and comparing robot navigation algorithms

Yazhini Chitra Pradeep, Kendrick Amezquita-Semprun, Manuel Del Rosario, Peter C.Y. Chen, The Pc metric: A performance measure for collision avoidance algorithms, Robotics and Autonomous Systems, Volume 109, 2018, Pages 125-138, DOI: 10.1016/j.robot.2018.08.005.

Despite the comprehensive development in the field of navigation algorithms for mobile robots, the research on performance metrics and evaluation procedures for making standardized quantitative comparison between different algorithms has gained attention only recently. This work attempts to contribute with such effort by introducing a performance metric for the assessment of collision avoidance algorithms for mobile robots. The proposed metric comprehensively evaluates the actions taken by the objects and their consequences, in a given scenario of any given collision avoidance algorithm, based on the concept of probability of collision. The contribution of the paper encompasses the definition of the metric, the methodology to estimate the metric, and the framework to apply the metric for any given scenario. Experiments and numerical simulations are conducted to validate and demonstrate the effectiveness of the proposed metric in performance evaluation and comparison among different collision avoidance algorithms.

A novel hybridization of semantic and topological maps applied to mapping and localization in outdoors

Fernando Bernuy, Javier Ruiz-del-Solar, Topological Semantic Mapping and Localization in Urban Road Scenarios, Journal of Intelligent & Robotic Systems, September 2018, Volume 92, Issue 1, pp 19–32, DOI: 10.1007/s10846-017-0744-x.

Autonomous vehicle self-localization must be robust to environment changes, such as dynamic objects, variable illumination, and atmospheric conditions. Topological maps provide a concise representation of the world by only keeping information about relevant places, being robust to environment changes. On the other hand, semantic maps correspond to a high level representation of the environment that includes labels associated with relevant objects and places. Hence, the use of a topological map based on semantic information represents a robust and efficient solution for large-scale outdoor scenes for autonomous vehicles and Advanced Driver Assistance Systems (ADAS). In this work, a novel topological semantic mapping and localization methodology for large-scale outdoor scenarios for autonomous driving and ADAS applications is presented. The methodology uses: (i) a deep neural network for obtaining semantic observations of the environment, (ii) a Topological Semantic Map (TSM) for storing selected semantic observations, and (iii) a topological localization algorithm which uses a Particle Filter for obtaining the vehicle’s pose in the TSM. The proposed methodology was tested on a real driving scenario, where a True Estimate Rate of the vehicle’s pose of 96.9% and a Mean Position Accuracy of 7.7[m] were obtained. These results are much better than the ones obtained by other two methods used for comparative purposes. Experiments also show that the method is able to obtain the pose of the vehicle when its initial pose is unknown.

A nice hybridization of RBPF, high-frequency scan matching and topological maps to perform SLAM, with an also nice state-of-the-art

Aristeidis G. Thallas, Emmanouil G. Tsardoulias, Loukas Petrou, Topological Based Scan Matching – Odometry Posterior Sampling in RBPF Under Kinematic Model Failures, Journal of Intelligent & Robotic Systems, September 2018, Volume 91, Issue 3–4, pp 543–568, DOI: 10.1007/s10846-017-0730-3.

Rao-Blackwellized Particle Filters (RBPF) have been utilized to provide a solution to the SLAM problem. One of the main factors that cause RBPF failure is the potential particle impoverishment. Another popular approach to the SLAM problem are Scan Matching methods, whose good results require environments with lots of information, however fail in the lack thereof. To face these issues, in the current work techniques are presented to combine Rao-Blackwellized particle filters with a scan matching algorithm (CRSM SLAM). The particle filter maintains the correct hypothesis in environments lacking features and CRSM is employed in feature-rich environments while simultaneously reduces the particle filter dispersion. Since CRSM’s good performance is based on its high iteration frequency, a multi-threaded combination is presented which allows CRSM to operate while RBPF updates its particles. Additionally, a novel method utilizing topological information is proposed, in order to reduce the number of particle filter resamplings. Finally, we present methods to address anomalous situations where scan matching can not be performed and the vehicle displays behaviors not modeled by the kinematic model, causing the whole method to collapse. Numerous experiments are conducted to support the aforementioned methods’ advantages.

Using reasoning to improve low-level robot navigation

Muhayyuddin, Aliakbar AkbariJan Rosell, A Real-Time Path-Planning Algorithm based on Receding Horizon Techniques, Journal of Intelligent & Robotic Systems, September 2018, Volume 91, Issue 3–4, pp 459–477, DOI: 10.1007/s10846-017-0698-z.

Physics-based motion planning is a challenging task, since it requires the computation of the robot motions while allowing possible interactions with (some of) the obstacles in the environment. Kinodynamic motion planners equipped with a dynamic engine acting as state propagator are usually used for that purpose. The difficulties arise in the setting of the adequate forces for the interactions and because these interactions may change the pose of the manipulatable obstacles, thus either facilitating or preventing the finding of a solution path. The use of knowledge can alleviate the stated difficulties. This paper proposes the use of an enhanced state propagator composed of a dynamic engine and a low-level geometric reasoning process that is used to determine how to interact with the objects, i.e. from where and with which forces. The proposal, called κ-PMP can be used with any kinodynamic planner, thus giving rise to e.g. κ-RRT. The approach also includes a preprocessing step that infers from a semantic abstract knowledge described in terms of an ontology the manipulation knowledge required by the reasoning process. The proposed approach has been validated with several examples involving an holonomic mobile robot, a robot with differential constraints and a serial manipulator, and benchmarked using several state-of-the art kinodynamic planners. The results showed a significant difference in the power consumption with respect to simple physics-based planning, an improvement in the success rate and in the quality of the solution paths.

A unifying framework for path planning in real-time (mainly for UAVs) and a nice summary of the state-of-the-art in modern path planning

M. Murillo, G. SánchezL. GenzelisL. Giovanini, A Real-Time Path-Planning Algorithm based on Receding Horizon Techniques, Journal of Intelligent & Robotic Systems, September 2018, Volume 91, Issue 3–4, pp 445–457, DOI: 10.1007/s10846-017-0740-1.

In this article we present a real-time path-planning algorithm that can be used to generate optimal and feasible paths for any kind of unmanned vehicle (UV). The proposed algorithm is based on the use of a simplified particle vehicle (PV) model, which includes the basic dynamics and constraints of the UV, and an iterated non-linear model predictive control (NMPC) technique that computes the optimal velocity vector (magnitude and orientation angles) that allows the PV to move toward desired targets. The computed paths are guaranteed to be feasible for any UV because: i) the PV is configured with similar characteristics (dynamics and physical constraints) as the UV, and ii) the feasibility of the optimization problem is guaranteed by the use of the iterated NMPC algorithm. As demonstration of the capabilities of the proposed path-planning algorithm, we explore several simulation examples in different scenarios. We consider the existence of static and dynamic obstacles and a follower condition.

A new variant of A* that is more computationally efficient

Adam Niewola, Leszek Podsedkowski, L* Algorithm—A Linear Computational Complexity Graph Searching Algorithm for Path Planning, Journal of Intelligent & Robotic Systems, September 2018, Volume 91, Issue 3–4, pp 425–444, DOI: 10.1007/s10846-017-0748-6.

The state-of-the-art graph searching algorithm applied to the optimal global path planning problem for mobile robots is the A* algorithm with the heap structured open list. In this paper, we present a novel algorithm, called the L* algorithm, which can be applied to global path planning and is faster than the A* algorithm. The structure of the open list with the use of bidirectional sublists (buckets) ensures the linear computational complexity of the L* algorithm because the nodes in the current bucket can be processed in any sequence and it is not necessary to sort the bucket. Our approach can maintain the optimality and linear computational complexity with the use of the cost expressed by floating-point numbers. The paper presents the requirements of the L* algorithm use and the proof of the admissibility of this algorithm. The experiments confirmed that the L* algorithm is faster than the A* algorithm in various path planning scenarios. We also introduced a method of estimating the execution time of the A* and the L* algorithm. The method was compared with the experimental results.

A summary on reward processing in psychophysiology

Dan Foti, Anna Weinberg, Reward and feedback processing: State of the field, best practices, and future directions, International Journal of Psychophysiology, Volume 132, Part B, 2018, Pages 171-174, DOI: 10.1016/j.ijpsycho.2018.08.006.

There is a long history of studies using event-related potentials (ERPs) to examine how the brain monitors performance. Many initial studies focused on error processing, both internal (i.e., neural activity elicited by error commission) (Falkenstein et al., 1991; Gehring et al., 1993) and external (i.e. neural activity elicited by feedback indicating an unfavorable outcome) (Gehring and Willoughby, 2002; Miltner et al., 1997). A frequent assumption in this line of research has been that correct performance and favorable outcomes served as reference conditions, and that any effects on ERP amplitudes specifically reflected error processing. This starting premise is at odds with the large human and animal neuroscience literatures on reward processing, which focus on the motivated pursuit of said favorable outcomes. In fact, reward and error processing are intrinsically linked, and both undergird effective task performance: the brain is highly sensitive to events that are better or worse than expected in order to continuously modulate behavior in line with task goals (Holroyd and Coles, 2002). In recent years, the ERP literature on feedback processing has broadened to explicitly incorporate reward processing, thereby enriching traditional studies focused on error processing. Specific developments in this regard include an expanded focus on multiple stages of reward processing (e.g., anticipation versus outcome), charting the development of reward processing across the lifespan, and the examination of aberrant sensitivity to reward in psychiatric illnesses. While these advances are highly promising, the general ERP literature on feedback processing continues to be fragmented with regard to terminology, analytic techniques, task designs, and interpretation of findings, ultimately limiting progress in the field.

The overarching goal of this special issue was to carefully examine the state of the art in our current understanding of feedback processing. The aim was to provide an integrative overview that covers multiple theoretical perspectives and methodological approaches. Consideration has been given in this collection of articles to both basic and applied research topics, and throughout the special issue there is an emphasis on providing specific recommendations for study design and the identification of important future research directions. In the remainder of this introductory editorial, we set the stage for these articles by highlighting complementary results and points of intersection across four themes: integrating perspectives on reward and error processing; experimental manipulations, psychometrics, and individual differences.