Category Archives: Robotics

A new model of reinforcement learning based on the human brain that copes with continuous spaces through continuous rewards, with a short but nice state-of-the-art of RL applied to large, continuous spaces

October 17, 2018 08:58 ,

Feifei Zhao, Yi Zeng, Guixiang Wang, Jun Bai, Bo Xu, A Brain-Inspired Decision Making Model Based on Top-Down Biasing of Prefrontal Cortex to Basal Ganglia and Its Application in Autonomous UAV Explorations, Cognitive Computation, Volume 10, Issue 2, pp 296–306, DOI: 10.1007/s12559-017-9511-3.

Decision making is a fundamental ability for intelligent agents (e.g., humanoid robots and unmanned aerial vehicles). During decision making process, agents can improve the strategy for interacting with the dynamic environment through reinforcement learning. Many state-of-the-art reinforcement learning models deal with relatively smaller number of state-action pairs, and the states are preferably discrete, such as Q-learning and Actor-Critic algorithms. While in practice, in many scenario, the states are continuous and hard to be properly discretized. Better autonomous decision making methods need to be proposed to handle these problems. Inspired by the mechanism of decision making in human brain, we propose a general computational model, named as prefrontal cortex-basal ganglia (PFC-BG) algorithm. The proposed model is inspired by the biological reinforcement learning pathway and mechanisms from the following perspectives: (1) Dopamine signals continuously update reward-relevant information for both basal ganglia and working memory in prefrontal cortex. (2) We maintain the contextual reward information in working memory. This has a top-down biasing effect on reinforcement learning in basal ganglia. The proposed model separates the continuous states into smaller distinguishable states, and introduces continuous reward function for each state to obtain reward information at different time. To verify the performance of our model, we apply it to many UAV decision making experiments, such as avoiding obstacles and flying through window and door, and the experiments support the effectiveness of the model. Compared with traditional Q-learning and Actor-Critic algorithms, the proposed model is more biologically inspired, and more accurate and faster to make decision.

Posted in: Applications of reinforcement learning to robots, Psycho-physiological bases of engineering , Tagged: ,

SLAM as a sampling problem, with some references to the signal sampling state-of-the-art

October 11, 2018 08:51 ,

Golnoosh Elhami, et. al Sampling at Unknown Locations: Uniqueness and Reconstruction Under Constraints, IEEE Transactions on Signal Processing, Vol 66 no. 22, DOI: 10.1109/TSP.2018.2872019.

Traditional sampling results assume that the sample locations are known. Motivated by simultaneous localization and mapping (SLAM) and structure from motion (SfM), we investigate sampling at unknown locations. Without further constraints, the problem is often hopeless. For example, we recently showed that, for polynomial and bandlimited signals, it is possible to find two signals, arbitrarily far from each other, that fit the measurements. However, we also showed that this can be overcome by adding constraints to the sample positions. In this paper, we show that these constraints lead to a uniform sampling of a composite of functions. Furthermore, the formulation retains the key aspects of the SLAM and SfM problems, whilst providing uniqueness, in many cases. We demonstrate this by studying two simple examples of constrained sampling at unknown locations. In the first, we consider sampling a periodic bandlimited signal composite with an unknown linear function. We derive the sampling requirements for uniqueness and present an algorithm that recovers both the bandlimited signal and the linear warping. Furthermore, we prove that, when the requirements for uniqueness are not met, the cases of multiple solutions have measure zero. For our second example, we consider polynomials sampled such that the sampling positions are constrained by a rational function. We previously proved that, if a specific sampling requirement is met, uniqueness is achieved. In addition, we present an alternate minimization scheme for solving the resulting non-convex optimization problem. Finally, fully reproducible simulation results are provided to support our theoretical analysis.

Posted in: Mobile robot SLAM, Systems and Signals

Sharing beliefs (pdfs) between human and robot

October 4, 2018 08:38 ,

Rina Tse, Mark Campbell, Human–Robot Communications of Probabilistic Beliefs via a Dirichlet Process Mixture of Statements, IEEE Transactions on Robotics, vol. 34, no. 5, DOI: 10.1109/TRO.2018.2830360.

This paper presents a natural framework for information sharing in cooperative tasks involving humans and robots. In this framework, all information gathered over time by a human–robot team is exchanged and summarized in the form of a fused probability density function (pdf). An approach for an intelligent system to describe its belief pdfs in English expressions is presented. This belief expression generation is achieved through two goodness measures: semantic correctness and information preservation. In order to describe complex, multimodal belief pdfs, a Mixture of Statements (MoS) model is proposed such that optimal expressions can be generated through compositions of multiple statements. The model is further extended to a nonparametric Dirichlet process MoS generation, such that the optimal number of statements required for describing a given pdf is automatically determined. Results based on information loss, human collaborative task performances, and correctness rating scores suggest that the proposed method for generating belief expressions is an effective approach for communicating probabilistic information between robots and humans.

Posted in: Human-robot interaction

Automatic design of a robot to perform given tasks with an optimal configuration

October 4, 2018 08:35 ,

Sehoon Ha et al., Computational Design of Robotic Devices From High-Level Motion Specifications, IEEE Transactions on Robotics, vol. 34, no. 5, DOI: 10.1109/TRO.2018.2830419.

We present a novel computational approach to design the robotic devices from high-level motion specifications. Our computational system uses a library of modular components—actuators, mounting brackets, and connectors—to define the space of possible robot designs. The process of creating a new robot begins with a set of input trajectories that specify how its end effectors and/or body should move. By searching through the combinatorial set of possible arrangements of modular components, our method generates a functional, as-simple-as-possible robotic device that is capable of tracking the input motion trajectories. To significantly improve the efficiency of this discrete optimization process, we propose a novel heuristic that guides the search for appropriate designs. Briefly, our heuristic function estimates how much an intermediate robot design needs to change before it becomes able to execute the target motion trajectories. We demonstrate the effectiveness of our computational design method by automatically creating a variety of robotic manipulators and legged robots. To generate these results, we define our own robotic kit that includes off-the-shelf actuators and 3-D printable connectors. We validate our results by fabricating two robotic devices designed with our method.

Posted in: Robot models , Tagged: ,

On the need to replanning in POMDPs when applied to real systems, due to imperfect sensing and computational cost of online planning

October 4, 2018 08:27 ,

Ali-akbar Agha-mohammadi et al., SLAP: Simultaneous Localization and Planning Under Uncertainty via Dynamic Replanning in Belief Space, IEEE Transactions on Robotics, vol. 34, no. 5, DOI: 10.1109/TRO.2018.2838556.

Simultaneous localization and planning (SLAP) is a crucial ability for an autonomous robot operating under uncertainty. In its most general form, SLAP induces a continuous partially observable Markov decision process (POMDP), which needs to be repeatedly solved online. This paper addresses this problem and proposes a dynamic replanning scheme in belief space. The underlying POMDP, which is continuous in state, action, and observation space, is approximated offline via sampling-based methods, but operates in a replanning loop online to admit local improvements to the coarse offline policy. This construct enables the proposed method to combat changing environments and large localization errors, even when the change alters the homotopy class of the optimal trajectory. It further outperforms the state-of-the-art Feedback-based Information RoadMap (FIRM) method by eliminating unnecessary stabilization steps. Applying belief space planning to physical systems brings with it a plethora of challenges. A key focus of this paper is to implement the proposed planner on a physical robot and show the SLAP solution performance under uncertainty, in changing environments and in the presence of large disturbances, such as a kidnapped robot situation.

Posted in: Robot motion planning, Robot task planning , Tagged: ,

Interesting study of the number of optimal points in SLAM, considering it as a non-linear, non-convex optimization problem

September 27, 2018 09:27 ,

Heng Wang, Shoudong Huang, Guanghong Yang, Gamini Dissanayake, Comparison of two different objective functions in 2D point feature SLAM, Automatica,
Volume 97, 2018, Pages 172-181, DOI: 10.1016/j.automatica.2018.08.009.

This paper compares two different objective functions in 2D point feature Simultaneous Localization and Mapping (SLAM). It is shown that the objective function can have a significant impact on the convergence of the iterative optimization techniques used in SLAM. When Frobenius norm is adopted for the error term of the orientation part of odometry, the SLAM problem has much better convergence properties, as compared with that using the angle difference as the error term. For one-step case, we have proved that there is one and only one minimum to the SLAM problem, and strong duality always holds. For two-step case, strong duality always holds except when three very special conditions hold simultaneously (which happens with probability zero), thus the global optimal solution to primal SLAM problem can be obtained by solving the corresponding Lagrangian dual problem in most cases. Further, for arbitrary m-step cases, we also show using examples that much better convergence results can be obtained. Simulation examples are given to demonstrate the different convergence properties using two different objective functions.

Posted in: Mobile robot SLAM , Tagged: ,

A performance metric for evaluating and comparing robot navigation algorithms

September 21, 2018 08:31 ,

Yazhini Chitra Pradeep, Kendrick Amezquita-Semprun, Manuel Del Rosario, Peter C.Y. Chen, The Pc metric: A performance measure for collision avoidance algorithms, Robotics and Autonomous Systems, Volume 109, 2018, Pages 125-138, DOI: 10.1016/j.robot.2018.08.005.

Despite the comprehensive development in the field of navigation algorithms for mobile robots, the research on performance metrics and evaluation procedures for making standardized quantitative comparison between different algorithms has gained attention only recently. This work attempts to contribute with such effort by introducing a performance metric for the assessment of collision avoidance algorithms for mobile robots. The proposed metric comprehensively evaluates the actions taken by the objects and their consequences, in a given scenario of any given collision avoidance algorithm, based on the concept of probability of collision. The contribution of the paper encompasses the definition of the metric, the methodology to estimate the metric, and the framework to apply the metric for any given scenario. Experiments and numerical simulations are conducted to validate and demonstrate the effectiveness of the proposed metric in performance evaluation and comparison among different collision avoidance algorithms.

Posted in: Robot motion planning

A novel hybridization of semantic and topological maps applied to mapping and localization in outdoors

September 21, 2018 08:26 ,

Fernando Bernuy, Javier Ruiz-del-Solar, Topological Semantic Mapping and Localization in Urban Road Scenarios, Journal of Intelligent & Robotic Systems, September 2018, Volume 92, Issue 1, pp 19–32, DOI: 10.1007/s10846-017-0744-x.

Autonomous vehicle self-localization must be robust to environment changes, such as dynamic objects, variable illumination, and atmospheric conditions. Topological maps provide a concise representation of the world by only keeping information about relevant places, being robust to environment changes. On the other hand, semantic maps correspond to a high level representation of the environment that includes labels associated with relevant objects and places. Hence, the use of a topological map based on semantic information represents a robust and efficient solution for large-scale outdoor scenes for autonomous vehicles and Advanced Driver Assistance Systems (ADAS). In this work, a novel topological semantic mapping and localization methodology for large-scale outdoor scenarios for autonomous driving and ADAS applications is presented. The methodology uses: (i) a deep neural network for obtaining semantic observations of the environment, (ii) a Topological Semantic Map (TSM) for storing selected semantic observations, and (iii) a topological localization algorithm which uses a Particle Filter for obtaining the vehicle’s pose in the TSM. The proposed methodology was tested on a real driving scenario, where a True Estimate Rate of the vehicle’s pose of 96.9% and a Mean Position Accuracy of 7.7[m] were obtained. These results are much better than the ones obtained by other two methods used for comparative purposes. Experiments also show that the method is able to obtain the pose of the vehicle when its initial pose is unknown.

Posted in: Mobile robot SLAM , Tagged: , ,

A nice hybridization of RBPF, high-frequency scan matching and topological maps to perform SLAM, with an also nice state-of-the-art

September 19, 2018 08:52 ,

Aristeidis G. Thallas, Emmanouil G. Tsardoulias, Loukas Petrou, Topological Based Scan Matching – Odometry Posterior Sampling in RBPF Under Kinematic Model Failures, Journal of Intelligent & Robotic Systems, September 2018, Volume 91, Issue 3–4, pp 543–568, DOI: 10.1007/s10846-017-0730-3.

Rao-Blackwellized Particle Filters (RBPF) have been utilized to provide a solution to the SLAM problem. One of the main factors that cause RBPF failure is the potential particle impoverishment. Another popular approach to the SLAM problem are Scan Matching methods, whose good results require environments with lots of information, however fail in the lack thereof. To face these issues, in the current work techniques are presented to combine Rao-Blackwellized particle filters with a scan matching algorithm (CRSM SLAM). The particle filter maintains the correct hypothesis in environments lacking features and CRSM is employed in feature-rich environments while simultaneously reduces the particle filter dispersion. Since CRSM’s good performance is based on its high iteration frequency, a multi-threaded combination is presented which allows CRSM to operate while RBPF updates its particles. Additionally, a novel method utilizing topological information is proposed, in order to reduce the number of particle filter resamplings. Finally, we present methods to address anomalous situations where scan matching can not be performed and the vehicle displays behaviors not modeled by the kinematic model, causing the whole method to collapse. Numerous experiments are conducted to support the aforementioned methods’ advantages.

Posted in: Mobile robot SLAM , Tagged: , , , ,

Using reasoning to improve low-level robot navigation

September 19, 2018 08:47 ,

Muhayyuddin, Aliakbar AkbariJan Rosell, A Real-Time Path-Planning Algorithm based on Receding Horizon Techniques, Journal of Intelligent & Robotic Systems, September 2018, Volume 91, Issue 3–4, pp 459–477, DOI: 10.1007/s10846-017-0698-z.

Physics-based motion planning is a challenging task, since it requires the computation of the robot motions while allowing possible interactions with (some of) the obstacles in the environment. Kinodynamic motion planners equipped with a dynamic engine acting as state propagator are usually used for that purpose. The difficulties arise in the setting of the adequate forces for the interactions and because these interactions may change the pose of the manipulatable obstacles, thus either facilitating or preventing the finding of a solution path. The use of knowledge can alleviate the stated difficulties. This paper proposes the use of an enhanced state propagator composed of a dynamic engine and a low-level geometric reasoning process that is used to determine how to interact with the objects, i.e. from where and with which forces. The proposal, called κ-PMP can be used with any kinodynamic planner, thus giving rise to e.g. κ-RRT. The approach also includes a preprocessing step that infers from a semantic abstract knowledge described in terms of an ontology the manipulation knowledge required by the reasoning process. The proposed approach has been validated with several examples involving an holonomic mobile robot, a robot with differential constraints and a serial manipulator, and benchmarked using several state-of-the art kinodynamic planners. The results showed a significant difference in the power consumption with respect to simple physics-based planning, an improvement in the success rate and in the quality of the solution paths.

Posted in: Robot motion planning , Tagged: , ,