Tag Archives: Reactive Navigation

Mapless (egocentric) navigation with hierarchical RL that includes a good survey of current RL approaches for that task

November 14, 2024 16:31 ,

Yan Gao, Feiqiang Lin, Boliang Cai, Jing Wu, Changyun Wei, Raphael Grech, Ze Ji, Mapless navigation via Hierarchical Reinforcement Learning with memory-decaying novelty, Robotics and Autonomous Systems, Volume 182, 2024, DOI: 10.1016/j.robot.2024.104815.

Hierarchical Reinforcement Learning (HRL) has shown superior performance for mapless navigation tasks. However, it remains limited in unstructured environments that might contain terrains like long corridors and dead corners, which can lead to local minima. This is because most HRL-based mapless navigation methods employ a simplified reward setting and exploration strategy. In this work, we propose a novel reward function for training the high-level (HL) policy, which contains two components: extrinsic reward and intrinsic reward. The extrinsic reward encourages the robot to move towards the target location, while the intrinsic reward is computed based on novelty, episode memory and memory decaying, making the agent capable of accomplishing spontaneous exploration. We also design a novel neural network structure that incorporates an LSTM network to augment the agent with memory and reasoning capabilities. We test our method in unknown environments and specific scenarios prone to the local minimum problem to evaluate the navigation performance and local minimum resolution ability. The results show that our method significantly increases the success rate when compared to advanced RL-based methods, achieving a maximum improvement of nearly 28%. Our method demonstrates effective improvement in addressing the local minimum issue, especially in cases where the baselines fail completely. Additionally, numerous ablation studies consistently confirm the effectiveness of our proposed reward function and neural network structure.

Posted in: Applications of reinforcement learning to robots, Robot motion planning, Robot task planning , Tagged: ,

A related work with a nice taxonomy of robot navigation algorithms

June 17, 2019 12:18 ,

Eduardo J. Molinos, Ángel Llamazares, Manuel Ocaña Dynamic window based approaches for avoiding obstacles in moving, Robotics and Autonomous Systems,
Volume 118, 2019, Pages 112-130 DOI: 10.1016/j.robot.2019.05.003.

In recent years, Unmanned Ground Vehicles (UGVs) have been widely used as service robots. Unlike industrial robots, which are situated in fixed and controlled positions, UGVs work in dynamic environments, sharing the environment with other vehicles and humans. These robots should be able to move without colliding with any obstacle, assuring its integrity and the environment safety. In this paper, we propose two adaptations of the classical Dynamic Window algorithm for dealing with dynamic obstacles like Dynamic Window for Dynamic Obstacles (DW4DO) and Dynamic Window for Dynamic Obstacles Tree (DW4DOT). These new algorithms are compared with our previous algorithms based on Curvature Velocity Methods: Predicted Curvature Velocity Method (PCVM) and Dynamic Curvature Velocity Method (DCVM). Proposals have been validated in both simulated and real environment using several robotic platforms.

Posted in: Robot motion planning , Tagged: ,

Using reasoning to improve low-level robot navigation

September 19, 2018 08:47 ,

Muhayyuddin, Aliakbar AkbariJan Rosell, A Real-Time Path-Planning Algorithm based on Receding Horizon Techniques, Journal of Intelligent & Robotic Systems, September 2018, Volume 91, Issue 3–4, pp 459–477, DOI: 10.1007/s10846-017-0698-z.

Physics-based motion planning is a challenging task, since it requires the computation of the robot motions while allowing possible interactions with (some of) the obstacles in the environment. Kinodynamic motion planners equipped with a dynamic engine acting as state propagator are usually used for that purpose. The difficulties arise in the setting of the adequate forces for the interactions and because these interactions may change the pose of the manipulatable obstacles, thus either facilitating or preventing the finding of a solution path. The use of knowledge can alleviate the stated difficulties. This paper proposes the use of an enhanced state propagator composed of a dynamic engine and a low-level geometric reasoning process that is used to determine how to interact with the objects, i.e. from where and with which forces. The proposal, called κ-PMP can be used with any kinodynamic planner, thus giving rise to e.g. κ-RRT. The approach also includes a preprocessing step that infers from a semantic abstract knowledge described in terms of an ontology the manipulation knowledge required by the reasoning process. The proposed approach has been validated with several examples involving an holonomic mobile robot, a robot with differential constraints and a serial manipulator, and benchmarked using several state-of-the art kinodynamic planners. The results showed a significant difference in the power consumption with respect to simple physics-based planning, an improvement in the success rate and in the quality of the solution paths.

Posted in: Robot motion planning , Tagged: , ,

A novel approach to avoid the minima problem in potential fields navigation

May 22, 2018 07:56 ,

Fedele, G., D’Alfonso, L., Chiaravalloti, F. et al., Obstacles Avoidance Based on Switching Potential Functions, J Intell Robot Syst (2018) 90: 387. DOI: 10.1007/s10846-017-0687-2.

In this paper, a novel path planning and obstacles avoidance method for a mobile robot is proposed. This method makes use of a switching strategy between the attractive potential of the target and a new helicoidal potential field which allows to bypass an obstacle by driving the robot around it. The new technique aims at overcoming the local minima problems of the well-known artificial potentials method, caused by the summation of two (or more) potential fields. In fact, in the proposed approach, only a single potential is used at a time. The resulting proposed technique uses only local information and ensures high robustness, in terms of achieved performance and computational complexity, w.r.t. the number of obstacles. Numerical simulations, together with comparisons with existing methods, confirm a very robust behavior of the method, also in the case of a framework with multiple obstacles.

Posted in: Robot motion planning , Tagged: ,

A novel motion planning algorithm for robot navigation taking into account the robot kinematic constraints and shape

March 28, 2018 09:18 ,

Muhannad Mujahed, Dirk Fischer, Bärbel Mertsching, Admissible gap navigation: A new collision avoidance approach, Robotics and Autonomous Systems,
Volume 103, 2018, Pages 93-110, DOI: 10.1016/j.robot.2018.02.008.

This paper proposes a new concept, the Admissible Gap (AG), for reactive collision avoidance. A gap is called admissible if it is possible to find a collision-free motion control that guides a robot through it, while respecting the vehicle constraints. By utilizing this concept, a new navigation approach was developed, achieving an outstanding performance in unknown dense environments. Unlike the widely used gap-based methods, our approach directly accounts for the exact shape and kinematics, rather than finding a direction solution and turning it later into a collision-free admissible motion. The key idea is to analyze the structure of obstacles and virtually locate an admissible gap, once traversed, the robot makes progress towards the goal. For this purpose, we introduce a strategy of traversing gaps that respect the kinematic constraints and provides a compromise between path length and motion safety. We also propose a new methodology for extracting gaps that eliminates useless ones, thus reducing oscillations. Experimental results along with performance evaluation demonstrate the outstanding behavior of the proposed AG approach. Furthermore, a comparison with existing state-of-the-art methods shows that the AG approach achieves the best results in terms of efficiency, robustness, safety, and smoothness.

Posted in: Robot motion planning , Tagged: ,

Real-time modification of user inputs in the teleoperation of an UAV in order to avoid obstacles with a reactive algorithm, transparently from the user control

July 26, 2017 08:55 ,

Daman Bareiss, Joseph R. Bourne & Kam K. Leang, On-board model-based automatic collision avoidance: application in remotely-piloted unmanned aerial vehicles, Auton Robot (2017) 41:1539–1554, DOI: 10.1007/s10514-017-9614-4.

This paper focuses on real-world implementation and verification of a local, model-based stochastic automatic collision avoidance algorithm, with application in
remotely-piloted (tele-operated) unmanned aerial vehicles (UAVs). Automatic collision detection and avoidance for tele-operated UAVs can reduce the workload of pilots to allow them to focus on the task at hand, such as searching for victims in a search and rescue scenario following a natural disaster. The proposed algorithm takes the pilot’s input and exploits the robot’s dynamics to predict the robot’s trajectory for determining whether a collision will occur. Using on-board sensors for obstacle detection, if a collision is imminent, the algorithm modifies the pilot’s input to avoid the collision while attempting to maintain the pilot’s intent. The algorithm is implemented using a low-cost on-board computer, flight-control system, and a two-dimensional laser illuminated detection and ranging sensor for obstacle detection along the trajectory of the robot. The sensor data is processed using a split-and-merge segmentation algorithm and an approximate Minkowski difference. Results from flight tests demonstrate the algorithm’s capabilities for teleoperated collision-free control of an experimental UAV.

Posted in: Robot motion planning , Tagged: , ,