Category Archives: Robotics

Modelling robot motion sequences through context-free grammars

October 20, 2020 19:38 ,

Rudolf Lioutikov, Guilherme Maeda, Filipe Veiga, Kristian Kersting, Jan Peters, Learning attribute grammars for movement primitive sequencing, The International Journal of Robotics Research, Vol 39, Issue 1, 2020, DOI: 10.1177/0278364919868279.

Movement primitives are a well studied and widely applied concept in modern robotics. However, composing primitives out of an existing library has shown to be a challenging problem. We propose the use of probabilistic context-free grammars to sequence a series of primitives to generate complex robot policies from a given library of primitives. The rule-based nature of formal grammars allows an intuitive encoding of hierarchically structured tasks. This hierarchical concept strongly connects with the way robot policies can be learned, organized, and re-used. However, the induction of context-free grammars has proven to be a complicated and yet unsolved challenge. We exploit the physical nature of robot movement primitives to restrict and efficiently search the grammar space. The grammar is learned by applying a Markov chain Monte Carlo optimization over the posteriors of the grammars given the observations. The proposal distribution is defined as a mixture over the probabilities of the operators connecting the search space. Moreover, we present an approach for the categorization of probabilistic movement primitives and discuss how the connectibility of two primitives can be determined. These characteristics in combination with restrictions to the operators guarantee continuous sequences while reducing the grammar space. In addition, a set of attributes and conditions is introduced that augments probabilistic context-free grammars in order to solve primitive sequencing tasks with the capability to adapt single primitives within the sequence. The method was validated on tasks that require the generation of complex sequences consisting of simple movement primitives using a seven-degree-of-freedom lightweight robotic arm.

Posted in: Robot motion planning , Tagged: ,

Mixing human advice and reward functions for improving reinforcement learning of motor skills in robots with a nice related work on interactive RL

October 20, 2020 19:31 ,

Carlos Celemin, Guilherme Maeda, Javier Ruiz-del-Solar, Jan Peters, Jens Kober, Reinforcement learning of motor skills using Policy Search and human corrective advice, The International Journal of Robotics Research, Vol 38, Issue 14, 2019, DOI: 10.1177/0278364919871998.

Robot learning problems are limited by physical constraints, which make learning successful policies for complex motor skills on real systems unfeasible. Some reinforcement learning methods, like Policy Search, offer stable convergence toward locally optimal solutions, whereas interactive machine learning or learning-from-demonstration methods allow fast transfer of human knowledge to the agents. However, most methods require expert demonstrations. In this work, we propose the use of human corrective advice in the actions domain for learning motor trajectories. Additionally, we combine this human feedback with reward functions in a Policy Search learning scheme. The use of both sources of information speeds up the learning process, since the intuitive knowledge of the human teacher can be easily transferred to the agent, while the Policy Search method with the cost/reward function take over for supervising the process and reducing the influence of occasional wrong human corrections. This interactive approach has been validated for learning movement primitives with simulated arms with several degrees of freedom in reaching via-point movements, and also using real robots in such tasks as “writing characters” and the ball-in-a-cup game. Compared with standard reinforcement learning without human advice, the results show that the proposed method not only converges to higher rewards when learning movement primitives, but also that the learning is sped up by a factor of 4–40 times, depending on the task.

Posted in: Applications of reinforcement learning to robots , Tagged:

Reinforcement learning for improving autonomy of mobile robots in calibrating visual sensors

October 20, 2020 15:31 ,

Fernando Nobre, Christoffer Heckman, Learning to calibrate: Reinforcement learning for guided calibration of visual–inertial rigs,. The International Journal of Robotics Research, 38(12–13), 1352–1374, DOI: 10.1177/0278364919844824.

We present a new approach to assisted intrinsic and extrinsic calibration with an observability-aware visual–inertial calibration system that guides the user through the calibration procedure by suggesting easy-to-perform motions that render the calibration parameters observable. This is done by identifying which subset of the parameter space is rendered observable with a rank-revealing decomposition of the Fisher information matrix, modeling calibration as a Markov decision process and using reinforcement learning to establish which discrete sequence of motions optimizes for the regression of the desired parameters. The goal is to address the assumption common to most calibration solutions: that sufficiently informative motions are provided by the operator. We do not make use of a process model and instead leverage an experience-based approach that is broadly applicable to any platform in the context of simultaneous localization and mapping. This is a step in the direction of long-term autonomy and “power-on-and-go” robotic systems, making repeatable and reliable calibration accessible to the non-expert operator.

Posted in: Applications of reinforcement learning to robots, Robot sensors , Tagged:

Grid maps extended with confidence information

November 25, 2019 16:03 ,

Ali-akbar Agha-mohammadi, Eric Heiden, Karol Hausman, Confidence-rich grid mapping,. The International Journal of Robotics Research, 38(12–13), 1352–1374, DOI: 10.1177/0278364919839762.

Representing the environment is a fundamental task in enabling robots to act autonomously in unknown environments. In this work, we present confidence-rich mapping (CRM), a new algorithm for spatial grid-based mapping of the 3D environment. CRM augments the occupancy level at each voxel by its confidence value. By explicitly storing and evolving confidence values using the CRM filter, CRM extends traditional grid mapping in three ways: first, it partially maintains the probabilistic dependence among voxels; second, it relaxes the need for hand-engineering an inverse sensor model and proposes the concept of sensor cause model that can be derived in a principled manner from the forward sensor model; third, and most importantly, it provides consistent confidence values over the occupancy estimation that can be reliably used in collision risk evaluation and motion planning. CRM runs online and enables mapping environments where voxels might be partially occupied. We demonstrate the performance of the method on various datasets and environments in simulation and on physical systems. We show in real-world experiments that, in addition to achieving maps that are more accurate than traditional methods, the proposed filtering scheme demonstrates a much higher level of consistency between its error and the reported confidence, hence, enabling a more reliable collision risk evaluation for motion planning.

Posted in: Mobile robot mapping , Tagged:

Robots with extended sensorization of their physical building materials

November 25, 2019 16:00 ,

Dana Hughes, Christoffer Heckman, Nikolaus Correll, Materials that make robots smart ,. The International Journal of Robotics Research, 38(12–13), 1338–1351, DOI: 10.1177/0278364919856099.

We posit that embodied artificial intelligence is not only a computational, but also a materials problem. While the importance of material and structural properties in the control loop are well understood, materials can take an active role during control by tight integration of sensors, actuators, computation, and communication. We envision such materials to abstract functionality, therefore making the construction of intelligent robots more straightforward and robust. For example, robots could be made of bones that measure load, muscles that move, skin that provides the robot with information about the kind and location of tactile sensations ranging from pressure to texture and damage, eyes that extract high-level information, and brain material that provides computation in a scalable manner. Such materials will not resemble any existing engineered materials, but rather the heterogeneous components out of which their natural counterparts are made. We describe the state-of-the-art in so-called “robotic materials,” their opportunities for revolutionizing applications ranging from manipulation to autonomous driving by describing two recent robotic materials, a smart skin and a smart tire in more depth, and conclude with open challenges that the robotics community needs to address in collaboration with allies, such as wireless sensor network researchers and polymer scientists.

Posted in: Robot sensors , Tagged:

A comparison / evaluation of bug algorithms for mobile robots and their bad performance when relying in only one sensor

October 10, 2019 15:16 ,

K.N. McGuire, G.C.H.E. de Croon, K. Tuyls, A comparative study of bug algorithms for robot navigation,. Robotics and Autonomous Systems, Volume 121, DOI: 10.1016/j.robot.2019.103261.

This paper presents a literature survey and a comparative study of Bug Algorithms, with the goal of investigating their potential for robotic navigation. At first sight, these methods seem to provide an efficient navigation paradigm, ideal for implementations on tiny robots with limited resources. Closer inspection, however, shows that many of these Bug Algorithms assume perfect global position estimate of the robot which in GPS-denied environments implies considerable expenses of computation and memory — relying on accurate Simultaneous Localization And Mapping (SLAM) or Visual Odometry (VO) methods. We compare a selection of Bug Algorithms in a simulated robot and environment where they endure different types noise and failure-cases of their on-board sensors. From the simulation results, we conclude that the implemented Bug Algorithms’ performances are sensitive to many types of sensor-noise, which was most noticeable for odometry-drift. This raises the question if Bug Algorithms are suitable for real-world, on-board, robotic navigation as is. Variations that use multiple sensors to keep track of their progress towards the goal, were more adept in completing their task in the presence of sensor-failures. This shows that Bug Algorithms must spread their risk, by relying on the readings of multiple sensors, to be suitable for real-world deployment.

Posted in: Robot motion planning , Tagged:

Analyzing effects of loads and terrain on wheel shapes in order to reduce errors in position estimation of a mobile wheeled robot

October 10, 2019 15:09 ,

Smieszek, M., Dobrzanska, M. & Dobrzanski, P. , The impact of load on the wheel rolling radius and slip in a small mobile platform. Auton Robot (2019) 43: 2095, DOI: 10.1007/s10514-019-09857-0.

Automated guided vehicles are used in a variety of applications. Their major purpose is to replace humans in onerous, monotonous and sometimes dangerous operations. Such vehicles are controlled and navigated by application-specific software. In the case of vehicles used in multiple environments and operating conditions, such as the vehicles which are the subject of this study, a reasonable approach is required when selecting the navigation system. The vehicle may travel around an enclosed hall and around an open yard. The pavement surface may be smooth or uneven. Vehicle wheels should be flexible and facilitate the isolation and absorption of vibrations in order to reduce the effect of surface unevenness to the load. Another important factor affecting the operating conditions are changes to vehicle load resulting from the distribution of the load and the weight carried. Considering all of the factors previously mentioned, the vehicle’s navigation and control system is required to meet two opposing criteria. One of them is low price and simplicity, the other is ensuring the required accuracy when following the preset route. In the course of this study, a methodology was developed and tested which aims to obtain a satisfactory compromise between those two conflicting criteria. During the study a vehicle made in Technical University of Rzeszow was used. The results of the experimental research have been analysed. The results of the analysis provided a foundation for the development of a methodology leading to a reduction in navigation errors. Movement simulations for the proposed vehicle system demonstrated the potential for a significant reduction in the number of positioning errors.

Posted in: Mobile Robot Localization, Robot motion planning

A kind of reinforcement learning that decouples modelling from planning using Gaussian Processes for the former

October 10, 2019 15:07 ,

Rakicevic, N. & Kormushev, P., Active learning via informed search in movement parameter space for efficient robot task learning and transfer. Auton Robot (2019) 43: 1917, DOI: 10.1007/s10514-019-09842-7.

Learning complex physical tasks via trial-and-error is still challenging for high-degree-of-freedom robots. Greatest challenges are devising a suitable objective function that defines the task, and the high sample complexity of learning the task. We propose a novel active learning framework, consisting of decoupled task model and exploration components, which does not require an objective function. The task model is specific to a task and maps the parameter space, defining a trial, to the trial outcome space. The exploration component enables efficient search in the trial-parameter space to generate the subsequent most informative trials, by simultaneously exploiting all the information gained from previous trials and reducing the task model’s overall uncertainty. We analyse the performance of our framework in a simulation environment and further validate it on a challenging bimanual-robot puck-passing task. Results show that the robot successfully acquires the necessary skills after only 100 trials without any prior information about the task or target positions. Decoupling the framework’s components also enables efficient skill transfer to new environments which is validated experimentally.

Posted in: Applications of reinforcement learning to robots , Tagged:

On the formalization and conceptualization of real-time basic concepts and methods (RMS, EDF) for robots

October 10, 2019 14:39 ,

Nicolas Gobillot, Charles Lesire, David Doose, A Design and Analysis Methodology for Component-Based Real-Time Architectures of Autonomous Systems. Journal of Intelligent & Robotic Systems, October 2019, Volume 96, Issue 1, pp 123–138, DOI: 10.1007/s10846-018-0967-5.

The integration of autonomous robots in real applications is a challenge. It needs that the behaviour of these robots is proved to be safe. In this paper, we focus on the real-time software embedded on the robot, and that supports the execution of safe and autonomous behaviours. We propose a methodology that goes from the design of component-based software architectures using a Domain Specific Language, to the analysis of the real-time constraints that arise when considering the safety of software applications. This methodology is supported by a code generation toolchain that ensures that the code eventually executed on the robot is consistent with the analysis performed. This methodology is applied on a ground robot exploring an area.

Posted in: Real-Time Systems, Robotic software , Tagged: , , ,

Hardware efficient collision avoidance for mobile robots through the use of interval arithmetics and parallelism

September 16, 2019 16:08 ,

Pranjal Vyas, Leena Vachhani, K Sridharan, Hardware-efficient interval analysis based collision detection and avoidance for mobile robots. Mechatronics, Volume 62, 2019, DOI: 10.1016/j.mechatronics.2019.102258.

Collision detection and avoidance is challenging when the mobile robot is moving among multiple dynamic obstacles. A hardware-efficient architecture supporting parallel implementation is presented in this work for low-power, faster and reliable collision-free motion planning. An approach based on interval analysis is developed for designing an efficient hardware architecture. The proposed architecture achieves parallelism which can be combined with any robotic task involving multiple obstacles. Interval arithmetic is used for representing the pose of the robot and the obstacle as velocity intervals in a fixed time period. These intervals correspond to sub-intervals such as arcs and line-segments. In particular, the collision detection problem for dynamic objects involves the computation of line segment-arc intersections and segment-segment intersections. The intersection of these boundary curves is carried out in a hardware-efficient manner so that it avoids complex arithmetic computations such as multiplication, division etc and exploits parallelism. We develop several results on intersection of these sub-intervals for collision detection and use them to obtain a hardware-efficient collision detection algorithm that requires only shift and add-type of computations. The algorithm is further used in developing a hardware-efficient technique for finding an exhaustive set of solutions for avoiding collision of the robot with dynamic obstacles. Simulation results in MatLab and experiments with a Field Programmable Gate Array (FPGA)-based robot show that a variety of collision avoidance techniques can be implemented using the proposed solution set that guarantees collision avoidance with multiple obstacles.

Posted in: Robot motion planning , Tagged: ,