Tag Archives: Manipulator Arms

RL to learn not only manipulator skills but also safety skills

January 25, 2024 12:37 ,

A. C. Ak, E. E. Aksoy and S. Sariel, Learning Failure Prevention Skills for Safe Robot Manipulation, IEEE Robotics and Automation Letters, vol. 8, no. 12, pp. 7994-8001, Dec. 2023 DOI: 10.1109/LRA.2023.3324587.

Robots are more capable of achieving manipulation tasks for everyday activities than before. However, the safety of manipulation skills that robots employ is still an open problem. Considering all possible failures during skill learning increases the complexity of the process and restrains learning an optimal policy. Nonetheless, safety-focused modularity in the acquisition of skills has not been adequately addressed in previous works. For that purpose, we reformulate skills as base and failure prevention skills, where base skills aim at completing tasks and failure prevention skills aim at reducing the risk of failures to occur. Then, we propose a modular and hierarchical method for safe robot manipulation by augmenting base skills by learning failure prevention skills with reinforcement learning and forming a skill library to address different safety risks. Furthermore, a skill selection policy that considers estimated risks is used for the robot to select the best control policy for safe manipulation. Our experiments show that the proposed method achieves the given goal while ensuring safety by preventing failures. We also show that with the proposed method, skill learning is feasible and our safe manipulation tools can be transferred to the real environment.

Posted in: Applications of reinforcement learning to robots , Tagged: ,

Using “empowerment” to better select actions in RL when there are only sparse rewards

December 21, 2023 09:37 ,

Dai, S., Xu, W., Hofmann, A. et al. An empowerment-based solution to robotic manipulation tasks with sparse rewards, Auton Robot 47, 617\u2013633 (2023) DOI: 10.1007/s10514-023-10087-8.

In order to provide adaptive and user-friendly solutions to robotic manipulation, it is important that the agent can learn to accomplish tasks even if they are only provided with very sparse instruction signals. To address the issues reinforcement learning algorithms face when task rewards are sparse, this paper proposes an intrinsic motivation approach that can be easily integrated into any standard reinforcement learning algorithm and can allow robotic manipulators to learn useful manipulation skills with only sparse extrinsic rewards. Through integrating and balancing empowerment and curiosity, this approach shows superior performance compared to other state-of-the-art intrinsic exploration approaches during extensive empirical testing. When combined with other strategies for tackling the exploration challenge, e.g. curriculum learning, our approach is able to further improve the exploration efficiency and task success rate. Qualitative analysis also shows that when combined with diversity-driven intrinsic motivations, this approach can help manipulators learn a set of diverse skills which could potentially be applied to other more complicated manipulation tasks and accelerate their learning process.

Posted in: Applications of reinforcement learning to robots , Tagged: ,

Model-based RL for controling a soft manipulator arm

February 13, 2019 11:52 ,

T. G. Thuruthel, E. Falotico, F. Renda and C. Laschi, Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators, IEEE Transactions on Robotics, vol. 35, no. 1, pp. 124-134, Feb. 2019. DOI: 10.1109/TRO.2018.2878318.

Dynamic control of soft robotic manipulators is an open problem yet to be well explored and analyzed. Most of the current applications of soft robotic manipulators utilize static or quasi-dynamic controllers based on kinematic models or linearity in the joint space. However, such approaches are not truly exploiting the rich dynamics of a soft-bodied system. In this paper, we present a model-based policy learning algorithm for closed-loop predictive control of a soft robotic manipulator. The forward dynamic model is represented using a recurrent neural network. The closed-loop policy is derived using trajectory optimization and supervised learning. The approach is verified first on a simulated piecewise constant strain model of a cable driven under-actuated soft manipulator. Furthermore, we experimentally demonstrate on a soft pneumatically actuated manipulator how closed-loop control policies can be derived that can accommodate variable frequency control and unmodeled external loads.

Posted in: Applications of reinforcement learning to control engineering , Tagged: ,

Calibrating a robotic manipulator through photogrammetry, and a nice state-of-the-art in the issue of robot calibration

September 20, 2017 08:03 ,

Alexandre Filion, Ahmed Joubair, Antoine S. Tahan, Ilian A. Bonev, Robot calibration using a portable photogrammetry system, Robotics and Computer-Integrated Manufacturing, Volume 49, 2018, Pages 77-87, DOI: 10.1016/j.rcim.2017.05.004.

This work investigates the potential use of a commercially-available portablephotogrammetry system (the MaxSHOT 3D) in industrial robot calibration. To demonstrate the effectiveness of this system, we take the approach of comparing the device with a laser tracker (the FARO laser tracker) by calibrating an industrial robot, with each device in turn, then comparing the obtained robot position accuracy after calibration. As the use of a portablephotogrammetry system in robot calibration is uncommon, this paper presents how to proceed. It will cover the theory of robot calibration: the robot’s forward and inverse kinematics, the elasto-geometrical model of the robot, the generation and ultimate selection of robot configurations to be measured, and the parameter identification. Furthermore, an experimental comparison of the laser tracker and the MaxSHOT3D is described. The obtained results show that the FARO laser trackerION performs slightly better: The absolute positional accuracy obtained with the laser tracker is 0.365mm and 0.147mm for the maximum and the mean position errors, respectively. Nevertheless, the results obtained by using the MaxSHOT3D are almost as good as those obtained by using the laser tracker: 0.469mm and 0.197mm for the maximum and the mean position errors, respectively. Performances in distance accuracy, after calibration (i.e. maximum errors), are respectively 0.329mm and 0.352mm, for the laser tracker and the MaxSHOT 3D. However, as the validation measurements were acquired with the laser tracker, bias favors this device. Thus, we may conclude that the calibration performances of the two measurement devices are very similar.

Posted in: Industrial robots, Robot models , Tagged: ,