Category Archives: Human Teleoperation

Shared autonomy where the target is predicted with POMDPs to cope with uncertain predictions

June 12, 2018 07:52 ,

Shervin Javdani, Henny Admoni, Stefania Pellegrinelli, Siddhartha S. Srinivasa, and J. Andrew Bagnell Shared autonomy via hindsight optimization for teleoperation and teaming, The International Journal of Robotics Research Vol 37, Issue 7, pp. 717 – 742 DOI: 10.1177/0278364918776060.

In shared autonomy, a user and autonomous system work together to achieve shared goals. To collaborate effectively, the autonomous system must know the user’s goal. As such, most prior works follow a predict-then-act model, first predicting the user’s goal with high confidence, then assisting given that goal. Unfortunately, confidently predicting the user’s goal may not be possible until they have nearly achieved it, causing predict-then-act methods to provide little assistance. However, the system can often provide useful assistance even when confidence for any single goal is low (e.g. move towards multiple goals). In this work, we formalize this insight by modeling shared autonomy as a partially observable Markov decision process (POMDP), providing assistance that minimizes the expected cost-to-go with an unknown goal. As solving this POMDP optimally is intractable, we use hindsight optimization to approximate. We apply our framework to both shared-control teleoperation and human–robot teaming. Compared with predict-then-act methods, our method achieves goals faster, requires less user input, decreases user idling time, and results in fewer user–robot collisions.

Posted in: Human teleoperation , Tagged: ,

Shared autonomy in robot teleoperation where the robot learns the user’s skills to adapt to them

May 11, 2018 08:28 ,

Enayati, N., Ferrigno, G. & De Momi, E. Real time implementation of socially acceptable collision avoidance of a low speed autonomous shuttle using the elastic band method, Auton Robot (2018) 42: 997, DOI: 10.1007/s10514-017-9675-4.

This work proposes a shared-control tele-operation framework that adapts its cooperative properties to the estimated skill level of the operator. It is hypothesized that different aspects of an operator’s performance in executing a tele-operated path tracking task can be assessed through conventional machine learning methods using motion-based and task-related features. To identify performance measures that capture motor skills linked to the studied task, an experiment is conducted where users new to tele-operation, practice towards motor skill proficiency in 7 training sessions. A set of classifiers are then learned from the acquired data and selected features, which can generate a skill profile that comprises estimations of user’s various competences. Skill profiles are exploited to modify the behavior of the assistive robotic system accordingly with the objective of enhancing user experience by preventing unnecessary restriction for skilled users. A second experiment is implemented in which novice and expert users execute the path tracking on different pathways while being assisted by the robot according to their estimated skill profiles. Results validate the skill estimation method and hint at feasibility of shared-control customization in tele-operated path tracking.

Posted in: Human teleoperation , Tagged:

A study of the influence of teleoperation in the remote driving of robots

March 12, 2018 09:45 ,

Storms, J. & Tilbury, D. J, A New Difficulty Index for Teleoperated Robots Driving through Obstacles, Intell Robot Syst (2018) 90: 147, DOI: 10.1007/s10846-017-0651-1.

Teleoperation allows humans to reach environments that would otherwise be too difficult or dangerous. The distance between the human operator and remote robot introduces a number of issues that can negatively impact system performance including degraded and delayed information exchange between the robot and human. Some operation scenarios and environments can tolerate these degraded conditions, while others cannot. However, little work has been done to investigate how factors such as communication delay, automation, and environment characteristics interact to affect teleoperation system performance. This paper presents results from a user study analyzing the effects of teleoperation factors including communication delay, autonomous assistance, and environment layout on user performance. A mobile robot driving task is considered in which subjects drive a robot to a goal location around obstacles as quickly (minimize time) and safely (avoid collisions) as possible. An environment difficulty index (ID) is defined in the paper and is shown to be able to predict the average time it takes for the human to drive the robot to a goal location with different obstacle configurations. The ID is also shown to predict the path chosen by the human better than travel time along that path.

Posted in: Human teleoperation

A framework to manage and switch between several sensor modalities in tele-operation

May 19, 2015 11:53 ,

Andrea Cherubini, Robin Passama, Philippe Fraisse, André Crosnier, A unified multimodal control framework for human–robot interaction, Robotics and Autonomous Systems, Volume 70, August 2015, Pages 106-115, ISSN 0921-8890, DOI: 10.1016/j.robot.2015.03.002.

In human–robot interaction, the robot controller must reactively adapt to sudden changes in the environment (due to unpredictable human behaviour). This often requires operating different modes, and managing sudden signal changes from heterogeneous sensor data. In this paper, we present a multimodal sensor-based controller, enabling a robot to adapt to changes in the sensor signals (here, changes in the human collaborator behaviour). Our controller is based on a unified task formalism, and in contrast with classical hybrid visicn–force–position control, it enables smooth transitions and weighted combinations of the sensor tasks. The approach is validated in a mock-up industrial scenario, where pose, vision (from both traditional camera and Kinect), and force tasks must be realized either exclusively or simultaneously, for human–robot collaboration.

Posted in: Human teleoperation , Tagged: ,

Estimating states of a human teleoperator and studying their influence in performing control

December 5, 2014 11:54 ,

Yunyi Jia, Ning Xi, Shuang Liu, Yunxia Wang, Xin Li, and Sheng Bi, Quality of teleoperator adaptive control for telerobotic operations The International Journal of Robotics Research December 2014 33: 1765-1781, first published on November 13, 2014. DOI: 10.1177/0278364914556124

Extensive studies have been conducted on telerobotic operations for decades due to their widespread applications in a variety of areas. Most studies have been focused on two major issues: stability and telepresence. Few have studied the influence of the operation status of the teleoperator on the performance of telerobotic operations. As subnormal operation status of the teleoperator may result in insufficient and even incorrect operations, the quality of teleoperator (QoT) is an important impact on the performance of the telerobotic operations in terms of the efficiency and safety even if both the stability and telepresence are guaranteed. Therefore, this paper investigates the online identification of the QoT and its application to telerobotic operations. The QoT is identified based on five QoT indicators which are generated based on the teleoperator’s brain EEG signals. A QoT adaptive control method is designed to adapt the velocity and responsivity of the robotic system to the operation status of the teleoperator such that the teleoperation efficiency and safety can be enhanced. The online QoT identification method was conducted on various teleoperators and the QoT adaptive control method was implemented on a mobile manipulator teleoperation system. The experimental results demonstrated the effectiveness and advantages of the proposed methods.

Posted in: Human teleoperation, Networked telerobots , Tagged: