Category Archives: Networked Telerobots

Networked differential telerrobot remotely controlled in spite of disturbances and delays

April 3, 2024 12:51 ,

Luca Nanu, Luigi Colangelo, Carlo Novara, Carlos Perez Montenegro, Embedded model control of networked control systems: An experimental robotic application, Mechatronics, Volume 99, 2024 DOI: 10.1016/j.mechatronics.2024.103160.

In Networked Control System (NCS), the absence of physical communication links in the loop leads to relevant issues, such as measurement delays and asynchronous execution of the control commands. In general, these issues may significantly compromise the performance of the NCS, possibly causing unstable behaviours. This paper presents an original approach to the design of a complete digital control unit for a system characterized by a varying sampling time and asynchronous command execution. The approach is based on the Embedded Model Control (EMC) methodology, whose key feature is the estimation of the disturbances, errors and nonlinearities affecting the plant to control and their online cancellation. In this way, measurement delays and execution asynchronicity are treated as errors and rejected up to a given frequency by the EMC unit. The effectiveness of the proposed approach is demonstrated in a real-world case-study, where the NCS consists of a differential-drive mobile robot (the plant) and a control unit, and the two subsystems communicate through the web without physical connection links. After a preliminary verification using a high-fidelity numerical simulator, the designed controller is validated in several experimental tests, carried out on a real-time embedded system incorporated in the robotic platform.

Posted in: Networked telerobots , Tagged:

Modelling network delay in the remote estimation of the robot state for networked telerobots

September 25, 2023 11:35 ,

Barnali Das, Gordon Dobie, Delay compensated state estimation for Telepresence robot navigation, Robotics and Autonomous Systems, Volume 146, 2021 DOI: 10.1016/j.robot.2021.103890.

Telepresence robots empower human operators to navigate remote environments. However, operating and navigating the robot in an unknown environment is challenging due to delay in the communication network (e.g.,�distance, bandwidth, communication drop-outs etc.), processing delays and slow dynamics of the mobile robots resulting in time-lagged in the system. Also, erroneous sensor data measurement which is important to estimate the robot\u2019s true state (positional information) in the remote environment, often create complications and make it harder for the system to control the robot. In this paper, we propose a new approach for state estimation assuming uncertain delayed sensor measurements of a Telepresence robot during navigation. A new real world experimental model, based on Augmented State Extended Kalman Filter (AS-EKF), is proposed to estimate the true position of the Telepresence robot. The uncertainty of the delayed sensor measurements have been modelled using probabilistic density functions (PDF). The proposed model was successfully verified in our proposed experimental framework which consists of a state-of-the-art differential-drive Telepresence robot and a motion tracking multi-camera system. The results show significant improvements compared to the traditional EKF that does not consider uncertain delays in sensor measurements. The proposed model will be beneficial to build a real time predictive display by reducing the effect of visual delay to navigate the robot under the operator\u2019s control command, without waiting for delayed sensor measurements.

Posted in: Networked telerobots , Tagged: , ,

Convergence in reference tracking by a nonlinear system, with a known model, remotely controlled through WiFi

July 27, 2018 07:52 ,

Ali Parsa, Alireza Farhadi, Measurement and control of nonlinear dynamic systems over the internet (IoT): Applications in remote control of autonomous vehicles, Automatica, Volume 95, 2018, Pages 93-103 DOI: 10.1016/j.automatica.2018.05.016.

This paper presents a new technique for almost sure asymptotic state tracking, stability and reference tracking of nonlinear dynamic systems by remote controller over the packet erasure channel, which is an abstract model for transmission via WiFi and the Internet. By implementing a suitable linearization method, a proper encoder and decoder are presented for tracking the state trajectory of nonlinear systems at the end of communication link when the measurements are sent through the packet erasure channel. Then, a controller for reference tracking of the system is designed. In the proposed technique linearization is applied when the error between the states and an estimate of these states at the decoder increases. It is shown that the proposed technique results in almost sure asymptotic reference tracking (and hence stability) over the packet erasure channel. The satisfactory performance of the proposed state trajectory and reference tracking technique is illustrated by computer simulations by applying this technique on the unicycle model, which represents the dynamic of autonomous vehicles.

Posted in: Networked telerobots

Shared autonomy where the target is predicted with POMDPs to cope with uncertain predictions

June 12, 2018 07:52 ,

Shervin Javdani, Henny Admoni, Stefania Pellegrinelli, Siddhartha S. Srinivasa, and J. Andrew Bagnell Shared autonomy via hindsight optimization for teleoperation and teaming, The International Journal of Robotics Research Vol 37, Issue 7, pp. 717 – 742 DOI: 10.1177/0278364918776060.

In shared autonomy, a user and autonomous system work together to achieve shared goals. To collaborate effectively, the autonomous system must know the user’s goal. As such, most prior works follow a predict-then-act model, first predicting the user’s goal with high confidence, then assisting given that goal. Unfortunately, confidently predicting the user’s goal may not be possible until they have nearly achieved it, causing predict-then-act methods to provide little assistance. However, the system can often provide useful assistance even when confidence for any single goal is low (e.g. move towards multiple goals). In this work, we formalize this insight by modeling shared autonomy as a partially observable Markov decision process (POMDP), providing assistance that minimizes the expected cost-to-go with an unknown goal. As solving this POMDP optimally is intractable, we use hindsight optimization to approximate. We apply our framework to both shared-control teleoperation and human–robot teaming. Compared with predict-then-act methods, our method achieves goals faster, requires less user input, decreases user idling time, and results in fewer user–robot collisions.

Posted in: Human teleoperation , Tagged: ,

Shared autonomy in robot teleoperation where the robot learns the user’s skills to adapt to them

May 11, 2018 08:28 ,

Enayati, N., Ferrigno, G. & De Momi, E. Real time implementation of socially acceptable collision avoidance of a low speed autonomous shuttle using the elastic band method, Auton Robot (2018) 42: 997, DOI: 10.1007/s10514-017-9675-4.

This work proposes a shared-control tele-operation framework that adapts its cooperative properties to the estimated skill level of the operator. It is hypothesized that different aspects of an operator’s performance in executing a tele-operated path tracking task can be assessed through conventional machine learning methods using motion-based and task-related features. To identify performance measures that capture motor skills linked to the studied task, an experiment is conducted where users new to tele-operation, practice towards motor skill proficiency in 7 training sessions. A set of classifiers are then learned from the acquired data and selected features, which can generate a skill profile that comprises estimations of user’s various competences. Skill profiles are exploited to modify the behavior of the assistive robotic system accordingly with the objective of enhancing user experience by preventing unnecessary restriction for skilled users. A second experiment is implemented in which novice and expert users execute the path tracking on different pathways while being assisted by the robot according to their estimated skill profiles. Results validate the skill estimation method and hint at feasibility of shared-control customization in tele-operated path tracking.

Posted in: Human teleoperation , Tagged:

A study of the influence of teleoperation in the remote driving of robots

March 12, 2018 09:45 ,

Storms, J. & Tilbury, D. J, A New Difficulty Index for Teleoperated Robots Driving through Obstacles, Intell Robot Syst (2018) 90: 147, DOI: 10.1007/s10846-017-0651-1.

Teleoperation allows humans to reach environments that would otherwise be too difficult or dangerous. The distance between the human operator and remote robot introduces a number of issues that can negatively impact system performance including degraded and delayed information exchange between the robot and human. Some operation scenarios and environments can tolerate these degraded conditions, while others cannot. However, little work has been done to investigate how factors such as communication delay, automation, and environment characteristics interact to affect teleoperation system performance. This paper presents results from a user study analyzing the effects of teleoperation factors including communication delay, autonomous assistance, and environment layout on user performance. A mobile robot driving task is considered in which subjects drive a robot to a goal location around obstacles as quickly (minimize time) and safely (avoid collisions) as possible. An environment difficulty index (ID) is defined in the paper and is shown to be able to predict the average time it takes for the human to drive the robot to a goal location with different obstacle configurations. The ID is also shown to predict the path chosen by the human better than travel time along that path.

Posted in: Human teleoperation

A framework to manage and switch between several sensor modalities in tele-operation

May 19, 2015 11:53 ,

Andrea Cherubini, Robin Passama, Philippe Fraisse, André Crosnier, A unified multimodal control framework for human–robot interaction, Robotics and Autonomous Systems, Volume 70, August 2015, Pages 106-115, ISSN 0921-8890, DOI: 10.1016/j.robot.2015.03.002.

In human–robot interaction, the robot controller must reactively adapt to sudden changes in the environment (due to unpredictable human behaviour). This often requires operating different modes, and managing sudden signal changes from heterogeneous sensor data. In this paper, we present a multimodal sensor-based controller, enabling a robot to adapt to changes in the sensor signals (here, changes in the human collaborator behaviour). Our controller is based on a unified task formalism, and in contrast with classical hybrid visicn–force–position control, it enables smooth transitions and weighted combinations of the sensor tasks. The approach is validated in a mock-up industrial scenario, where pose, vision (from both traditional camera and Kinect), and force tasks must be realized either exclusively or simultaneously, for human–robot collaboration.

Posted in: Human teleoperation , Tagged: ,

Mathematical model of quartz crystal clocks and Kalman Filter estimation for clock synchronization

January 23, 2015 17:46 ,

Giorgi, G., An Event-Based Kalman Filter for Clock Synchronization, Instrumentation and Measurement, IEEE Transactions on , vol.64, no.2, pp.449,457, Feb. 2015, DOI: 10.1109/TIM.2014.2340631

The distribution of a time reference has long been a significant research topic in measurement and different solutions have been proposed over the years. In this context, the design of servo clocks plays an important role to get better performances by smoothing the influence of noise sources affecting a synchronization system. A servo clock is asked to provide an adaptive and conservative measure of the time distance between the local clock and the time reference by minimizing, if possible, the energy consumption. In this paper, we propose a servo clock based on an efficient implementation of the Kalman filter (KF), called in the following event-based KF that allows to overcome drawbacks of existing KF-based servo clocks with furthermore a significant reduction of the computational cost. An in-depth analysis of the synchronization uncertainty has been reported to completely characterize the proposed solution; and finally, some guidelines on how to correctly initialize the KF are provided.

Posted in: Networked telerobots , Tagged: ,

Estimating the bandwidth of a communication channel for adjusting the bitrate in high-definition video streaming, using Pareto and Gamma distributions (that are conjugate) in a bayesian estimation framework

December 9, 2014 17:48 ,

Javadtalab, A.; Semsarzadeh, M.; Khanchi, A.; Shirmohammadi, S.; Yassine, A., Continuous One-Way Detection of Available Bandwidth Changes for Video Streaming Over Best-Effort Networks, Instrumentation and Measurement, IEEE Transactions on , vol.64, no.1, pp.190,203, Jan. 2015. DOI: 10.1109/TIM.2014.2331423

Video streaming over best-effort networks, such as the Internet, is now a significant application used by most Internet users. However, best-effort networks are characterized by dynamic and unpredictable changes in the available bandwidth, which adversely affect the quality of video. As such, it is important to have real-time detection mechanisms of bandwidth changes to ensure that video is adapted to the available bandwidth and transmitted at the highest quality. In this paper, we propose a Bayesian instantaneous end-to-end bandwidth change prediction model and method to detect and predict one-way bandwidth changes at the receiver. Unlike existing congestion detection mechanisms, which use network parameters such as packet loss probability, round trip time (RTT), or jitter, our approach uses weighted interarrival time of video packets at the receiver side. Furthermore, our approach is continuous, since it measures available bandwidth changes with each incoming video packet, and therefore detects congestion occurrence in <200 ms, on average, which is significantly faster than existing approaches. In addition, it is a one-way scheme, since it only takes into account the characteristics of the incoming path and not the outgoing path, as opposed to other approaches, which use RTT and are hence less accurate. In this paper, we provide extensive experimental simulations and real-world network implementation. Our results indicate that the proposed detection method is superior to existing solutions.

Posted in: QoS in networked telerobots , Tagged: , ,

Estimating states of a human teleoperator and studying their influence in performing control

December 5, 2014 11:54 ,

Yunyi Jia, Ning Xi, Shuang Liu, Yunxia Wang, Xin Li, and Sheng Bi, Quality of teleoperator adaptive control for telerobotic operations The International Journal of Robotics Research December 2014 33: 1765-1781, first published on November 13, 2014. DOI: 10.1177/0278364914556124

Extensive studies have been conducted on telerobotic operations for decades due to their widespread applications in a variety of areas. Most studies have been focused on two major issues: stability and telepresence. Few have studied the influence of the operation status of the teleoperator on the performance of telerobotic operations. As subnormal operation status of the teleoperator may result in insufficient and even incorrect operations, the quality of teleoperator (QoT) is an important impact on the performance of the telerobotic operations in terms of the efficiency and safety even if both the stability and telepresence are guaranteed. Therefore, this paper investigates the online identification of the QoT and its application to telerobotic operations. The QoT is identified based on five QoT indicators which are generated based on the teleoperator’s brain EEG signals. A QoT adaptive control method is designed to adapt the velocity and responsivity of the robotic system to the operation status of the teleoperator such that the teleoperation efficiency and safety can be enhanced. The online QoT identification method was conducted on various teleoperators and the QoT adaptive control method was implemented on a mobile manipulator teleoperation system. The experimental results demonstrated the effectiveness and advantages of the proposed methods.

Posted in: Human teleoperation, Networked telerobots , Tagged: