Monthly Archives: June 2023

You are browsing the site archives by month.

Combination of analytical models with NN learning for predicting action effects

June 28, 2023 11:47 ,

Kloss A, Schaal S, Bohg J. , Combining learned and analytical models for predicting action effects from sensory data . The International Journal of Robotics Research. 2022;41(8):778-797 DOI: 10.1177/0278364920954896.

One of the most basic skills a robot should possess is predicting the effect of physical interactions with objects in the environment. This enables optimal action selection to reach a certain goal state. Traditionally, dynamics are approximated by physics-based analytical models. These models rely on specific state representations that may be hard to obtain from raw sensory data, especially if no knowledge of the object shape is assumed. More recently, we have seen learning approaches that can predict the effect of complex physical interactions directly from sensory input. It is, however, an open question how far these models generalize beyond their training data. In this work, we investigate the advantages and limitations of neural-network-based learning approaches for predicting the effects of actions based on sensory input and show how analytical and learned models can be combined to leverage the best of both worlds. As physical interaction task, we use planar pushing, for which there exists a well-known analytical model and a large real-world dataset. We propose the use of a convolutional neural network to convert raw depth images or organized point clouds into a suitable representation for the analytical model and compare this approach with using neural networks for both, perception and prediction. A systematic evaluation of the proposed approach on a very large real-world dataset shows two main advantages of the hybrid architecture. Compared with a pure neural network, it significantly (i) reduces required training data and (ii) improves generalization to novel physical interaction.

Posted in: Developmental robotics , Tagged: ,

“Early exit” deep neural networks (i.e., CNN that provide outputs at intermediate points)

June 28, 2023 11:36 ,

Scardapane, S., Scarpiniti, M., Baccarelli, E. et al. , Why Should We Add Early Exits to Neural Networks? . Cogn Comput 12, 954–966 (2020) DOI: 10.1007/s12559-020-09734-4.

Deep neural networks are generally designed as a stack of differentiable layers, in which a prediction is obtained only after running the full stack. Recently, some contributions have proposed techniques to endow the networks with early exits, allowing to obtain predictions at intermediate points of the stack. These multi-output networks have a number of advantages, including (i) significant reductions of the inference time, (ii) reduced tendency to overfitting and vanishing gradients, and (iii) capability of being distributed over multi-tier computation platforms. In addition, they connect to the wider themes of biological plausibility and layered cognitive reasoning. In this paper, we provide a comprehensive introduction to this family of neural networks, by describing in a unified fashion the way these architectures can be designed, trained, and actually deployed in time-constrained scenarios. We also describe in-depth their application scenarios in 5G and Fog computing environments, as long as some of the open research questions connected to them.

Posted in: Psycho-physiological bases of engineering , Tagged:

A so-called universal approach for modelling and controlling robots

June 28, 2023 10:25 ,

Tarokh, M., A unified kinematics modeling, optimization and control of universal robots: from serial and parallel manipulators to walking, rolling and hybrid robots, . Auton Robot 44, 1233–1248 (2020) DOI: 10.1007/s10514-020-09929-6.

The paper develops a unified kinematics modeling, optimization and control that is applicable to a wide range of autonomous and non-autonomous robots. These include hybrid robots that combine two or more modes of operations, such as combination of walking and rolling, or rolling and manipulation, as well as parallel robots in various configurations. The equations of motion are derived in compact forms that embed an optimization criterion. These equations are used to obtain various useful forms of the robot kinematics such as recursive, body and limb-end kinematic forms. Using the modeling, actuation and control equations are derived that ensure traversing a desired path while maintaining balanced operations and tip-over avoidance. Various simulation results are provided for a hybrid rolling-walking robot, which demonstrate the capabilities and effectiveness of the developed methodologies.

Posted in: Robot models , Tagged: , ,

Improving the realism of a simulator through deep learning

June 28, 2023 10:21 ,

Allevato, A.D., Schaertl Short, E., Pryor, M. et al. , Iterative residual tuning for system identification and sim-to-real robot learning, . Auton Robot 44, 1167–1182 (2020) DOI: 10.1007/s10514-020-09925-w.

Robots are increasingly learning complex skills in simulation, increasing the need for realistic simulation environments. Existing techniques for approximating real-world physics with a simulation require extensive observation data and/or thousands of simulation samples. This paper presents iterative residual tuning (IRT), a deep learning system identification technique that modifies a simulator’s parameters to better match reality using minimal real-world observations. IRT learns to estimate the parameter difference between two parameterized models, allowing repeated iterations to converge on the true parameters similarly to gradient descent. In this paper, we develop and analyze IRT in depth, including its similarities and differences with gradient descent. Our IRT implementation, TuneNet, is pre-trained via supervised learning over an auto-generated simulated dataset. We show that TuneNet can perform rapid, efficient system identification even when the true parameter values lie well outside those in the network’s training data, and can also learn real-world parameter values from visual data. We apply TuneNet to a sim-to-real task transfer experiment, allowing a robot to perform a dynamic manipulation task with a new object after a single observation.

Posted in: Robot models , Tagged: ,

Improving the simulation-to-real transfer of learning robotic skills by learning smaller skills and how to connect them in reality

June 26, 2023 12:25 ,

Julian RC, Heiden E, He Z, et al., Scaling simulation-to-real transfer by learning a latent space of robot skills, . The International Journal of Robotics Research. 2020;39(10-11):1259-1278 DOI: 10.1177/0278364920944474.

We present a strategy for simulation-to-real transfer, which builds on recent advances in robot skill decomposition. Rather than focusing on minimizing the simulation–reality gap, we propose a method for increasing the sample efficiency and robustness of existing simulation-to-real approaches which exploits hierarchy and online adaptation. Instead of learning a unique policy for each desired robotic task, we learn a diverse set of skills and their variations, and embed those skill variations in a continuously parameterized space. We then interpolate, search, and plan in this space to find a transferable policy which solves more complex, high-level tasks by combining low-level skills and their variations. In this work, we first characterize the behavior of this learned skill space, by experimenting with several techniques for composing pre-learned latent skills. We then discuss an algorithm which allows our method to perform long-horizon tasks never seen in simulation, by intelligently sequencing short-horizon latent skills. Our algorithm adapts to unseen tasks online by repeatedly choosing new skills from the latent space, using live sensor data and simulation to predict which latent skill will perform best next in the real world. Importantly, our method learns to control a real robot in joint-space to achieve these high-level tasks with little or no on-robot time, despite the fact that the low-level policies may not be perfectly transferable from simulation to real, and that the low-level skills were not trained on any examples of high-level tasks. In addition to our results indicating a lower sample complexity for families of tasks, we believe that our method provides a promising template for combining learning-based methods with proven classical robotics algorithms such as model-predictive control.

Posted in: Applications of reinforcement learning to robots , Tagged: ,

The problems of the initial state in filtering and its effects in the estimation

June 26, 2023 12:18 ,

He Kong, Mao Shan, Daobilige Su, Yongliang Qiao, Abdullah Al-Azzawi, Salah Sukkarieh, Filtering for systems subject to unknown inputs without a priori initial information, . Automatica, Volume 120, 2020 DOI: 10.1016/j.automatica.2020.109122.

The last few decades have witnessed much development in filtering of systems with Gaussian noises and arbitrary unknown inputs. Nonetheless, there are still some important design questions that warrant thorough discussions. Especially, the existing literature has shown that for unbiased and minimum variance estimation of the state and the unknown input, the initial guess of the state has to be unbiased. This clearly raises the question of whether and under what conditions one can design an unbiased and minimum variance filter, without making such a stringent assumption. The above-mentioned question will be investigated systematically in this paper, i.e., design of the filter is sought to be independent of a priori information about the initial conditions. In particular, for both cases with and without direct feedthrough, we establish necessary and sufficient conditions for unbiased and minimum variance estimation of the state/unknown input, independently of a priori initial conditions, respectively. When the former conditions do not hold, we carry out a thorough analysis of all possible scenarios. For each scenario, we present detailed discussions regarding whether and what can be achieved in terms of unbiased estimation, independently of a priori initial conditions. Extensions to the case with time-delays, conceptually like Kalman smoothing where future measurements are allowed in estimation, will also be presented, amongst others.

Shunyi Zhao, Biao Huang, Trial-and-error or avoiding a guess? Initialization of the Kalman filter, . Automatica, Volume 121, 2020 DOI: 10.1016/j.automatica.2020.109184.

As a recursive state estimation algorithm, the Kalman filter (KF) assumes initial state distribution is known a priori, while in practice the initial distribution is commonly treated as design parameters. In this paper, we will answer three questions concerning initialization: (1) At each time step, how does the KF respond to measurements, control signals, and more importantly, initial states? (2) What is the price (in terms of accuracy) one has to pay if inaccurate initial states are used? and (3) Can we find a better strategy rather than through guessing to improve the performance of KF in the initial estimation phase when the initial condition is unknown? To these ends, the classical recursive KF is first transformed into an equivalent but batch form, from which the responses of the KF to measurements, control signal, and initial state can be clearly separated and observed. Based on this, we isolate the initial distribution by dividing the original state into two parts and reconstructing a new state-space model. An initialization algorithm is then proposed by employing the Bayesian inference technique to estimate all the unknown variables simultaneously. By analyzing its performance, an improved version is further developed. Two simulation examples demonstrate that the proposed initialization approaches can be considered as competitive alternatives of various existing initialization methods when initial condition is unknown.

Posted in: Bayesian filtering , Tagged: ,

A new theory: we are curious about tasks that increase our ability to solve as many future tasks as possible

June 26, 2023 12:12 ,

Franziska Brändle, Charley M. Wu, Eric Schulz, What Are We Curious about?, . Trends in Cognitive Sciences, Volume 24, Issue 9, 2020 DOI: 10.1016/j.tics.2020.05.010.

(no abstract).

Posted in: Psycho-physiological bases of engineering, Reinforcement learning in AI , Tagged:

Predicting optimistically seems to lead to better response of the agent to achieve the best goals

June 26, 2023 12:09 ,

Zekun Sun, Chaz Firestone, Optimism and Pessimism in the Predictive Brain, . Trends in Cognitive Sciences, Volume 24, Issue 9, 2020 DOI: 10.1016/j.tics.2020.06.001.

(no abstract).

Posted in: Psycho-physiological bases of engineering , Tagged:

Combination of RL with human provided models for navigation

June 26, 2023 11:58 ,

Amarildo Likmeta, Alberto Maria Metelli, Andrea Tirinzoni, Riccardo Giol, Marcello Restelli, Danilo Romano, Combining reinforcement learning with rule-based controllers for transparent and general decision-making in autonomous driving, . Robotics and Autonomous Systems, Volume 131, 2020 DOI: 10.1016/j.robot.2020.103568.

The design of high-level decision-making systems is a topical problem in the field of autonomous driving. In this paper, we combine traditional rule-based strategies and reinforcement learning (RL) with the goal of achieving transparency and robustness. On the one hand, the use of handcrafted rule-based controllers allows for transparency, i.e., it is always possible to determine why a given decision was made, but they struggle to scale to complex driving scenarios, in which several objectives need to be considered. On the other hand, black-box RL approaches enable us to deal with more complex scenarios, but they are usually hardly interpretable. In this paper, we combine the best properties of these two worlds by designing parametric rule-based controllers, in which interpretable rules can be provided by domain experts and their parameters are learned via RL. After illustrating how to apply parameter-based RL methods (PGPE) to this setting, we present extensive numerical simulations in the highway and in two urban scenarios: intersection and roundabout. For each scenario, we show the formalization as an RL problem and we discuss the results of our approach in comparison with handcrafted rule-based controllers and black-box RL techniques.

Posted in: Applications of reinforcement learning to robots, Robot motion planning , Tagged:

Fast and more exact triangulation method for robot localization using range measurements

June 26, 2023 11:53 ,

Pınar Oğuz-Ekim, Lambiotte R., Lefebvre E., TDOA based localization and its application to the initialization of LiDAR based autonomous robots, . Robotics and Autonomous Systems, Volume 131, 2020, DOI: 10.1016/j.robot.2020.103590.

This work considers the problem of locating a single robot given a set of squared noisy range difference measurements to a set of points (anchors) whose positions are known. In the sequel, localization problem is solved in the Least-Squares (LS) sense by writing the robot position in polar/spherical coordinates. This representation transforms the original nonconvex/multimodal cost function into the quotient of two quadratic forms, whose constrained maximization is more tractable than the original problem. Simulation results indicate that the proposed method has similar accuracy to state-of-the-art optimization-based localization algorithms in its class, and the simple algorithmic structure and computational efficiency makes it appealing for applications with strong computational constraints. Additionally, location information is used to find the initial orientation of the robot with respect to the previously obtained map in scan matching. Thus, the crucial problem of the autonomous initialization and localization in robotics is solved.

Posted in: Mobile Robot Localization , Tagged: ,