Regression to help in finding the optimal policy in MDPs based on duality theory

April 26, 2018 07:44 ,

H. Zhu, F. Ye and E. Zhou, Solving the Dual Problems of Dynamic Programs via Regression, IEEE Transactions on Automatic Control, vol. 63, no. 5, pp. 1340-1355, DOI: 10.1109/TAC.2017.2747405.

In recent years, information relaxation and duality in dynamic programs have been studied extensively, and the resulted primal-dual approach has become a powerful procedure in solving dynamic programs by providing lower-upper bounds on the optimal value function. Theoretically, with the so-called value-based optimal dual penalty, the optimal value function could be recovered exactly via strong duality. However, in practice, obtaining tight dual bounds usually requires good approximations of the optimal dual penalty, which could be time consuming if analytical computation is not possible and nested simulation has to be used to estimate the conditional expectations inside the dual penalty. In this paper, we will develop a framework of a regression approach to approximating the optimal dual penalty in a nonnested manner, by exploring the structure of the function space consisting of all feasible dual penalties. The resulted approximations maintain to be feasible dual penalties, and thus yielding valid dual bounds on the optimal value function. We show that the proposed framework is computationally efficient, and the resulted dual penalties lead to numerically tractable dual problems. Finally, we apply the framework to a high-dimensional dynamic trading problem to demonstrate its effectiveness in solving the dual problems of complex dynamic programs.

Posted in: Probability and statistics , Tagged: ,

A mathematical study of controllers that produce paths with beautfiul shapes to reach a target point by a unicycle vehicle

April 26, 2018 07:25 ,

T. Tripathy and A. Sinha, Unicycle With Only Range Input: An Array of Patterns, IEEE Transactions on Automatic Control, vol. 63, no. 5, pp. 1300-1312, DOI: 10.1109/TAC.2017.2736940.

The objective of this paper is to generate planar patterns using an autonomous agent modeled as a unicycle. The patterns are generated about a stationary point referred to as the target. To achieve the same, the paper proposes a family of control inputs that are continuous functions of range, which is the distance between the unicycle and the target. The paper studies in detail a characterization of the resulting trajectories, which are a plethora of patterns of parametric curves (circles, spirals, epicyclic curves like hypotrochoids) and more. These appealing patterns find applications in exploration, coverage, land mine detection, etc., where the target represents any point of interest like a landmark or a beacon. The paper also investigates the necessary conditions on the control laws in order to generate patterns of desired shapes and bounds. Furthermore, to generate desired patterns with arbitrary initial conditions, a switching strategy is proposed which is illustrated using an algorithm. The paper presents a series of simulations of appealing patterns generated using the proposed control laws.

Posted in: Control Engineering, Robot motion planning

How a robot can learn to recognize itself on a mirror

April 20, 2018 07:50 ,

Zeng, Y., Zhao, Y., Bai, J. et al., Toward Robot Self-Consciousness (II): Brain-Inspired Robot Bodily Self Model for Self-Recognition, Cogn Comput (2018) 10: 307, DOI: 10.1007/s12559-017-9505-1.

The neural correlates and nature of self-consciousness is an advanced topic in Cognitive Neuroscience. Only a few animal species have been testified to be with this cognitive ability. From artificial intelligence and robotics point of view, few efforts are deeply rooted in the neural correlates and brain mechanisms of biological self-consciousness. Despite the fact that the scientific understanding of biological self-consciousness is still in preliminary stage, we make our efforts to integrate and adopt known biological findings of self-consciousness to build a brain-inspired model for robot self-consciousness. In this paper, we propose a brain-inspired robot bodily self model based on extensions to primate mirror neuron system and apply it to humanoid robot for self recognition. In this model, the robot firstly learns the correlations between self-generated actions and visual feedbacks in motion by learning with spike timing dependent plasticity (STDP), and then learns the appearance of body part with the expectation that the visual feedback is consistent with its motion. Based on this model, the robot uses multisensory integration to learn its own body in real world and in mirror. Then it can distinguish itself from others. In a mirror test setting with three robots with the same appearance, with the proposed brain-inspired robot bodily self model, each of them can recognize itself in the mirror after these robots make random movements at the same time. The theoretic modeling and experimental validations indicate that the brain-inspired robot bodily self model is biologically inspired, and computationally feasible as a foundation for robot self recognition.

Posted in: Developmental robotics, Psycho-physiological bases of engineering , Tagged:

Methods for estimating periods of noisy signals

April 19, 2018 07:44 ,

W. Fan, Y. Li, K. L. Tsui and Q. Zhou, A Noise Resistant Correlation Method for Period Detection of Noisy Signals, IEEE Transactions on Signal Processing, vol. 66, no. 10, pp. 2700-2710, DOI: 10.1109/TSP.2018.2813305.

This paper develops a novel method called the noise resistant correlation method for detecting the hidden period from the contaminated (noisy) signals with strong white Gaussian noise. A novel correlation function is proposed based on a newly constructed periodic signal and the contaminated signal to effectively detect the target hidden period. In contrast with the conventional autocorrelation analysis (AUTOC) method, this method demonstrates excellent performance, especially when facing strong noise. Fault diagnoses of rolling element bearings and gears are presented as application examples and the performance of the proposed method is compared with that of the AUTOC method.

Another paper about this in the same issue: 10.1109/TSP.2018.2818080.

Posted in: Electronics , Tagged: ,

A novel algorithm for coverage path planning with very strong guarantees

April 19, 2018 07:38 ,

J. Song and S. Gupta, $varepsilon ^{star }$: An Online Coverage Path Planning Algorithm, IEEE Transactions on Robotics, vol. 34, no. 2, pp. 526-533, DOI: 10.1109/TRO.2017.2780259.

This paper presents an algorithm called ε*, for online coverage path planning of unknown environment. The algorithm is built upon the concept of an Exploratory Turing Machine (ETM), which acts as a supervisor to the autonomous vehicle to guide it with adaptive navigation commands. The ETM generates a coverage path online using Multiscale Adaptive Potential Surfaces (MAPS), which are hierarchically structured and dynamically updated based on sensor information. The ε*-algorithm is computationally efficient, guarantees complete coverage, and does not suffer from the local extrema problem. Its performance is validated by 1) high-fidelity simulations on Player/Stage and 2) actual experiments in a laboratory setting on autonomous vehicles.

Posted in: Robot motion planning, Robot task planning , Tagged:

Rao-Blackwellized Particle Filter SLAM with grid maps in which particles do not contain the whole map but only a part

April 19, 2018 07:32 ,

H. Jo, H. M. Cho, S. Jo and E. Kim, Efficient Grid-Based Rao–Blackwellized Particle Filter SLAM With Interparticle Map Sharing, IEEE/ASME Transactions on Mechatronics, vol. 23, no. 2, pp. 714-724, DOI: 10.1109/TMECH.2018.2795252.

In this paper, we propose a novel and efficient grid-based Rao-Blackwellized particle filter simultaneous localization and mapping (RBPF-SLAM) with interparticle map shaping (IPMS). The proposed method aims at saving the computational memory in the grid-based RBPF-SLAM while maintaining the mapping accuracy. Unlike conventional RBPF-SLAM in which each particle has its own map of the whole environment, each particle has only a small map of the nearby environment called an individual map in the proposed method. Instead, the map of the remaining large environment is shared by the particles. The part shared by the particles is called a base map. If the individual small maps become reliable enough to trust, they are merged with the base map. To determine when and which part of an individual map should be merged with the base map, we propose two map sharing criteria. Finally, the proposed IPMS RBPF-SLAM is applied to the real-world datasets and benchmark datasets. The experimental results show that our method outperforms conventional methods in terms of map accuracy versus memory consumption.

Posted in: Mobile robot SLAM , Tagged: , ,

What is Cognitive Computational Neuroscience

April 18, 2018 07:29 ,

Thomas Naselaris, Danielle S. Bassett, Alyson K. Fletcher, Konrad Kording, Nikolaus Kriegeskorte, Hendrikje Nienborg, Russell A. Poldrack, Daphna Shohamy, Kendrick Kay, Cognitive Computational Neuroscience: A New Conference for an Emerging Discipline, Trends in Cognitive Sciences, Volume 22, Issue 5, 2018, Pages 365-367, DOI: 10.1016/j.tics.2018.02.008.

Understanding the computational principles that underlie complex behavior is a central goal in cognitive science, artificial intelligence, and neuroscience. In an attempt to unify these disconnected communities, we created a new conference called Cognitive Computational Neuroscience (CCN). The inaugural meeting revealed considerable enthusiasm but significant obstacles remain.

Posted in: Cognitive sciences, Psycho-physiological bases of engineering , Tagged:

Robot topological navigation

April 13, 2018 07:25 ,

Sergio Miguel-Tomé, Navigation through unknown and dynamic open spaces using topological notions,Connection Science vol. 30, iss. 2, DOI: 10.1080/09540091.2016.1277691.

Until now, most algorithms used for navigation have had the purpose of directing system towards one point in space. However, humans communicate tasks by specifying spatial relations among elements or places. In addition, the environments in which humans develop their activities are extremely dynamic. The only option that allows for successful navigation in dynamic and unknown environments is making real-time decisions. Therefore, robots capable of collaborating closely with human beings must be able to make decisions based on the local information registered by the sensors and interpret and express spatial relations. Furthermore, when one person is asked to perform a task in an environment, this task is communicated given a category of goals so the person does not need to be supervised. Thus, two problems appear when one wants to create multifunctional robots: how to navigate in dynamic and unknown environments using spatial relations and how to accomplish this without supervision. In this article, a new architecture to address the two cited problems is presented, called the topological qualitative navigation architecture. In previous works, a qualitative heuristic called the heuristic of topological qualitative semantics (HTQS) has been developed to establish and identify spatial relations. However, that heuristic only allows for establishing one spatial relation with a specific object. In contrast, navigation requires a temporal sequence of goals with different objects. The new architecture attains continuous generation of goals and resolves them using HTQS. Thus, the new architecture achieves autonomous navigation in dynamic or unknown open environments.

Posted in: Robot task planning , Tagged:

A novel fast algorithm for clock synchronization in a wireless network, with a nice introduction but assuming negligible communication times and thus not directly applicable in teleoperation

April 8, 2018 09:39 ,

Kan Xie, Qianqian Cai, Minyue Fu, A fast clock synchronization algorithm for wireless sensor networks, Automatica, Volume 92, 2018, Pages 133-142, DOI: 10.1016/j.automatica.2018.03.004.

This paper proposes a novel clock synchronization algorithm for wireless sensor networks (WSNs). The algorithm is derived using a fast finite-time average consensus idea, and is fully distributed, meaning that each node relies only on its local clock readings and reading announcements from its neighbours. For networks with an acyclic graph, the algorithm converges in only d iterations for clock rate synchronization and another d iterations for clock offset synchronization, where d is the graph diameter. The algorithm enjoys low computational and communicational complexities and robustness against transmission adversaries. Each node can execute the algorithm asynchronously without the need for global coordination. Due to its fast convergence, the algorithm is most suitable for large-scale WSNs. For WSNs with a cyclic graph, a fast distributed depth-first-search (DFS) algorithm can be applied first to form a spanning tree before applying the proposed synchronization algorithm.

Posted in: Communication networks , Tagged: ,

POMDPs aware of the data association problem

March 29, 2018 09:32 ,

Shashank Pathak, Antony Thomas, and Vadim Indelman, A unified framework for data association aware robust belief space planning and perception, The International Journal of Robotics Research Vol 37, Issue 2-3, pp. 287 – 315, DOI: 10.1177/0278364918759606.

We develop a belief space planning approach that advances the state of the art by incorporating reasoning about data association within planning, while considering additional sources of uncertainty. Existing belief space planning approaches typically assume that data association is given and perfect, an assumption that can be harder to justify during operation in the presence of localization uncertainty, or in ambiguous and perceptually aliased environments. By contrast, our data association aware belief space planning (DA-BSP) approach explicitly reasons about data association within belief evolution owing to candidate actions, and as such can better accommodate these challenging real-world scenarios. In particular, we show that, owing to perceptual aliasing, a posterior belief can become a mixture of probability distribution functions and design cost functions, which measure the expected level of ambiguity and posterior uncertainty given candidate action. Furthermore, we also investigate more challenging situations, such as when prior belief is multimodal and when data association aware planning is performed over several look-ahead steps. Our framework models the belief as a Gaussian mixture model. Another unique aspect of this approach is that the number of components of this Gaussian mixture model can increase as well as decrease, thereby reflecting reality more accurately. Using these and standard costs (e.g. control penalty, distance to goal) within the objective function yields a general framework that reliably represents action impact and, in particular, is capable of active disambiguation. Our approach is thus applicable to both robust perception in a passive setting with data given a priori and in an active setting, such as in autonomous navigation in perceptually aliased environments. We demonstrate key aspects of DA-BSP in a theoretical example, in a Gazebo-based realistic simulation, and also on the real robotic platform using a Pioneer robot in an office environment.

Posted in: Probability and statistics, Robot task planning , Tagged: ,