Tag Archives: Optimization

Relation between optimization and reinforcement learning

Megumi Miyashita, Shiro Yano, Toshiyuki Kondo Mirror descent search and its acceleration, Robotics and Autonomous Systems, Volume 106, 2018, Pages 107-116 DOI: 10.1016/j.robot.2018.04.009.

In recent years, attention has been focused on the relationship between black-box optimization problem and reinforcement learning problem. In this research, we propose the Mirror Descent Search (MDS) algorithm which is applicable both for black box optimization problems and reinforcement learning problems. Our method is based on the mirror descent method, which is a general optimization algorithm. The contribution of this research is roughly twofold. We propose two essential algorithms, called MDS and Accelerated Mirror Descent Search (AMDS), and two more approximate algorithms: Gaussian Mirror Descent Search (G-MDS) and Gaussian Accelerated Mirror Descent Search (G-AMDS). This research shows that the advanced methods developed in the context of the mirror descent research can be applied to reinforcement learning problem. We also clarify the relationship between an existing reinforcement learning algorithm and our method. With two evaluation experiments, we show our proposed algorithms converge faster than some state-of-the-art methods.

Optimization algorithms inspired in chemical reactions

Nazmul Siddique, Hojjat Adeli, Nature-Inspired Chemical Reaction Optimisation Algorithms, Cognitive Computation, Volume 9, Issue 4, pp 411–422, DOI: 10.1007/s12559-017-9485-1.

Nature-inspired meta-heuristic algorithms have dominated the scientific literature in the areas of machine learning and cognitive computing paradigm in the last three decades. Chemical reaction optimisation (CRO) is a population-based meta-heuristic algorithm based on the principles of chemical reaction. A chemical reaction is seen as a process of transforming the reactants (or molecules) through a sequence of reactions into products. This process of transformation is implemented in the CRO algorithm to solve optimisation problems. This article starts with an overview of the chemical reactions and how it is applied to the optimisation problem. A review of CRO and its variants is presented in the paper. Guidelines from the literature on the effective choice of CRO parameters for solution of optimisation problems are summarised.

Very efficient global non-linear optimization for real-time robotic problems through the re-use of pre-computed solutions

K. Hauser, “Learning the Problem-Optimum Map: Analysis and Application to Global Optimization in Robotics,” in IEEE Transactions on Robotics, vol. 33, no. 1, pp. 141-152, Feb. 2017. DOI: 10.1109/TRO.2016.2623345.

This paper describes a data-driven framework for approximate global optimization in which precomputed solutions to a sample of problems are retrieved and adapted during online use to solve novel problems. This approach has promise for real-time applications in robotics, since it can produce near globally optimal solutions orders of magnitude faster than standard methods. This paper establishes theoretical conditions on how many and where samples are needed over the space of problems to achieve a given approximation quality. The framework is applied to solve globally optimal collision-free inverse kinematics problems, wherein large solution databases are used to produce near-optimal solutions in a submillisecond time on a standard PC.