Category Archives: Control Engineering

Achieving smooth motion in robotic manipulators on-line through their controller, and a nice state-of-the-art of the problem of smooth motion

February 7, 2018 08:52 ,

Yu-Sheng Lu, Yi-Yi Lin, Smooth motion control of rigid robotic manipulators with constraints on high-order kinematic variables, Mechatronics,
Volume 49, 2018, Pages 11-25, DOI: 10.1016/j.mechatronics.2017.11.003.

This paper presents a design for a jerk-constrained, time-optimal controller (JCTOC) that allows the smooth control of rigid robotic manipulators, in which time-optimal output responses are attained with confined jerk. A snap-constrained, time-optimal control (SCTOC) scheme is also proposed to produce even smoother output responses that are time-optimal, with a constraint on the maximum admissible snap. In contrast to conventional path-planning approaches that involve a bounded jerk/snap, the proposed JCTOC and SCTOC practically limit the corresponding high-order kinematic variables in real time. Using the structure of the computed torque control, the PD control, the JCTOC and the SCTOC are experimentally compared in terms of specific performance indices, including a chatter index, which is used to measure the unevenness of a signal.

Posted in: Control Engineering, Robot motion planning , Tagged:

Interesting study about the concepts to be taught in control system engineering

November 1, 2017 09:08 ,

R. M. Reck, Common Learning Objectives for Undergraduate Control Systems Laboratories,IEEE Transactions on Education, vol. 60, no. 4, pp. 257-264, DOI: 10.1109/TE.2017.2681624.

Course objectives, like research objectives and product requirements, help provide clarity and direction for faculty and students. Unfortunately, course and laboratory objectives are not always clearly stated. Without a clear set of objectives, it can be hard to design a learning experience and determine whether students are achieving the intended outcomes of the course or laboratory. In this paper, a common set of laboratory objectives, concepts, and components of a laboratory apparatus for undergraduate control systems laboratories were identified. A panel of 40 control systems faculty members completed a multi-round Delphi survey to bring them toward consensus on the common aspects of their laboratories. These panelists identified 15 laboratory objectives, 26 concepts, and 15 components common to their laboratories. Then an 45 additional faculty members and practitioners completed a follow-up survey to gather feedback on the results. In both surveys, each participant rated the importance of each item. While average ratings differed slightly between the two groups, the order in which the items were ranked was similar. Important examples of common learning objectives include connecting theory to what is implemented in the laboratory, designing controllers, and modeling systems. The most common component in both groups was MathWorks software. Some of the common concepts include block diagrams, stability, and PID control. Defining common aspects of undergraduate control systems laboratories enables common development, detailed comparisons, and simplified adaptation of equipment and experiments between campuses and programs.

Posted in: Control Engineering, Education

An interesting simulation educational software for control systems engineering based on controlling a quadrotor

November 1, 2017 09:04 ,

S. Khan, M. H. Jaffery, A. Hanif and M. R. Asif, Teaching Tool for a Control Systems Laboratory Using a Quadrotor as a Plant in MATLAB, IEEE Transactions on Education, vol. 60, no. 4, pp. 249-256, DOI: 10.1109/TE.2017.2653762.

This paper presents a MATLAB-based application to teach the guidance, navigation, and control concepts of a quadrotor to undergraduate students, using a graphical user interface (GUI) and 3-D animations. The Simulink quadrotor model is controlled by a proportional integral derivative controller and a linear quadratic regulator controller. The GUI layout’s many components can be easily programmed to perform various experiments by considering the simulation of the quadrotor as a plant; it incorporates control systems (CS) fundamentals such as time domain response, transfer function and state-space form, pole-zero location, root locus, frequency domain response, steady-state error, position and disturbance response, controller design and tuning, unity, and the use of a Kalman filter as a feedback sensor. 3-D animations are used to display the quadrotor flying in any given condition selected by the user. For each simulation, users can view the output response in the form of 3-D animations, and can run time plots. The quadrotor educational tool (QET) helps students in the CS laboratory understand basic CS concepts. The QET was evaluated based on student feedback, grades, satisfaction, and interest in CS.

Posted in: Control Engineering, Education , Tagged: , , ,

Value iteration applied in control systems when the model of the plant is substituted by data acquired from the plant

June 27, 2017 08:33 ,

Yongqiang Li, Zhongsheng Hou, Yuanjing Feng, Ronghu Chi, Data-driven approximate value iteration with optimality error bound analysis, Automatica, Volume 78, April 2017, Pages 79-87, ISSN 0005-1098, DOI: 10.1016/j.automatica.2016.12.019.

Features of the data-driven approximate value iteration (AVI) algorithm, proposed in Li et al. (2014) for dealing with the optimal stabilization problem, include that only process data is required and that the estimate of the domain of attraction for the closed-loop is enlarged. However, the controller generated by the data-driven AVI algorithm is an approximate solution for the optimal control problem. In this work, a quantitative analysis result on the error bound between the optimal cost and the cost under the designed controller is given. This error bound is determined by the approximation error of the estimation for the optimal cost and the approximation error of the controller function estimator. The first one is concretely determined by the approximation error of the data-driven dynamic programming (DP) operator to the DP operator and the approximation error of the value function estimator. These three approximation errors are zeros when the data set of the plant is sufficient and infinitely complete, and the number of samples in the interested state space is infinite. This means that the cost under the designed controller equals to the optimal cost when the number of iterations is infinite.

NOTE: Another paper on the same issue in the same journal.

Posted in: Applications of reinforcement learning to control engineering , Tagged: ,

A study of the influence of uncertain, stochastic delays in the stability of LTI SISO systems

June 22, 2017 11:54 ,

T. Qi, J. Zhu and J. Chen, “Fundamental Limits on Uncertain Delays: When Is a Delay System Stabilizable by LTI Controllers?,” in IEEE Transactions on Automatic Control, vol. 62, no. 3, pp. 1314-1328, March 2017. DOI: 10.1109/TAC.2016.2584007.

This paper concerns the stabilization of linear time-invariant (LTI) systems subject to uncertain, possibly time-varying delays. The fundamental issue under investigation, referred to as the delay margin problem, addresses the question: What is the largest range of delay such that there exists a single LTI feedback controller capable of stabilizing all the plants for delays within that range? Drawing upon analytic interpolation and rational approximation techniques, we derive fundamental bounds on the delay margin, within which the delay plant is guaranteed to be stabilizable by a certain LTI output feedback controller. Our contribution is threefold. First, for single-input single-output (SISO) systems with an arbitrary number of plant unstable poles and nonminimum phase zeros, we provide an explicit, computationally efficient bound on the delay margin, which requires computing only the largest real eigenvalue of a constant matrix. Second, for multi-input multi-output (MIMO) systems, we show that estimates on the variation ranges of multiple delays can be obtained by solving LMI problems, and further, by finding bounds on the radius of delay variations. Third, we show that these bounds and estimates can be extended to systems subject to time-varying delays. When specialized to more specific cases, e.g., to plants with one unstable pole but possibly multiple nonminimum phase zeros, our results give rise to analytical expressions exhibiting explicit dependence of the bounds and estimates on the pole and zeros, thus demonstrating how fundamentally unstable poles and nonminimum phase zeros may limit the range of delays over which a plant can be stabilized by a LTI controller.

Posted in: Control Engineering , Tagged: ,

Model-based reinforcement learning with a reduced number of basis functions to aproximate the value function, a study of its convergence guarantees, and a nice state of the art on the use of (mdel-based) reinforcement learning for automatic control

June 20, 2017 11:05 ,

Rushikesh Kamalapurkar, Joel A. Rosenfeld, Warren E. Dixon, Efficient model-based reinforcement learning for approximate online optimal control, Automatica, Volume 74, 2016, Pages 247-258, ISSN 0005-1098, DOI: 10.1016/j.automatica.2016.08.004.

An infinite horizon optimal regulation problem is solved online for a deterministic control-affine nonlinear dynamical system using a state following (StaF) kernel method to approximate the value function. Unlike traditional methods that aim to approximate a function over a large compact set, the StaF kernel method aims to approximate a function in a small neighborhood of a state that travels within a compact set. Simulation results demonstrate that stability and approximate optimality of the control system can be achieved with significantly fewer basis functions than may be required for global approximation methods.

Posted in: Applications of reinforcement learning to control engineering , Tagged: ,

Value iteration applied to continuous LTI systems control

June 8, 2017 11:19 ,

Tao Bian, Zhong-Ping Jiang, Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design, Automatica, Volume 71, September 2016, Pages 348-360, ISSN 0005-1098, DOI: 10.1016/j.automatica.2016.05.003.

This paper presents a novel non-model-based, data-driven adaptive optimal controller design for linear continuous-time systems with completely unknown dynamics. Inspired by the stochastic approximation theory, a continuous-time version of the traditional value iteration (VI) algorithm is presented with rigorous convergence analysis. This VI method is crucial for developing new adaptive dynamic programming methods to solve the adaptive optimal control problem and the stochastic robust optimal control problem for linear continuous-time systems. Fundamentally different from existing results, the a priori knowledge of an initial admissible control policy is no longer required. The efficacy of the proposed methodology is illustrated by two examples and a brief comparative study between VI and earlier policy-iteration methods.

Posted in: Control Engineering , Tagged:

Cognitive control: a nice bunch of definitions and state-of-the-art

June 6, 2017 12:20 ,

S. Haykin, M. Fatemi, P. Setoodeh and Y. Xue, Cognitive Control, in Proceedings of the IEEE, vol. 100, no. 12, pp. 3156-3169, Dec. 2012., DOI: 10.1109/JPROC.2012.2215773.

This paper is inspired by how cognitive control manifests itself in the human brain and does so in a remarkable way. It addresses the many facets involved in the control of directed information flow in a dynamic system, culminating in the notion of information gap, defined as the difference between relevant information (useful part of what is extracted from the incoming measurements) and sufficient information representing the information needed for achieving minimal risk. The notion of information gap leads naturally to how cognitive control can itself be defined. Then, another important idea is described, namely the two-state model, in which one is the system’s state and the other is the entropic state that provides an essential metric for quantifying the information gap. The entropic state is computed in the perceptual part (i.e., perceptor) of the dynamic system and sent to the controller directly as feedback information. This feedback information provides the cognitive controller the information needed about the environment and the system to bring reinforcement leaning into play; reinforcement learning (RL), incorporating planning as an integral part, is at the very heart of cognitive control. The stage is now set for a computational experiment, involving cognitive radar wherein the cognitive controller is enabled to control the receiver via the environment. The experiment demonstrates how RL provides the mechanism for improved utilization of computational resources, and yet is able to deliver good performance through the use of planning. The paper finishes with concluding remarks.

Posted in: Control Engineering , Tagged: , , ,

Incremental (hierarchical) search for the optimal policy on markov decision processes

April 20, 2016 16:26 ,

Vu Anh Huynh, Sertac Karaman, and Emilio Frazzoli, An incremental sampling-based algorithm for stochastic optimal control, The International Journal of Robotics Research April 2016 35: 305-333, DOI: 10.1177/0278364915616866.

In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Using the Markov chain approximation method and recent advances in sampling-based algorithms for deterministic path planning, we propose a novel algorithm called the incremental Markov Decision Process to incrementally compute control policies that approximate arbitrarily well an optimal policy in terms of the expected cost. The main idea behind the algorithm is to generate a sequence of finite discretizations of the original problem through random sampling of the state space. At each iteration, the discretized problem is a Markov Decision Process that serves as an incrementally refined model of the original problem. We show that with probability one, (i) the sequence of the optimal value functions for each of the discretized problems converges uniformly to the optimal value function of the original stochastic optimal control problem, and (ii) the original optimal value function can be computed efficiently in an incremental manner using asynchronous value iterations. Thus, the proposed algorithm provides an anytime approach to the computation of optimal control policies of the continuous problem. The effectiveness of the proposed approach is demonstrated on motion planning and control problems in cluttered environments in the presence of process noise.

Posted in: Control Engineering , Tagged:

Reinforcement learning in the automatic control area

December 22, 2015 17:41 ,

Yu Jiang; Zhong-Ping Jiang, Global Adaptive Dynamic Programming for Continuous-Time Nonlinear Systems, in Automatic Control, IEEE Transactions on , vol.60, no.11, pp.2917-2929, Nov. 2015, DOI: 10.1109/TAC.2015.2414811.

This paper presents a novel method of global adaptive dynamic programming (ADP) for the adaptive optimal control of nonlinear polynomial systems. The strategy consists of relaxing the problem of solving the Hamilton-Jacobi-Bellman (HJB) equation to an optimization problem, which is solved via a new policy iteration method. The proposed method distinguishes from previously known nonlinear ADP methods in that the neural network approximation is avoided, giving rise to significant computational improvement. Instead of semiglobally or locally stabilizing, the resultant control policy is globally stabilizing for a general class of nonlinear polynomial systems. Furthermore, in the absence of the a priori knowledge of the system dynamics, an online learning method is devised to implement the proposed policy iteration technique by generalizing the current ADP theory. Finally, three numerical examples are provided to validate the effectiveness of the proposed method.

Posted in: Applications of reinforcement learning to control engineering , Tagged: ,