Category Archives: Control Engineering

A remote Matlab laboratory for LTI system identification

Z. Lei, H. Zhou, W. Hu and G. -P. Liu, Teaching and Comprehensive Learning With Remote Laboratories and MATLAB for an Undergraduate System Identification Course, EEE Transactions on Education, vol. 65, no. 3, pp. 402-408, Aug. 2022 DOI: 10.1109/TE.2021.3123302.

Contribution: This article introduces the teaching and learning with remote laboratories and MATLAB for an undergraduate system identification (SI) course, which can be employed for students at the advanced level with a control background. Background: SI has been widely used in all engineering fields; thus, the SI course that includes complex theories, concepts, and formulas is crucial for engineering education. Constraints, such as time, space, cost, and maintenance work, pose limitations for conventional laboratories, and current remote laboratories may not offer experiences to enhance control-oriented practical skills. Intended Outcomes: The proposed teaching and learning using remote laboratories is intended to facilitate the understanding of theories and concepts, and enhance the ability of design and implementation of control algorithms, the conducting of experiments, data collecting, data analysis, and the conducting of SI with MATLAB. Application Design: In the classroom teaching, theoretical lectures regarding SI are delivered to students by the teacher, along with the classroom demonstration with the networked control system laboratory for online experimentation. Then, the laboratory work is required to be completed by the students using the remote laboratory. A tailored laboratory report is supposed to be handed in by each student after the experimentation. Findings: The effectiveness of the proposed method was evaluated through the analysis of student performance and student responses to surveys. The student performance analysis indicates that the application of the remote laboratories is effective, and the feedback from students shows that they can benefit from the application of remote laboratories, and they would like the remote laboratories to be expanded to other courses.

Abstraction of continuous control problems considered as MDPs

H. G. Tanner and A. Stager, Data-Driven Abstractions for Robots With Stochastic Dynamics, IEEE Transactions on Robotics, vol. 38, no. 3, pp. 1686-1702, June 2022 DOI: 10.1109/TRO.2021.3119209.

This article describes the construction of stochastic, data-based discrete abstractions for uncertain random processes continuous in time and space. Motivated by the fact that modeling processes often introduce errors which interfere with the implementation of control strategies, here the abstraction process proceeds in reverse: the methodology does not abstract models; rather it models abstractions. Specifically, it first formalizes a template for a family of stochastic abstractions, and then fits the parameters of that template to match the dynamics of the underlying process and ground the abstraction. The article also shows how the parameter-fitting approach can be implemented based on a probabilistic model validation approach which draws from randomized algorithms, and results in a discrete abstract model which is approximately simulated by the actual process physics, at a desired confidence level. In this way, the models afford the implementation of symbolic control plans with probabilistic guarantees at a desired level of fidelity.

RL in manufacturing control

Vladimir Samsonov, Karim Ben Hicham, Tobias Meisen, Reinforcement Learning in Manufacturing Control: Baselines, challenges and ways forward, Engineering Applications of Artificial Intelligence, Volume 112, 2022 DOI: 10.1016/j.engappai.2022.104868.

The field of Neural Combinatorial Optimization (NCO) offers multiple learning-based approaches to solve well-known combinatorial optimization tasks such as Traveling Salesman or Knapsack problem capable of competing with classical optimization approaches in terms of both solution quality and speed. This brought the attention of the research community to the tasks of Manufacturing Control (MC) with combinatorial nature. In this paper we outline the main components of MC tasks, select the most promising application fields and analyze dedicated learning-based solutions available in the literature. We draw multiple parallels to the current state of the art in the NCO field and allocate the main research gaps and directions on the perception, cognition and interaction levels. Using a set of practical examples we implement and benchmark common design patterns for single-agent Reinforcement Learning (RL) solutions. Along with testing existing solutions, we build on the ranked reward idea (Laterre et al., 2018) and offer a novel Multi-Instance Ranked Reward (m-R2) approach tailored to MC optimization tasks. It minimizes the reward shaping effort and defines a suitable training curriculum for more stable learning by separately tracking the agent\u2019s performance on every scheduling task and rewarding only policies contributing towards better scheduling solutions. We implement all solution design patterns as a set of interchangeable modules with a shared API, unified in a benchmarking framework with the focus on standardization of training and evaluation processes, reproducibility and simplified experiment lifecycle management. In addition to the framework, we make available our discrete-event simulation of a job shop production.


Zhihao Liu, Quan Liu, Wenjun Xu, Lihui Wang, Zude Zhou,
Robot learning towards smart robotic manufacturing: A review,
Robotics and Computer-Integrated Manufacturing,
Volume 77,
ISSN 0736-5845,

Shorter exploration stage in RL through the use of expert (a PID) that sets the expectation of the explored action

J. Enrique Sierra-Garcia, Matilde Santos, Ravi Pandit, Wind turbine pitch reinforcement learning control improved by PID regulator and learning observer, Engineering Applications of Artificial Intelligence, Volume 111, 2022 DOI: 10.1016/j.engappai.2022.104769.

Wind turbine (WT) pitch control is a challenging issue due to the non-linearities of the wind device and its complex dynamics, the coupling of the variables and the uncertainty of the environment. Reinforcement learning (RL) based control arises as a promising technique to address these problems. However, its applicability is still limited due to the slowness of the learning process. To help alleviate this drawback, in this work we present a hybrid RL-based control that combines a RL-based controller with a proportional\u2013integral\u2013derivative (PID) regulator, and a learning observer. The PID is beneficial during the first training episodes as the RL based control does not have any experience to learn from. The learning observer oversees the learning process by adjusting the exploration rate and the exploration window in order to reduce the oscillations during the training and improve convergence. Simulation experiments on a small real WT show how the learning significantly improves with this control architecture, speeding up the learning convergence up to 37%, and increasing the efficiency of the intelligent control strategy. The best hybrid controller reduces the error of the output power by around 41% regarding a PID regulator. Moreover, the proposed intelligent hybrid control configuration has proved more efficient than a fuzzy controller and a neuro-control strategy.

A practical setup for control engineering courses

A. Chevalier, K. Dekemele, J. Juchem and M. Loccufier, Student Feedback on Educational Innovation in Control Engineering: Active Learning in Practice, IEEE Transactions on Education, vol. 64, no. 4, pp. 432-437, Nov. 2021, DOI: 10.1109/TE.2021.3077278.

Contribution: An education innovation in control engineering using practical setups and its evaluation based on a three-year student feedback study and examination grades. Background: Based on extensive research, education\u2019s transition toward active learning and more practical experience has been shown to increase learning outcomes. Contrary to virtual and remote labs, a practical session with an individual setup for each student provides the most practical experience. Intended Outcomes: To show a positive effect on learning performance by integrating practical sessions in basic control engineering. Application Design: Presenting low cost setups that can be mass produced and adapt to the course\u2019s growing complexity. These setups are evaluated during a three-year feedback study. Findings: The developed setups increased understanding of theoretical concepts. The new methodology significantly improved students\u2019 average grades. The students\u2019 interest in control theory is triggered. This case study could guide other institutions toward successfully implementing highly individual practical sessions for large groups.

Using a physical simulator for sampled rollouts in stochastic optimal control

Carius J, Ranftl R, Farshidian F, Hutter M. Constrained stochastic optimal control with learned importance sampling: A path integral approach, The International Journal of Robotics Research. 2022;41(2):189-209, DOI: 10.1177/02783649211047890.

Modern robotic systems are expected to operate robustly in partially unknown environments. This article proposes an algorithm capable of controlling a wide range of high-dimensional robotic systems in such challenging scenarios. Our method is based on the path integral formulation of stochastic optimal control, which we extend with constraint-handling capabilities. Under our control law, the optimal input is inferred from a set of stochastic rollouts of the system dynamics. These rollouts are simulated by a physics engine, placing minimal restrictions on the types of systems and environments that can be modeled. Although sampling-based algorithms are typically not suitable for online control, we demonstrate in this work how importance sampling and constraints can be used to effectively curb the sampling complexity and enable real-time control applications. Furthermore, the path integral framework provides a natural way of incorporating existing control architectures as ancillary controllers for shaping the sampling distribution. Our results reveal that even in cases where the ancillary controller would fail, our stochastic control algorithm provides an additional safety and robustness layer. Moreover, in the absence of an existing ancillary controller, our method can be used to train a parametrized importance sampling policy using data from the stochastic rollouts. The algorithm may thereby bootstrap itself by learning an importance sampling policy offline and then refining it to unseen environments during online control. We validate our results on three robotic systems, including hardware experiments on a quadrupedal robot.

Identifying state-space-models of systems with autoencoders

Daniele Masti, Alberto Bemporad, Learning nonlinear state–space models using autoencoders, . Automatica, Volume 129, 2021 DOI: 10.1016/j.automatica.2021.109666.

We propose a methodology for the identification of nonlinear state–space models from input/output data using machine-learning techniques based on autoencoders and neural networks. Our framework simultaneously identifies the nonlinear output and state-update maps of the model. After formulating the approach and providing guidelines for tuning the related hyper-parameters (including the model order), we show its capability in fitting nonlinear models on different nonlinear system identification benchmarks. Performance is assessed in terms of open-loop prediction on test data and of controlling the system via nonlinear model predictive control (MPC) based on the identified nonlinear state–space model.

Cubature (fixed point representation of uncertainties, as in UKF) Kalman Filter

Juan-Carlos Santos-León, Ramón Orive, Daniel Acosta, Leopoldo Acosta, The Cubature Kalman Filter revisited, . Automatica, Volume 127, 2021 DOI: 10.1016/j.automatica.2021.109541.

In this paper, the construction and effectiveness of the so-called Cubature Kalman Filter (CKF) is revisited, as well as its extensions for higher degrees of precision. In this sense, some stable (with respect to the dimension) cubature rules with a quasi-optimal number of nodes are built, and their numerical performance is checked in comparison with other known formulas. All these cubature rules are suitably placed in the mathematical framework of numerical integration in several variables. A method based on the discretization of higher order partial derivatives by certain divided differences is used to provide stable rules of degrees d=5 and d=7, though it can also be applied for higher dimensions. The application of these old and new formulas to the filter algorithm is tested by means of some examples.

Learning the parameters of Bernoulli for modelling the transmission times in remote control with known plant dynamics

Konstantinos Gatsis, George J. Pappas, Statistical learning for analysis of networked control systems over unknown channels, . Automatica, Volume 125, 2021 DOI: 10.1016/j.automatica.2020.109386.

Recent control trends are increasingly relying on communication networks and wireless channels to close the loop for Internet-of-Things applications. Traditionally these approaches are model-based, i.e., assuming a network or channel model they are focused on stability analysis and appropriate controller designs. However the availability of such wireless channel modeling is fundamentally challenging in practice as channels are typically unknown a priori and only available through data samples. In this work we aim to develop algorithms that rely on channel sample data to determine the mean square stability and performance of networked control tasks. In this regard our work is the first to characterize the amount of channel modeling that is required to answer such a question. Specifically we examine how many channel data samples are required in order to answer with high confidence whether a given networked control system is stable or not. This analysis is based on the notion of sample complexity from the learning literature and is facilitated by concentration inequalities. Moreover we establish a direct relation between the sample complexity and the networked system stability margin, i.e., the underlying packet success rate of the channel and the spectral radius of the dynamics of the control system. This illustrates that it becomes impractical to verify stability under a large range of plant and channel configurations. We validate our theoretical results in numerical simulations.

Abstraction of controllers

Stanley W. Smith, Murat Arcak, Majid Zamani, Approximate abstractions of control systems with an application to aggregation, Automatica, 119 (2020) DOI: 10.1016/j.automatica.2020.109065.

Previous approaches to constructing abstractions for control systems rely on geometric conditions or, in the case of an interconnected control system, a condition on the interconnection topology. Since these conditions are not always satisfiable, we relax the restrictions on the choice of abstractions, instead opting to select ones which nearly satisfy such conditions via optimization-based approaches. To quantify the resulting effect on the error between the abstraction and concrete control system, we introduce the notions of practical simulation functions and practical storage functions. We show that our approach facilitates the procedure of aggregation, where one creates an abstraction by partitioning agents into aggregate areas. We demonstrate the results on an application where we regulate the temperature in three separate zones of a building.