Category Archives: Control Engineering

Taking into account the influence of a recommender in the change of behaviour of the agent using it

Jonathan P. Epperlein, Sergiy Zhuk, Robert Shorten, Recovering Markov models from closed-loop data, Automatica, Volume 103, 2019, Pages 116-125, DOI: 10.1016/j.automatica.2019.01.022.

Situations in which recommender systems are used to augment decision making are becoming prevalent in many application domains. Almost always, these prediction tools (recommenders) are created with a view to affecting behavioural change. Clearly, successful applications actuating behavioural change, affect the original model underpinning the predictor, leading to an inconsistency. This feedback loop is often not considered in standard machine learning techniques which rely upon machine learning/statistical learning machinery. The objective of this paper is to develop tools that recover unbiased user models in the presence of recommenders. More specifically, we assume that we observe a time series which is a trajectory of a Markov chain R modulated by another Markov chain S, i.e. the transition matrix of R is unknown and depends on the current state of S. The transition matrix of the latter is also unknown. In other words, at each time instant, S selects a transition matrix for R within a given set which consists of known and unknown matrices. The state of S, in turn, depends on the current state of R thus introducing a feedback loop. We propose an Expectation–Maximisation (EM) type algorithm, which estimates the transition matrices of S and R. Experimental results are given to demonstrate the efficacy of the approach.

Model-based RL for controling a soft manipulator arm

T. G. Thuruthel, E. Falotico, F. Renda and C. Laschi, Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators, IEEE Transactions on Robotics, vol. 35, no. 1, pp. 124-134, Feb. 2019. DOI: 10.1109/TRO.2018.2878318.

Dynamic control of soft robotic manipulators is an open problem yet to be well explored and analyzed. Most of the current applications of soft robotic manipulators utilize static or quasi-dynamic controllers based on kinematic models or linearity in the joint space. However, such approaches are not truly exploiting the rich dynamics of a soft-bodied system. In this paper, we present a model-based policy learning algorithm for closed-loop predictive control of a soft robotic manipulator. The forward dynamic model is represented using a recurrent neural network. The closed-loop policy is derived using trajectory optimization and supervised learning. The approach is verified first on a simulated piecewise constant strain model of a cable driven under-actuated soft manipulator. Furthermore, we experimentally demonstrate on a soft pneumatically actuated manipulator how closed-loop control policies can be derived that can accommodate variable frequency control and unmodeled external loads.

A novel method for compacting a continuous high-dimensional value function for MDPs

Gorodetsky, A., Karaman, S., & Marzouk, Y., High-dimensional stochastic optimal control using continuous tensor decompositions, The International Journal of Robotics Research, 37(2–3), 340–377, DOI: 10.1177/0278364917753994.

Motion planning and control problems are embedded and essential in almost all robotics applications. These problems are often formulated as stochastic optimal control problems and solved using dynamic programming algorithms. Unfortunately, most existing algorithms that guarantee convergence to optimal solutions suffer from the curse of dimensionality: the run time of the algorithm grows exponentially with the dimension of the state space of the system. We propose novel dynamic programming algorithms that alleviate the curse of dimensionality in problems that exhibit certain low-rank structure. The proposed algorithms are based on continuous tensor decompositions recently developed by the authors. Essentially, the algorithms represent high-dimensional functions (e.g. the value function) in a compressed format, and directly perform dynamic programming computations (e.g. value iteration, policy iteration) in this format. Under certain technical assumptions, the new algorithms guarantee convergence towards optimal solutions with arbitrary precision. Furthermore, the run times of the new algorithms scale polynomially with the state dimension and polynomially with the ranks of the value function. This approach realizes substantial computational savings in “compressible” problem instances, where value functions admit low-rank approximations. We demonstrate the new algorithms in a wide range of problems, including a simulated six-dimensional agile quadcopter maneuvering example and a seven-dimensional aircraft perching example. In some of these examples, we estimate computational savings of up to 10 orders of magnitude over standard value iteration algorithms. We further demonstrate the algorithms running in real time on board a quadcopter during a flight experiment under motion capture.

A formal definition of autonomy and of its degrees

Antsaklis, P.J. & Rahnama, A. , Control and Machine Intelligence for System Autonomy, Journal of Intelligent & Robotic Systems
July 2018, Volume 91, Issue 1, pp 23–34 DOI: 10.1007/s10846-018-0832-6.

Autonomous systems evolve from control systems by adding functionalities that increase the level of system autonomy. It is very important to the research in the field that autonomy be well defined and so in the present paper a precise, useful definition of autonomy is introduced and discussed. Autonomy is defined as the ability of the system to attain a set of goals under a set of uncertainties. This leads to the notion of degrees or levels of autonomy. The Quest for Autonomy in engineered systems throughout the centuries is noted, connections to research work of 30 years ago are made and a hierarchical functional architecture for autonomous systems together with needed functionalities are outlined. Adaptation and Learning, which are among the most important functions in achieving high levels of autonomy are then highlighted and recent research contributions are briefly discussed.

A survey on the concept of Entropy as a measure of the intelligence and autonomy of a system, modeled hierarchically

Valavanis, K.P., The Entropy Based Approach to Modeling and Evaluating Autonomy and Intelligence of Robotic Systems, J Intell Robot Syst (2018) 91: 7 DOI: 10.1007/s10846-018-0905-6.

This review paper presents the Entropy approach to modeling and performance evaluation of Intelligent Machines (IMs), which are modeled as hierarchical, multi-level structures. It provides a chronological summary of developments related to intelligent control, from its origins to current advances. It discusses fundamentals of the concept of Entropy as a measure of uncertainty and as a control function, which may be used to control, evaluate and improve through adaptation and learning performance of engineering systems. It describes a multi-level, hierarchical, architecture that is used to model such systems, and it defines autonomy and machine intelligence for engineering systems, with the aim to set foundations necessary to tackle related challenges. The modeling philosophy for the systems under consideration follows the mathematically proven principle of Increasing Precision with Decreasing Intelligence (IPDI). Entropy is also used in the context of N-Dimensional Information Theory to model the flow of information throughout such systems and contributes to quantitatively evaluate uncertainty, thus, autonomy and intelligence. It is explained how Entropy qualifies as a unique, single, measure to evaluate autonomy, intelligence and precision of task execution. The main contribution of this review paper is that it brings under one forum research findings from the 1970’s and 1980’s, and that it supports the argument that even today, given the unprecedented existing computational power, advances in Artificial Intelligence, Deep Learning and Control Theory, the same foundational framework may be followed to study large-scale, distributed Cyber Physical Systems (CPSs), including distributed intelligence and multi-agent systems, with direct applications to the SmartGrid, transportation systems and multi-robot teams, to mention but a few applications.

On the effects of delays in the stability of a network controlled plant due to both clocks not being synchronized

K. Okano, M. Wakaiki, G. Yang and J. P. Hespanha, Stabilization of Networked Control Systems Under Clock Offsets and Quantization, IEEE Transactions on Automatic Control, vol. 63, no. 6, pp. 1708-1723 DOI: 10.1109/TAC.2017.2753938.

This paper studies the impact of clock mismatches and quantization on networked control systems. We consider a scenario where the plant’s state is measured by a sensor that communicates with the controller through a network. Variable communication delays and clock jitter do not permit a perfect synchronization between the clocks of the sensor and controller. We investigate limitations on the clock offset tolerable for stabilization of the feedback system. For a process with a scalar-valued state, we show that there exists a tight bound on the offset above which the closed-loop system cannot be stabilized with any causal controllers. For higher dimensional plants, if the plant has two distinct poles, then the effect of clock mismatches can be canceled with a finite number of measurements, and hence there is no fundamental limitation. We also consider the case where the measurements are subject to quantization in addition to clock mismatches. For first-order plants, we present necessary conditions and sufficient conditions for stabilizability, which show that a larger clock offset requires a finer quantization.

On the use of flipped classroom for control engineering classes and its problem with the required (longer) time for learning

Y. Kim and C. Ahn, Effect of Combined Use of Flipped Learning and Inquiry-Based Learning on a System Modeling and Control Course, IEEE Transactions on Education, vol. 61, no. 2, pp. 136-142, DOI: 10.1109/TE.2017.2774194.

Contribution: This paper illustrates how to design and implement curricula in terms of the combined use of flipped learning and inquiry-based learning in an engineering course. Background: Elementary courses in engineering schools are conventional and foundational, and involve a considerable amount of knowledge. Throughout such courses, students are also expected to develop insight, which cannot be obtained by only listening to instructors. Having relevant discussions is also difficult for most instructors. Intended Outcomes: The combined use of flipped learning and inquiry-based learning would be beneficial to broaden student achievement. Application Design: Based on an epistemological approach about knowledge and knowing, this paper applies the combined use of flipped learning and inquiry-based learning to enhance student knowledge and advance ways of thinking on a System Modeling and Control course. Findings: The extended learning time and the collective responsibility for learning are discussed as critical issues in applying the combined use of flipped learning and inquiry-based learning in an engineering school.

A mathematical study of controllers that produce paths with beautfiul shapes to reach a target point by a unicycle vehicle

T. Tripathy and A. Sinha, Unicycle With Only Range Input: An Array of Patterns, IEEE Transactions on Automatic Control, vol. 63, no. 5, pp. 1300-1312, DOI: 10.1109/TAC.2017.2736940.

The objective of this paper is to generate planar patterns using an autonomous agent modeled as a unicycle. The patterns are generated about a stationary point referred to as the target. To achieve the same, the paper proposes a family of control inputs that are continuous functions of range, which is the distance between the unicycle and the target. The paper studies in detail a characterization of the resulting trajectories, which are a plethora of patterns of parametric curves (circles, spirals, epicyclic curves like hypotrochoids) and more. These appealing patterns find applications in exploration, coverage, land mine detection, etc., where the target represents any point of interest like a landmark or a beacon. The paper also investigates the necessary conditions on the control laws in order to generate patterns of desired shapes and bounds. Furthermore, to generate desired patterns with arbitrary initial conditions, a switching strategy is proposed which is illustrated using an algorithm. The paper presents a series of simulations of appealing patterns generated using the proposed control laws.

Using EKF estimation in a PI controller for improving its performance under noise

Y. Zhou, Q. Zhang, H. Wang, P. Zhou and T. Chai, EKF-Based Enhanced Performance Controller Design for Nonlinear Stochastic Systems, IEEE Transactions on Automatic Control, vol. 63, no. 4, pp. 1155-1162, DOI: 10.1109/TAC.2017.2742661.

In this paper, a novel control algorithm is presented to enhance the performance of the tracking property for a class of nonlinear and dynamic stochastic systems subjected to non-Gaussian noises. Although the existing standard PI controller can be used to obtain the basic tracking of the systems, the desired tracking performance of the stochastic systems is difficult to achieve due to the random noises. To improve the tracking performance, an enhanced performance loop is constructed using the EKF-based state estimates without changing the existing closed loop with a PI controller. Meanwhile, the gain of the enhanced performance loop can be obtained based upon the entropy optimization of the tracking error. In addition, the stability of the closed loop system is analyzed in the mean-square sense. The simulation results are given to illustrate the effectiveness of the proposed control algorithm.

Achieving smooth motion in robotic manipulators on-line through their controller, and a nice state-of-the-art of the problem of smooth motion

Yu-Sheng Lu, Yi-Yi Lin, Smooth motion control of rigid robotic manipulators with constraints on high-order kinematic variables, Mechatronics,
Volume 49, 2018, Pages 11-25, DOI: 10.1016/j.mechatronics.2017.11.003.

This paper presents a design for a jerk-constrained, time-optimal controller (JCTOC) that allows the smooth control of rigid robotic manipulators, in which time-optimal output responses are attained with confined jerk. A snap-constrained, time-optimal control (SCTOC) scheme is also proposed to produce even smoother output responses that are time-optimal, with a constraint on the maximum admissible snap. In contrast to conventional path-planning approaches that involve a bounded jerk/snap, the proposed JCTOC and SCTOC practically limit the corresponding high-order kinematic variables in real time. Using the structure of the computed torque control, the PD control, the JCTOC and the SCTOC are experimentally compared in terms of specific performance indices, including a chatter index, which is used to measure the unevenness of a signal.