Category Archives: Mathematics

A new method for estimating inertial sensor signals

December 14, 2017 09:26 ,

M. Ghobadi, P. Singla and E. T. Esfahani, Robust Attitude Estimation from Uncertain Observations of Inertial Sensors Using Covariance Inflated Multiplicative Extended Kalman Filter, IEEE Transactions on Instrumentation and Measurement, vol. 67, no. 1, pp. 209-217, DOI: 10.1109/TIM.2017.2761230.

This paper presents an attitude estimation method from uncertain observations of inertial sensors, which is highly robust against different uncertainties. The proposed method of covariance inflated multiplicative extended Kalman filter (CI-MEKF) takes the advantage of non-singularity of covariance in MEKF as well as a novel covariance inflation (CI) approach to fuse inconsistent information. The proposed CI approach compensates the undesired effect of magnetic distortion and body acceleration (as inherent biases of magnetometer and accelerometer sensors data, respectively) on the estimated attitude. Moreover, the CI-MEKF can accurately estimate the gyro bias. A number of simulation scenarios are designed to compare the performance of the proposed method with the state of the art in attitude estimation. The results show the proposed method outperforms the state of the art in terms of estimation accuracy and robustness. Moreover, the proposed CI-MEKF method is shown to be significantly robust against different uncertainties, such as large body acceleration, magnetic distortion, and errors, in the initial condition of the attitude.

Posted in: Bayesian filtering , Tagged:

A new method for nonlinear optimization aimed to embedded computers, and a nice state of the art of that problem

December 7, 2017 09:08 ,

N. Y. Chiang, R. Huang and V. M. Zavala, An Augmented Lagrangian Filter Method for Real-Time Embedded Optimization, IEEE Transactions on Automatic Control, vol. 62, no. 12, pp. 6110-6121, DOI: 10.1109/TAC.2017.2694806.

We present a filter line-search algorithm for nonconvex continuous optimization that combines an augmented Lagrangian function and a constraint violation metric to accept and reject steps. The approach is motivated by real-time optimization applications that need to be executed on embedded computing platforms with limited memory and processor speeds. The proposed method enables primal-dual regularization of the linear algebra system that in turn permits the use of solution strategies with lower computing overheads. We prove that the proposed algorithm is globally convergent and we demonstrate the developments using a nonconvex real-time optimization application for a building heating, ventilation, and air conditioning system. Our numerical tests are performed on a standard processor and on an embedded platform. We demonstrate that the approach reduces solution times by a factor of over 1000.

Posted in: Mathematics, Real-Time Systems , Tagged: ,

Extending bayesian fusion from Euclidean spaces to Lie groups

October 25, 2017 09:09 ,

Kevin C. Wolfe, Michael Mashner, Gregory S. Chirikjian, Bayesian Fusion on Lie Groups, JOURNAL OF ALGEBRAIC STATISTICS Vol. 2, No. 1, 2011, 75-97, DOI: 10.18409/jas.v2i1.11.

An increasing number of real-world problems involve the measurement of data, and the computation of estimates, on Lie groups. Moreover, establishing confidence in the resulting estimates is important. This paper therefore seeks to contribute to a larger theoretical framework that generalizes classical multivariate statistical analysis from Euclidean space to the setting of Lie groups. The particular focus here is on extending Bayesian fusion, based on exponential families of probability densities, from the Euclidean setting to Lie groups. The definition and properties of a new kind of Gaussian distribution for connected unimodular Lie groups are articulated, and explicit formulas and algorithms are given for finding the mean and covariance of the fusion model based on the means and covariances of the constituent probability densities. The Lie groups that find the most applications in engineering are rotation groups and groups of rigid-body motions. Orientational (rotation-group) data and associated algorithms for estimation arise in problems including satellite attitude, molecular spectroscopy, and global geological studies. In robotics and manufacturing, quantifying errors in the position and orientation of tools and parts are important for task performance and quality control. Developing a general way to handle problems on Lie groups can be applied to all of these problems. In particular, we study the issue of how to ‘fuse’ two such Gaussians and how to obtain a new Gaussian of the same form that is ‘close to’ the fused density.This is done at two levels of approximation that result from truncating the Baker-Campbell-Hausdorff formula with different numbers of terms. Algorithms are developed and numerical results are presented that are shown to generate the equivalent fused density with good accuracy

Posted in: Bayesian filtering , Tagged:

Kalman Filter as the extreme case of finite impulse response filters as the horizon increases in length

October 16, 2017 08:42 ,

Shunyi Zhao, Biao Huang, Yuriy S. Shmaliy, Bayesian state estimation on finite horizons: The case of linear state–space model,Automatica, Volume 85, 2017, Pages 91-99, DOI: 10.1016/j.automatica.2017.07.043.

The finite impulse response (FIR) filter and infinite impulse response filter including the Kalman filter (KF) are generally considered as two different types of state estimation methods. In this paper, the sequential Bayesian philosophy is extended to a filter design using a fixed amount of most recent measurements, with the aim of exploiting the FIR structure and unifying some basic FIR filters with the KF. Specifically, the conditional mean and covariance of the posterior probability density functions are first derived to show the FIR counterpart of the KF. To remove the dependence on initial states, the corresponding likelihood is further maximized and realized iteratively. It shows that the maximum likelihood modification unifies the existing unbiased FIR filters by tuning a weighting matrix. Moreover, it converges to the Kalman estimate with the increase of horizon length, and can thus be considered as a link between the FIR filtering and the KF. Several important properties including stability and robustness against errors of noise statistics are illustrated. Finally, a moving target tracking example and an experiment with a three degrees-of-freedom helicopter system are introduced to demonstrate effectiveness.

Posted in: Bayesian filtering , Tagged: ,

Example of learning a Bayesian network using expert knowledge

October 5, 2017 08:55 ,

H. Amirkhani, M. Rahmati, P. J. F. Lucas and A. Hommersom, Exploiting Experts’ Knowledge for Structure Learning of Bayesian Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 11, pp. 2154-2170, DOI: 10.1109/TPAMI.2016.2636828.

Learning Bayesian network structures from data is known to be hard, mainly because the number of candidate graphs is super-exponential in the number of variables. Furthermore, using observational data alone, the true causal graph is not discernible from other graphs that model the same set of conditional independencies. In this paper, it is investigated whether Bayesian network structure learning can be improved by exploiting the opinions of multiple domain experts regarding cause-effect relationships. In practice, experts have different individual probabilities of correctly labeling the inclusion or exclusion of edges in the structure. The accuracy of each expert is modeled by three parameters. Two new scoring functions are introduced that score each candidate graph based on the data and experts’ opinions, taking into account their accuracy parameters. In the first scoring function, the experts’ accuracies are estimated using an expectation-maximization-based algorithm and the estimated accuracies are explicitly used in the scoring process. The second function marginalizes out the accuracy parameters to obtain more robust scores when it is not possible to obtain a good estimate of experts’ accuracies. The experimental results on simulated and real world datasets show that exploiting experts’ knowledge can improve the structure learning if we take the experts’ accuracies into account.

Posted in: Probability and statistics , Tagged: , ,

Dealing with nonlinearities in Kalman filters through Monte Carlo modelling for minimizing divergence

October 5, 2017 08:51 ,

S. Gultekin and J. Paisley, Nonlinear Kalman Filtering With Divergence Minimization, IEEE Transactions on Signal Processing, vol. 65, no. 23, pp. 6319-6331, DOI: 10.1109/TSP.2017.2752729.

We consider the nonlinear Kalman filtering problem using Kullback-Leibler (KL) and α-divergence measures as optimization criteria. Unlike linear Kalman filters, nonlinear Kalman filters do not have closed form Gaussian posteriors because of a lack of conjugacy due to the nonlinearity in the likelihood. In this paper, we propose novel algorithms to approximate this posterior by optimizing the forward and reverse forms of the KL divergence, as well as the α-divergence that contains these two as limiting cases. Unlike previous approaches, our algorithms do not make approximations to the divergences being optimized, but use Monte Carlo techniques to derive unbiased algorithms for direct optimization. We assess performance on radar and sensor tracking, and options pricing, showing general improvement over the extended, unscented, and ensemble Kalman filters, as well as competitive performance with particle filtering.

Posted in: Bayesian filtering , Tagged:

On how the simplification on physics made in computer games for real-time execution can explain the simplification on physics made by infants when understanding the world

August 17, 2017 11:39 ,

Tomer D. Ullman, Elizabeth Spelke, Peter Battaglia, Joshua B. Tenenbaum, Mind Games: Game Engines as an Architecture for Intuitive Physics, Trends in Cognitive Sciences, Volume 21, Issue 9, 2017, Pages 649-665, DOI: 10.1016/j.tics.2017.05.012.

We explore the hypothesis that many intuitive physical inferences are based on a mental physics engine that is analogous in many ways to the machine physics engines used in building interactive video games. We describe the key features of game physics engines and their parallels in human mental representation, focusing especially on the intuitive physics of young infants where the hypothesis helps to unify many classic and otherwise puzzling phenomena, and may provide the basis for a computational account of how the physical knowledge of infants develops. This hypothesis also explains several ‘physics illusions’, and helps to inform the development of artificial intelligence (AI) systems with more human-like common sense.

Posted in: Probability and statistics, Psycho-physiological bases of engineering , Tagged:

The problem of the interdependence among particles in PF after the resampling step, and an approach to solve it

August 10, 2017 10:05 ,

R. Lamberti, Y. Petetin, F. Desbouvries and F. Septier, Independent Resampling Sequential Monte Carlo Algorithms, IEEE Transactions on Signal Processing, vol. 65, no. 20, pp. 5318-5333, DOI: 10.1109/TSP.2017.2726971.

Sequential Monte Carlo algorithms, or particle filters, are Bayesian filtering algorithms, which propagate in time a discrete and random approximation of the a posteriori distribution of interest. Such algorithms are based on importance sampling with a bootstrap resampling step, which aims at struggling against weight degeneracy. However, in some situations (informative measurements, high-dimensional model), the resampling step can prove inefficient. In this paper, we revisit the fundamental resampling mechanism, which leads us back to Rubin’s static resampling mechanism. We propose an alternative rejuvenation scheme in which the resampled particles share the same marginal distribution as in the classical setup, but are now independent. This set of independent particles provides a new alternative to compute a moment of the target distribution and the resulting estimate is analyzed through a CLT. We next adapt our results to the dynamic case and propose a particle filtering algorithm based on independent resampling. This algorithm can be seen as a particular auxiliary particle filter algorithm with a relevant choice of the first-stage weights and instrumental distributions. Finally, we validate our results via simulations, which carefully take into account the computational budget.

Posted in: Bayesian filtering , Tagged: ,

Clustering in hypergraphs

August 10, 2017 09:45 ,

P. Purkait, T. J. Chin, A. Sadri and D. Suter, Clustering with Hypergraphs: The Case for Large Hyperedges, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 9, pp. 1697-1711, DOI: 10.1109/TPAMI.2016.2614980.

The extension of conventional clustering to hypergraph clustering, which involves higher order similarities instead of pairwise similarities, is increasingly gaining attention in computer vision. This is due to the fact that many clustering problems require an affinity measure that must involve a subset of data of size more than two. In the context of hypergraph clustering, the calculation of such higher order similarities on data subsets gives rise to hyperedges. Almost all previous work on hypergraph clustering in computer vision, however, has considered the smallest possible hyperedge size, due to a lack of study into the potential benefits of large hyperedges and effective algorithms to generate them. In this paper, we show that large hyperedges are better from both a theoretical and an empirical standpoint. We then propose a novel guided sampling strategy for large hyperedges, based on the concept of random cluster models. Our method can generate large pure hyperedges that significantly improve grouping accuracy without exponential increases in sampling costs. We demonstrate the efficacy of our technique on various higher-order grouping problems. In particular, we show that our approach improves the accuracy and efficiency of motion segmentation from dense, long-term, trajectories.

Posted in: Graph theory , Tagged: ,

POMDPs with multicriteria in the cost to optimize – a hierarchical approach

August 7, 2017 09:25 ,

Seyedshams Feyzabadi, Stefano Carpin, Planning using hierarchical constrained Markov decision processes, Autonomous Robots, Volume 41, Issue 8, pp 1589–1607, DOI: 10.1007/s10514-017-9630-4.

Constrained Markov decision processes offer a principled method to determine policies for sequential stochastic decision problems where multiple costs are concurrently considered. Although they could be very valuable in numerous robotic applications, to date their use has been quite limited. Among the reasons for their limited adoption is their computational complexity, since policy computation requires the solution of constrained linear programs with an extremely large number of variables. To overcome this limitation, we propose a hierarchical method to solve large problem instances. States are clustered into macro states and the parameters defining the dynamic behavior and the costs of the clustered model are determined using a Monte Carlo approach. We show that the algorithm we propose to create clustered states maintains valuable properties of the original model, like the existence of a solution for the problem. Our algorithm is validated in various planning problems in simulation and on a mobile robot platform, and we experimentally show that the clustered approach significantly outperforms the non-hierarchical solution while experiencing only moderate losses in terms of objective functions.

Posted in: Artificial Intelligence, Probability and statistics, Robot task planning , Tagged: , ,