Category Archives: Mathematics

A new Kalman Filter that is more robust under certain deviations of the gaussian hypothesis

June 22, 2017 09:46 ,

Badong Chen, Xi Liu, Haiquan Zhao, Jose C. Principe, Maximum correntropy Kalman filter, Automatica, Volume 76, February 2017, Pages 70-77, ISSN 0005-1098, DOI: 10.1016/j.automatica.2016.10.004.

Traditional Kalman filter (KF) is derived under the well-known minimum mean square error (MMSE) criterion, which is optimal under Gaussian assumption. However, when the signals are non-Gaussian, especially when the system is disturbed by some heavy-tailed impulsive noises, the performance of KF will deteriorate seriously. To improve the robustness of KF against impulsive noises, we propose in this work a new Kalman filter, called the maximum correntropy Kalman filter (MCKF), which adopts the robust maximum correntropy criterion (MCC) as the optimality criterion, instead of using the MMSE. Similar to the traditional KF, the state mean vector and covariance matrix propagation equations are used to give prior estimations of the state and covariance matrix in MCKF. A novel fixed-point algorithm is then used to update the posterior estimations. A sufficient condition that guarantees the convergence of the fixed-point algorithm is also given. Illustration examples are presented to demonstrate the effectiveness and robustness of the new algorithm.

Posted in: Bayesian filtering , Tagged: ,

A promising survey on robust estimation methods aimed at robotic applications

June 22, 2017 09:35 ,

Michael Bosse, Gabriel Agamennoni and Igor Gilitschenski (2016), “Robust Estimation and Applications in Robotics”, Foundations and Trends® in Robotics: Vol. 4: No. 4, pp 225-269. DOI: 10.1561/2300000047.

Solving estimation problems is a fundamental component of numerous robotics applications. Prominent examples involve pose estimation, point cloud alignment, or object tracking. Algorithms for solving these estimation problems need to cope with new challenges due to an increased use of potentially poor low-cost sensors, and an ever growing deployment of robotic algorithms in consumer products which operate in potentially unknown environments. These algorithms need to be capable of being robust against strong nonlinearities, high uncertainty levels, and numerous outliers. However, particularly in robotics, the Gaussian assumption is prevalent in solutions to multivariate parameter estimation problems without providing the desired level of robustness. The goal of this tutorial is helping to address the aforementioned challenges by providing an introduction to robust estimation with a particular focus on robotics. First, this is achieved by giving a concise overview of the theory on M-estimation. M-estimators share many of the convenient properties of least-squares estimators, and at the same time are much more robust to deviations from the Gaussian model assumption. Second, we present several example applications where M-Estimation is used to increase robustness against nonlinearities and outliers.

Posted in: Probability and statistics , Tagged:

A new method of clustering of data with many advantages w.r.t. others

June 22, 2017 08:44 ,

A. Sharma, K. A. Boroevich, D. Shigemizu, Y. Kamatani, M. Kubo and T. Tsunoda, “Hierarchical Maximum Likelihood Clustering Approach,” in IEEE Transactions on Biomedical Engineering, vol. 64, no. 1, pp. 112-122, Jan. 2017. DOI: 10.1109/TBME.2016.2542212.

In this paper, we focused on developing a clustering approach for biological data. In many biological analyses, such as multiomics data analysis and genome-wide association studies analysis, it is crucial to find groups of data belonging to subtypes of diseases or tumors. Methods: Conventionally, the k-means clustering algorithm is overwhelmingly applied in many areas including biological sciences. There are, however, several alternative clustering algorithms that can be applied, including support vector clustering. In this paper, taking into consideration the nature of biological data, we propose a maximum likelihood clustering scheme based on a hierarchical framework. Results: This method can perform clustering even when the data belonging to different groups overlap. It can also perform clustering when the number of samples is lower than the data dimensionality. Conclusion: The proposed scheme is free from selecting initial settings to begin the search process. In addition, it does not require the computation of the first and second derivative of likelihood functions, as is required by many other maximum likelihood-based methods. Significance: This algorithm uses distribution and centroid information to cluster a sample and was applied to biological data. A MATLAB implementation of this method can be downloaded from the web link http://www.riken.jp/en/research/labs/ims/med_sci_math/.

Posted in: Machine learning , Tagged: , ,

Bayesian estimation when computing the likelihood is hard

June 21, 2017 12:04 ,

Kirthevasan Kandasamy, Jeff Schneider, Barnabás Póczos, Query efficient posterior estimation in scientific experiments via Bayesian active learning, Artificial Intelligence, Volume 243, February 2017, Pages 45-56, ISSN 0004-3702, DOI: 10.1016/j.artint.2016.11.002.

A common problem in disciplines of applied Statistics research such as Astrostatistics is of estimating the posterior distribution of relevant parameters. Typically, the likelihoods for such models are computed via expensive experiments such as cosmological simulations of the universe. An urgent challenge in these research domains is to develop methods that can estimate the posterior with few likelihood evaluations.In this paper, we study active posterior estimation in a Bayesian setting when the likelihood is expensive to evaluate. Existing techniques for posterior estimation are based on generating samples representative of the posterior. Such methods do not consider efficiency in terms of likelihood evaluations. In order to be query efficient we treat posterior estimation in an active regression framework. We propose two myopic query strategies to choose where to evaluate the likelihood and implement them using Gaussian processes. Via experiments on a series of synthetic and real examples we demonstrate that our approach is significantly more query efficient than existing techniques and other heuristics for posterior estimation.

Posted in: Bayesian filtering

Subgraph matching (isomorphism) using GPUs for managing commonsense knowledge, and a short list of other graph problems that have had benefit from multiprocessing

June 21, 2017 10:48 ,

Ha-Nguyen Tran, Erik Cambria, Amir Hussain, Towards GPU-Based Common-Sense Reasoning: Using Fast Subgraph Matching, Cognitive Computation, December 2016, Volume 8, Issue 6, pp 1074–1086, DOI: 10.1007/s12559-016-9418-4.

Common-sense reasoning is concerned with simulating cognitive human ability to make presumptions about the type and essence of ordinary situations encountered every day. The most popular way to represent common-sense knowledge is in the form of a semantic graph. Such type of knowledge, however, is known to be rather extensive: the more concepts added in the graph, the harder and slower it becomes to apply standard graph mining techniques.In this work, we propose a new fast subgraph matching approach to overcome these issues. Subgraph matching is the task of finding all matches of a query graph in a large data graph, which is known to be a non-deterministic polynomial time-complete problem. Many algorithms have been previously proposed to solve this problem using central processing units. Here, we present a new graphics processing unit-friendly method for common-sense subgraph matching, termed GpSense, which is designed for scalable massively parallel architectures, to enable next-generation Big Data sentiment analysis and natural language processing applications.We show that GpSense outperforms state-of-the-art algorithms and efficiently answers subgraph queries on large common-sense graphs.

Posted in: Artificial Intelligence, Graph theory , Tagged: ,

Partially observed boolean dynamic systems

June 20, 2017 12:55 ,

M. Imani and U. M. Braga-Neto, “Maximum-Likelihood Adaptive Filter for Partially Observed Boolean Dynamical Systems,” in IEEE Transactions on Signal Processing, vol. 65, no. 2, pp. 359-371, Jan.15, 15 2017.DOI: 10.1109/TSP.2016.2614798.

We present a framework for the simultaneous estimation of state and parameters of partially observed Boolean dynamical systems (POBDS). Simultaneous state and parameter estimation is achieved through the combined use of the Boolean Kalman filter and Boolean Kalman smoother, which provide the minimum mean-square error state estimators for the POBDS model, and maximum-likelihood (ML) parameter estimation; in the presence of continuous parameters, ML estimation is performed using the expectation-maximization algorithm. The performance of the proposed ML adaptive filter is demonstrated by numerical experiments with a POBDS model of gene regulatory networks observed through noisy next-generation sequencing (RNA-seq) time series data using the well-known p53-MDM2 negative-feedback loop gene regulatory model.

Posted in: Bayesian filtering , Tagged: ,

Performing filtering on graphs instead of individual signals

June 20, 2017 12:52 ,

E. Isufi, A. Loukas, A. Simonetto and G. Leus, “Autoregressive Moving Average Graph Filtering,” in IEEE Transactions on Signal Processing, vol. 65, no. 2, pp. 274-288, Jan.15, 15 2017. DOI: 10.1109/TSP.2016.2614793.

One of the cornerstones of the field of signal processing on graphs are graph filters, direct analogs of classical filters, but intended for signals defined on graphs. This paper brings forth new insights on the distributed graph filtering problem. We design a family of autoregressive moving average (ARMA) recursions, which are able to approximate any desired graph frequency response, and give exact solutions for specific graph signal denoising and interpolation problems. The philosophy to design the ARMA coefficients independently from the underlying graph renders the ARMA graph filters suitable in static and, particularly, time-varying settings. The latter occur when the graph signal and/or graph topology are changing over time. We show that in case of a time-varying graph signal, our approach extends naturally to a two-dimensional filter, operating concurrently in the graph and regular time domain. We also derive the graph filter behavior, as well as sufficient conditions for filter stability when the graph and signal are time varying. The analytical and numerical results presented in this paper illustrate that ARMA graph filters are practically appealing for static and time-varying settings, as predicted by theoretical derivations.

Posted in: Graph theory, Probability and statistics , Tagged: , ,

A variant of particle filters that uses feedback to model how particles move towards the real posterior

June 8, 2017 11:17 ,

T. Yang, P.~G. Mehta, S.~P. Meyn, Feedback particle filter, IEEE Transactions on Automatic Control, 58 (10) (2013), pp. 2465â–2480, DOI: 10.1109/TAC.2013.2258825.

The feedback particle filter introduced in this paper is a new approach to approximate nonlinear filtering, motivated by techniques from mean-field game theory. The filter is defined by an ensemble of controlled stochastic systems (the particles). Each particle evolves under feedback control based on its own state, and features of the empirical distribution of the ensemble. The feedback control law is obtained as the solution to an optimal control problem, in which the optimization criterion is the Kullback-Leibler divergence between the actual posterior, and the common posterior of any particle. The following conclusions are obtained for diffusions with continuous observations: 1) The optimal control solution is exact: The two posteriors match exactly, provided they are initialized with identical priors. 2) The optimal filter admits an innovation error-based gain feedback structure. 3) The optimal feedback gain is obtained via a solution of an Euler-Lagrange boundary value problem; the feedback gain equals the Kalman gain in the linear Gaussian case. Numerical algorithms are introduced and implemented in two general examples, and a neuroscience application involving coupled oscillators. In some cases it is found that the filter exhibits significantly lower variance when compared to the bootstrap particle filter.

Posted in: Bayesian filtering , Tagged:

A gentle introduction to Box-Particle Filters

June 7, 2017 12:12 ,

A. Gning, B. Ristic, L. Mihaylova and F. Abdallah, An Introduction to Box Particle Filtering [Lecture Notes], in IEEE Signal Processing Magazine, vol. 30, no. 4, pp. 166-171, July 2013. DOI: 10.1109/MSP.2013.225460.

Resulting from the synergy between the sequential Monte Carlo (SMC) method [1] and interval analysis [2], box particle filtering is an approach that has recently emerged [3] and is aimed at solving a general class of nonlinear filtering problems. This approach is particularly appealing in practical situations involving imprecise stochastic measurements that result in very broad posterior densities. It relies on the concept of a box particle that occupies a small and controllable rectangular region having a nonzero volume in the state space. Key advantages of the box particle filter (box-PF) against the standard particle filter (PF) are its reduced computational complexity and its suitability for distributed filtering. Indeed, in some applications where the sampling importance resampling (SIR) PF may require thousands of particles to achieve accurate and reliable performance, the box-PF can reach the same level of accuracy with just a few dozen box particles. Recent developments [4] also show that a box-PF can be interpreted as a Bayes? filter approximation allowing the application of box-PF to challenging target tracking problems [5].

Posted in: Bayesian filtering , Tagged: ,

Using multiple RANSACs for tracking

February 24, 2016 19:13 ,

Peter C. Niedfeldt and Randal W. Beard, Convergence and Complexity Analysis of Recursive-RANSAC: A New Multiple Target Tracking Algorithm, in IEEE Transactions on Automatic Control , vol.61, no.2, pp.456-461, Feb. 2016, DOI: 10.1109/TAC.2015.2437518.

The random sample consensus (RANSAC) algorithm was developed as a regression algorithm that robustly estimates the parameters of a single signal in clutter by forming many simple hypotheses and computing how many measurements support that hypothesis. In essence, RANSAC estimates the data association problem of a single target in clutter by identifying the hypothesis with the most supporting measurements. The newly developed recursive-RANSAC (R-RANSAC) algorithm extends the traditional RANSAC algorithm to track multiple targets recursively by storing a set of hypotheses between time steps. In this technical note we show that R-RANSAC converges to the minimum mean-squared solution for well-spaced targets. We also show that the worst-case computational complexity of R-RANSAC is quadratic in the number of new measurements and stored models.

Posted in: Mathematics , Tagged: ,