Using data from extensive vibrational tests of the new aircraft Saab 2000 three different methods for vibration analysis are studied. These methods are ERA (eigensystem realization algorithm), N4SID (a subspace method) and PEM (prediction error approach). We find that both the ERA and N4SID methods give good initial model parameter estimates that can be further improved by the use of PEM. We also find that all methods give good insights into the vibrational modes.
In this paper we discuss smooth and sensitive norms for prediction error system identification when the disturbances are magnitude bounded. Formal conditions for sensitive norms, which give an order of magnitude faster convergence of the parameter estimate variance, are developed. However, it also is shown that the parameter estimate variance convergence rate of sensitive norms is arbitrarily bad for certain distributions. A necessary condition for a norm to be statistically robust with respect to the family F(C) of distributions with support [-C, C] for some arbitrary C>0 is that its second derivative does not vanish on the support. A direct consequence of this observation is that the quadratic norm is statistically robust among all lp-norms, p⩽2<∞ for F(C).
Auditory attention identification methods attempt to identify the sound source of a listeners interest by analyzing measurements of electrophysiological data. We present a tutorial on the numerous techniques that have been developed in recent decades, and we present an overview of current trends in multivariate correlation-based and model-based learning frameworks. The focus is on the use of linear relations between electrophysiological and audio data. The way in which these relations are computed differs. For example, canonical correlation analysis (CCA) finds a linear subset of electrophysiological data that best correlates to audio data and a similar subset of audio data that best correlates to electrophysiological data. Model-based (encoding and decoding) approaches focus on either of these two sets. We investigate the similarities and differences between these linear model philosophies. We focus on (1) correlation-based approaches (CCA), (2) encoding/decoding models based on dense estimation, and (3) (adaptive) encoding/decoding models based on sparse estimation. The specific focus is on sparsity-driven adaptive encoding models and comparing the methodology in state-of-the-art models found in the auditory literature. Furthermore, we outline the main signal processing pipeline for how to identify the attended sound source in a cocktail party environment from the raw electrophysiological data with all the necessary steps, complemented with the necessary MATLAB code and the relevant references for each step. Our main aim is to compare the methodology of the available methods, and provide numerical illustrations to some of them to get a feeling for their potential. A thorough performance comparison is outside the scope of this tutorial.
In this contribution aspects of inter-sample input signal behavior are examined. The starting point is that parametric identification always is performed on basis of discrete-time data. This is valid for identification of discrete-time models as well as continuous-time models. The usual assumptions on the input signal are; i) it is band-limited, ii) it is piecewise constant or iii) it is piecewise linear. One point made in this paper is that if a discrete-time model is used, the best possible (in the model structure) adjustment to data is made. This is independent of the assumption on the input signal. However, a transformation of the obtained discrete model to a continuous one is not possible without additional assumptions on the input signal. The other point made is that the frequency functions of the discrete models very well coincides with the frequency functions of the discretized continuous time models and the continuous time transfer function fitted in the frequency domain.
In this contribution aspects of inter-sample input signal behavior are examined. The starting point is that parametric identication always is performed on basis of discrete-time data. This is valid for identication of discrete-time models as well as continuous-time models. The usual assumptions on the input signal are; i) it is band-limited, ii) it is piecewise constant or iii) it is piecewise linear. One point made in this paper is that if a discrete-time model is used, the best possible (in the model structure) adjustment to data is made. This is independent of the assumption on the input signal. However, a transformation of the obtained discrete model to a continuous one is not possible without additional assumptions on the input signal. The other point made is that the frequency functions of the discrete models very well coincides with the frequency functions of the discretized continuous time models and the continuous time transfer function fitted in the frequency domain.
State-space smoothing has found many applications in science and engineering. Under linear and Gaussian assumptions, smoothed estimates can be obtained using efficient recursions, for example Rauch Tung Striebel and Mayne Fraser algorithms. Such schemes are equivalent to linear algebraic techniques that minimize a convex quadratic objective function with structure induced by the dynamic model. These classical formulations fall short in many important circumstances. For instance, smoothers obtained using quadratic penalties can fail when outliers are present in the data, and cannot track impulsive inputs and abrupt state changes. Motivated by these shortcomings, generalized Kalman smoothing formulations have been proposed in the last few years, replacing quadratic models with more suitable, often nonsmooth, convex functions. In contrast to classical models, these general estimators require use of iterated algorithms, and these have received increased attention from control, signal processing, machine learning, and optimization communities. In this survey we show that the optimization viewpoint provides the control and signal processing community great freedom in the development of novel modeling and inference frameworks for dynamical systems. We discuss general statistical models for dynamic systems, making full use of nonsmooth convex penalties and constraints, and providing links to important models in signal processing and machine learning. We also survey optimization techniques for these formulations, paying close attention to dynamic problem structure. Modeling concepts and algorithms are illustrated with numerical examples. (C) 2017 Elsevier Ltd. All rights reserved.
In this paper the effect of some weighting matrices on the asymptotic variance of the estimates of linear discrete time state space systems estimated using subspace methods is investigated. The analysis deals with systems with white or without observed inputs and refers to the Larimore type of subspace procedures. The main result expresses the asymptotic variance of the system matrix estimates in canonical form as a function of some of the user choices, clarifying the question on how to choose them optimally. It is shown, that the CCA weighting scheme leads to optimal accuracy. The expressions for the asymptotic variance can be implemented more efficiently as compared to the ones previously published.
This paper addresses the problem of identification of hybrid dynamical systems, by focusing the attention on hinging hyperplanes (HHARX) and wiener piecewise affine (W-PWARX) autoregressive exogenous models. In particular, we provide algorithms based on mixed-integer linear or quadratic programming which are guaranteed to converge to a global optimum. We also discuss issues of state-space realization of HHARX and W-PWARX models into several existing discrete-time hybrid state-space forms.
The nonlinear estimation problem in navigation using terrain height variations is studied. The optimal Bayesian solution to the problem is derived. The implementation is grid based, calculating the probability of a set of points on an adaptively dense mesh. The Cramer-Rao bound is derived. Monte Carlo simulations over a commercial map shows that the algorithm, after convergence, reaches the Cramer-Rao lower bound.
The performance of terrain-aided navigation of aircraft depends on the size of the terrain gradient in the area. The point-mass filter (PMF) described in this work yields an approximate Bayesian solution that is well suited for the unstructured nonlinear estimation problem in terrain navigation. It recursively propagates a density function of the aircraft position. The shape of the point-mass density reflects the estimate quality; this information is crucial in navigation applications, where estimates from different sources often are fused in a central filter. Monte Carlo simulations show that the approximation can reach the optimal performance, and realistic simulations show that the navigation performance is very high compared with other algorithms and that the point-mass filter solves the recursive estimation problem for all the types of terrain covered in the test. The main advantages of the PMF is that it works for many kinds of nonlinearities and many kinds of noise and prior distributions. The mesh support and resolution are automatically adjusted and controlled using a few intuitive design parameters. The main disadvantage is that it cannot solve estimation problems of very high dimension since the computational complexity of the algorithm increases drastically with the dimension of the state space. The implementation used in this work shows real-time performance for 2D and in some cases 3D models, but higher state dimensions are usually intractable.
In aircraft navigation the demands on reliability and safety are very high. The importance of accurate position and velocity information becomes crucial when flying an aircraft at low altitudes, and especially during the landing phase. Not only should the navigation system have a consistent description of the position of the aircraft, but also a description of the surrounding terrain, buildings and other objects that are close to the aircraft. Terrain navigation is a navigation scheme that utilizes variations in the terrain height along the aircraft flight path. Integrated with an Inertial Navigation System (INS), it yields high performance position estimates in an autonomous manner, ie without any support information sent to the aircraft. In order to obtain these position estimates, a nonlinear recursive estimation problem must be solved on-line. Traditionally, this filtering problem has been solved by local linearization of the terrain at one or several assumed aircraft positions. Due to changing terrain characteristics, these linearizations will in some cases result in diverging position estimates. In this work, we show how the Bayesian approach gives a comprehensive framework for solving the recursive estimation problem in terrain navigation. Instead of approximating the model of the estimation problem, the analytical solution is approximately implemented. The proposed navigation filter computes a probability mass distribution of the aircraft position and updates this description recursively with each new measurement. The navigation filter is evaluated over a commercial terrain database, yielding accurate position estimates over several types of terrain characteristics. Moreover, in a Monte Carlo analysis, it shows optimal performance as it reaches the Cramér-Rao lower bound.
This paper reviews and evaluates suggested methods for estimating the time-delay of linear systems in automatic control applications. A classification of the methods according to the underlying principles is suggested. The evaluation, done by analyzing the estimates of the methods from extensive simulated data in open loop, shows that different classes of methods have different properties and are suitable in different cases. Some method are clearly inferior to others. Recommendations are given on how to choose estimation method and input signal.
A promising method for estimation of the time-delay in continuous-time linear dynamical systems uses the phase of the all-pass part of a discrete-time model of the system. We have discovered that this method can sometimes fail totally and we suggest a method for avoiding such failures.
In this paper the asymptotic normality of a large class of prediction error estimators is established. (Prediction error identification methods were introduced in [1] and further developed in [2] and [3].) The observed processes in this paper are assumed to be stationary and ergodic and the parameterized process models are taken to be non-linear regression models. In the gaussian case the results presented in this paper constitute substantial generalizations of previous results concerning the asymptotic normality of maximum likelihood estimators for (i) processes of independent random variables [9,4] and (ii) Markov processes [5]; these results also generalize previous results on the asymptotic normality of least squares estimators for autoregressive moving average processes [6,7]. The asymptotic normality theorem gives formulae for the covariances of the asymptotic distributions of the parameter estimation errors arising from the specified class of prediction error identification methods. Employing these formulae it is demonstrated that the prediction error method using the determinant of the residual error covariance matrix as loss function is asymptotically efficient with respect to the specified class of prediction error estimators regardless of the distribution of the observed processes.
The present contribution deals with the accuracy of the transfer function of state-space parametric models estimated under the prediction error identification framework. More precisely, we intend to propagate the Cramer-Rao bound usually available on the covariance matrix of the state-space parameter estimates to that of the coefficients of the corresponding input-to-output transfer function. A natural way to solve this problem is to take advantage of the Jacobian matrix of the state-space to transfer function transformation while applying Gauss' formula for evaluating the covariance of the transfer function coefficients. Here, we focus on the computational aspects of the evaluation of this Jacobian matrix. In doing so, we show that the most computationally efficient way to access this matrix is to evaluate it as the product of the Jacobian matrices associated to the two following transformations: firstly, from the original state-space model to a state-space representation where the state-feedback matrix is diagonal and, secondly, from this latter state-space representation to the model transfer function. Note that the elements of these two Jacobian matrices are evaluated analytically.
This paper deals with the issue of estimating the parameters in a continuous-time nonlinear dynamical model from sampled data. We focus on the issue of bias-variance trade-offs. In particular, we show that the bias error can be significantly reduced by using a particular form of sampled data model based on truncated Taylor series. This model retains the conceptual simplicity of models based on Euler integration but has much improved accuracy as a function of the sampled period.
Model estimation and structure detection with short data records are two issues that receive increasing interests in System Identification. In this paper, a multiple kernel-based regularization method is proposed to handle those issues. Multiple kernels are conic combinations of fixed kernels suitable for impulse response estimation, and equip the kernel-based regularization method with three features. First, multiple kernels can better capture complicated dynamics than single kernels. Second, the estimation of their weights by maximizing the marginal likelihood favors sparse optimal weights, which enables this method to tackle various structure detection problems, e. g., the sparse dynamic network identification and the segmentation of linear systems. Third, the marginal likelihood maximization problem is a difference of convex programming problem. It is thus possible to find a locally optimal solution efficiently by using a majorization minimization algorithm and an interior point method where the cost of a single interior-point iteration grows linearly in the number of fixed kernels. Monte Carlo simulations show that the locally optimal solutions lead to good performance for randomly generated starting points.
Regularization methods with regularization matrix in quadratic form have received increasing attention. For those methods, the design and tuning of the regularization matrix are two key issues that are closely related. For systems with complicated dynamics, it would be preferable that the designed regularization matrix can bring the hyper-parameter estimation problem certain structure such that a locally optimal solution can be found efficiently. An example of this idea is to use the so-called multiple kernel Chen et al. (2014) for kernel-based regularization methods. In this paper, we propose to use the multiple regularization matrix for the filter-based regularization. Interestingly, the marginal likelihood maximization with the multiple regularization matrix is also a difference of convex programming problem, and a locally optimal solution could be found with sequential convex optimization techniques. (C) 2018, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved.
The first order stable spline (SS-1) kernel (also known as the tunedcorrelated kernel) is used extensively in regularized system identification, where the impulse response is modeled as a zero-mean Gaussian process whose covariance function is given by well designed and tuned kernels. In this paper, we discuss the maximum entropy properties of this kernel. In particular, we formulate the exact maximum entropy problem solved by the SS-1 kernel without Gaussian and uniform sampling assumptions. Under general sampling assumption, we also derive the special structure of the SS-1 kernel (e.g. its tridiagonal inverse and factorization have closed form expression), also giving to it a maximum entropy covariance completion interpretation.
There has been recently a trend to study linear system identification with high order finite impulse response (FIR) models using the regularized least-squares approach. One key of this approach is to solve the hyper-parameter estimation problem that is usually nonconvex. Our goal here is to investigate implementation of algorithms for solving the hyper-parameter estimation problem that can deal with both large data sets and possibly ill-conditioned computations. In particular, a QR factorization based matrix-inversion-free algorithm is proposed to evaluate the cost function in an efficient and accurate way. It is also shown that the gradient and Hessian of the cost function can be computed based on the same QR factorization. Finally, the proposed algorithm and ideas are verified by Monte-Carlo simulations on a large data-bank of test systems and data sets.
This contribution aims to enrich the recently introduced kernel-based regularization method for linear system identification. Instead of a single kernel, we use multiple kernels, which can be instances of any existing kernels for the impulse response estimation of linear systems. We also introduce a new class of kernels constructed based on output error (OE) model estimates. In this way, a more flexible and richer representation of the kernel is obtained. Due to this representation the associated hyper-parameter estimation problem has two good features. First, it is a difference of convex functions programming (DCP) problem. While it is still nonconvex, it can be transformed into a sequence of convex optimization problems with majorization minimization (MM) algorithms and a local minima can thus be found iteratively. Second, it leads to sparse hyper-parameters and thus sparse multiple kernels. This feature shows the kernel-based regularization method with multiple kernels has the potential to tackle various problems of finding sparse solutions in linear system identification.
FIR (finite impulse response) model is widely used in tackling the problem of the impulse response estimation with quantized measurements. Its use is, however, limited, in the case when a high order FIR model is required to capture a slowly decaying impulse response. This is because the high variance for high order FIR models would override the low bias and thus lead to large MSE (mean square error). In this contribution, we apply the recently introduced regularized FIR model approach to the problem of the impulse response estimation with binary measurements. We show by Monte Carlo simulations that the proposed approach can yield both better accuracy and better robustness than a recently introduced FIR model based approach.
In this companion paper, the choice of kernels for estimating the impulse response of linear stable systems is considered from a classical, “frequentist”, point of view. The kernel determines the regularization matrix in a regularized least squares estimate of an FIR model. The quality is assessed from a mean square error (MSE) perspective, and measures and algorithms for optimizing the MSE are discussed. The ideas are tested on the same data bank as used in Part I of the companion papers. The resulting findings and conclusions in the two papers are very similar despite the different perspectives.
Intrigued by some recent results on impulse response estimation by kernel and nonparametric techniques, we revisit the old problem of transfer function estimation from input-output measurements.We formulate a classical regularization approach, focused on ﬁnite impulse response (FIR) models, and ﬁnd that regularization is necessary to cope with the high variance problem. This basic, regularized least squares approach is then a focal point for interpreting other techniques, like Bayesian inference and Gaussian process regression.
Intrigued by some recent results on impulse response estimation by kernel and nonparametric techniques, we revisit the old problem of transfer function estimation from input-output measurements. We formulate a classical regularization approach, focused on finite impulse response (FIR) models, and find that regularization is necessary to cope with the high variance problem. This basic, regularized least squares approach is then a focal point for interpreting other techniques, like Bayesian inference and Gaussian process regression. The main issue is how to determine a suitable regularization matrix (Bayesian prior or kernel). Several regularization matrices are provided and numerically evaluated on a data bank of test systems and data sets. Our findings based on the data bank are as follows. The classical regularization approach with carefully chosen regularization matrices shows slightly better accuracy and clearly better robustness in estimating the impulse response than the standard approach - the prediction error method/maximum likelihood (PEM/ML) approach. If the goal is to estimate a model of given order as well as possible, a low order model is often better estimated by the PEM/ML approach, and a higher order model is often better estimated by model reduction on a high order regularized FIR model estimated with careful regularization. Moreover, an optimal regularization matrix that minimizes the mean square error matrix is derived and studied. The importance of this result lies in that it gives the theoretical upper bound on the accuracy that can be achieved for this classical regularization approach.
The stable spline kernel and the diagonal correlated kernel are two kernels that have been tested extensively in kernel-based regularization methods for LTI system identification. As shown in our recent works, although these two kernels are introduced in different ways, they share some common features, e.g., they all belong to the class of exponentially convex locally stationary kernels, and state-space model induced kernels. In this work, we further show that similar to the derivation of the stable spline kernel, the continuous-time diagonal correlated kernel can be derived by applying the same "stable" coordinate change to a "generalized" first order spline kernel, and thus can be interpreted as a stable generalized first order spline kernel. This interpretation provides new facets to understand the properties of the diagonal correlated kernel. Due to this interpretation, new eigendecompositions, explicit expression of the norm, and new maximum entropy interpretation of the diagonal correlated kernel are derived accordingly.
System identification with regularization methods has attracted increasing attention recently and is a complement to the current standard maximum likelihood/ prediction error method. In this paper, we focus on the kernel-based regularization method and give a spectral analysis of the so-called diagonal correlated (DC) kernel, one family of kernel structures that has been proven useful for linear time-invariant system identification. In particular, using the theory of Bessel functions, we derive the eigenvalues and corresponding eigenfunctions of the DC kernel. Accordingly, we derive the Karhunen-Loeve expansion of the stochastic process whose covariance function is the DC kernel.
In this paper, a new particle filter (PF) which we refer to as the decentralized PF (DPF) is proposed. By first decomposing the state into two parts, the DPF splits the filtering problem into two nested sub-problems and then handles the two nested sub-problems using PFs. The DPF has an advantage over the regular PF that the DPF can increase the level of parallelism of the PF. In particular, part of the resampling in the DPF bears a parallel structure and thus can be implemented in parallel. The parallel structure of the DPF is created by decomposing the state space, differing from the parallel structure of the distributed PFs which is created by dividing the sample space. This difference results in a couple of unique features of the DPF in contrast with the existing distributed PFs. Simulation results from a numerical example indicates that the DPF has a potential to achieve the same level of performance as the regular PF, in a shorter execution time.
In this paper, a new particle filter (PF) which we refer to as the decentralized PF (DPF) is proposed. By first decomposing the state into two parts, the DPF splits the filtering problem into two nested subproblems and then handles the two nested subproblems using PFs. The DPF has the advantage over the regular PF that the DPF can increase the level of parallelism of the PF. In particular, part of the resampling in the DPF bears a parallel structure and can thus be implemented in parallel. The parallel structure of the DPF is created by decomposing the state space, differing from the parallel structure of the distributed PFs which is created by dividing the sample space. This difference results in a couple of unique features of the DPF in contrast with the existing distributed PFs. Simulation results of two examples indicate that the DPF has a potential to achieve in a shorter execution time the same level of performance as the regular PF.
The estimation of Linear Time Invariant (LTI) models is a standard procedure in system identification. Any real-life system will however be nonlinear and time-varying, and the estimated model will converge to the LTI second order equivalent (LTI-SOE) of the true system. In this paper we consider some aspects of this convergence and the distance between the true system and its LTI-SOE. We show that there may be cases where even the slightest nonlinearity may cause big differences in the LTI-SOE. We also show a result that gives conditions that guarantee that the LTI-SOE is close to "the natural" LTI approximant. Finally, an upper bound on the distance between the LTI-SOE of a nonlinear FIR system with a white input signal and the linear part of the system is derived.
Nonlinear systems can be approximated by linear time-invariant (LTI) models in-many ways. Here, LTI models that are optimal approximations in the mean-square error sense are analyzed. A necessary and sufficient condition on the input signal for the optimal LTI approximation of an arbitrary nonlinear finite impulse response (NFIR) system to be a linear finite impulse response (FIR) model is presented. This condition says that the in ut should be separable of a certain order, i.e., that certain conditional expectations should be,P linear. For the special case of Gaussian input signals, this condition is closely related to a generalized version of Bussgang's classic theorem about static nonlinearities. It is shown that this generalized theorem can be used for structure identification and for the identification of generalized Wiener-Hammerstein systems.
Approximations of slightly nonlinear systems with linear time-invariant (LTI) models are often used in applications. Here, LTI models that are optimal approximations in the mean-square error sense are studied. It is shown that these models can be very sensitive to small nonlinearities. Furthermore, the significance of the distribution of the input process is discussed. From the examples studied here, it seems that LTI approximations for inputs with distributions that are Gaussian or almost Gaussian are less sensitive to small nonlinearities.