Department of Information Technology

# Identification of dynamic errors-in-variables models

Identification of linear dynamic systems from noise-corrupted output measurements is a fundamental research problem which has been investigated in the past decades. On the other hand, the estimation of the parameters for linear dynamic systems when also the input is affected by noise is recognized as a more difficult problem which only recently has received increasing attention.

Representations where errors or measurement noises are present on both the inputs and outputs are usually called errors-in-variables models and play an important role when the identification purpose is the determination of the inner laws that describe the process, rather than the prediction of its future behaviour. The class of scientific disciplines which make use of such a kind of representations is very broad, such as time series modeling, array signal processing for direction-of-arrival estimation, blind channel equalization, multivariate calibration in analytical chemistry, image processing, astronomical data reduction, etc.

As a typical model example, consider the following system with noise-corrupted input and output measurements. The noise-free input is denoted by uo(t) and the undisturbed output by yo(t). They are linked through the linear difference equation

A(q^-1) y_o(t) = B(q^-1) u_o(t) , (1)

where A(q-1) and B(q-1) are polynomials of the type

A(q^-1) = 1 + a1 q^-1 + ... + an q^-n

B(q^-1) = b0 + b1 q^-1 + ... + bn q^-n (2)

and q^-1 is the backward shift operator, i.e. q^-1 x(t) = x(t-1). It is not restrictive to assume that the polynomials A(q^-1), B(q^-1) have equal degree n, which represents the order of the system.

In an errors-in-variables environment we assume that the observations are corrupted by additive measurement noises eu(t) and ey(t), at the input and output respectively. Therefore, the available signals are of the form

u(t) = u_o(t) + eu(t)

y(t) = y_o(t) + ey(t). (3)

The problem is to determine the system characteristics, i.e. the transfer function

G(q^-1) = B(q^-1)/A(q^-1) (4)

of the system given the available measurements u(t) and y(t) for t=1,...,N.

The problem has been examined from different point of views. It is possible to device the statistically efficient maximum likelihood estimate [3]. This method is computationally expensive. But the computationally simple methods [4] are often inaccurate. There are subspace fitting approaches [8], [1], which are more accurate in cost of higher computational load. Significant improvement in the accuracy with low computational load is reported in [2]. There are frequency domain approaches using a two dimensional time series model [5]. Some overview and further perspectives can be found in [21], [7], [6], [10]. The case when the unperturbed input is periodic is treated in [11]. The local convergence properties of bias-eliminating least squares (BELS) algorithms for EIV identification were investigated in [12], [13]. For the Frisch and BELS algorithms, the estimation accuracy was analyzed in [14], [22], [15] and [24]. The Frisch scheme was extended to correlated ouput noise cases in [23] and [27], and a simplified form of the BELS method was proposed in [25]. A new method, called the extended compensated least square (ECLS) method, was proposed and further analyzed in [16] and [17]. In [18], some approaches for continuous-time modeling in EIV identification were discussed. A licentiate thesis comprising some of the above results appeared in 2005, [19].

Overviews of many identifiability and estimation aspects was given in [20], [21], [37] and [44].

Recently, in [26], the relations between BELS, Frisch scheme and ECLS methods were considered. More recent studies, [39], [40], [42] show how to imbed all these methods into a joint framework (called the generalized instrumental variable estimator, GIVE), where certain choices of algorithmic parameters can result in the different special cases earlier known as specific methods. Accuracy comparisons for three Frisch methods were made in [28]. A more general study of some bias-compensated least squares problems appeared in [51].

The statistical analysis of a third-order cumulants based algorithm was investigated in [30].

An application of EIV techniques for data driven controller design appears in [42].

The sample maximum likelihood method (SML) is based on periodic excitation, runs in the frequency domain, and allows for an arbitrary, nonparametric noise model. Another ML method is mostly defined in the time-domain, and is based on parametric models both of the measurement noise, and the statistics of the unperturbed input. These two approaches are compared in [29],[32], and it is shown that under some conditions, they can give similar, but not identical, behavior. A comparison of the accuracy of the ML estimates for EIV problems compared to prediction error estimates is investigated in [43].

A covariance matching approach has been proposed and analysed from different viewpoints in [31], [33], [34], [35], [36], [38], [41], [48], [49], [52], [53]. Both discrete-time and continuous-time models are included, and the the case of discrete-time models is so-far analysed in quite some detail. The underlying idea is to first condense the information in the recorded, noisy input-output measurements in a small set of covariances (with some different lag values). Next a parameteric model is fitted to these covariances using a (weighted) nonlinear least squares approach. The method is quite promising and gives much higher accuracy than GIVE, and require a modestly higher computational complexity. It is also shown that the approach is closely related to structural equation modeling (SEM) in multivariate statistics.

The contribution [50] presents a discussion under what conditions errors-in-variables problems can be successivefully handled for systems operating under feedback control.

## Partners:

Mei Hong Bjerstedt, Volvo Construction Equipment, Eskilstuna, Sweden
Alireza Karimi, EPFL, Lausanne, Switzerland;
David Kreiberg, Uppsala University, Sweden;
Kaushik Mahata, University of Newcastle, Australia;
Rik Pintelon, Free University of Brussels, Belgium;
Johan Schoukens, Free University of Brussels, Belgium;
Umberto Soverini, University of Bologna, Italy;
Stephane Thil, Nancy-University, France;
Klaske Van Heusden, EPFL, Lausanne, Switzerland;
Fan Wallentin, Uppsala University, Sweden;
Liuping Wang, RMIT University, Melbourne, Australia
Wei Xing Zheng, University of Western Sydney, Australia
Juan Yuz, Universidad Santa Maria, Valparaiso, Chile
Wei Xing Zheng, University of Western Sydney, Penrith South, Australia.

## Previous Researchers:

Mats Cedervall, Anders Eriksson, Joakim Sorelius, Petre Stoica.

Initially, the project was supported by the Swedish Research Council for Engineering Sciences as a part of a multi-project on Statistical Signal Processing. Currently it is supported by a grant from the Swedish Research Council.

## References

[1] M Cedervall and P Stoica. System identification from noisy measurements by using instrumental variables and subspace fitting. Circuits, Systems and Signal Processing, 15:2:275-290, 1996.

[2] K. Mahata and T. Söderström. Identification of dynamic errors-in-variables model using prefiltered data. Proc. 15th IFAC World Congress, July 21-26, 2002, Barcelona, Spain.

[3] T Söderström. Identification of stochastic linear systems in presence of input noise. Automatica, 17:713-725, 1981.

[4] T. Söderström and K. Mahata. On instrumental variable and total least squares approaches for identification of noisy systems. International Journal of Control, vol 75, no 6, pp 381-389, April 2002.

[5] T. Söderström, K. Mahata, and U. Soverini. Identification of dynamic errors-in-variables model using a frequency domain Frisch scheme. Proc. 15th IFAC World Congress, July 21-26, 2002, Barcelona, Spain.

[6] T. Söderström, U.Soverini and K. Mahata. Perspectives on errors-in-variables estimation for dynamic systems. Signal Processing, vol 82, no 8, 1139-1154, August 2002.

[7] U. Soverini and T. Söderström. Identification methods of dynamic systems in presence of input noise. In Proc. SYSID 2000, IFAC 12th Symposium on System Identification, Santa Barbara, California, June 21-23 2000.

[8] P Stoica, M Cedervall, and A Eriksson. Combined instrumental variable and subspace fitting approach to parameter estimation of noisy input-output systems. IEEE Transactions on Signal Processing, 43:2386-2397, 1995.

[9] T. Söderström, K. Mahata, and U. Soverini. Identification of dynamic errors-in-varaibles model: Approaches based on two-dimensional ARMA modelling of the data. Automatica, vol 39, no 5, 929-935, May 2003.

[10] T. Söderström. Why are errors-in-varibles problems often tricky? European Control Conference, ECC 2003, September 01-04, 2003, Cambridge, UK.

[11] T. Söderström and M. Hong. Identification of dynamic errors-in-variables systems with periodic data. Proc. of 16th IFAC World Congress, Prague, Czech Republic, July 4-8, 2005.

[12] T. Söderström, M. Hong and W. X. Zheng. Convergence properties of bias-eliminating algorithms for errors-in-variables identification. International Journal of Adaptive Control and Signal Processing, vol 19, pp 703-722, November 2005.

[13] T. Söderström, M. Hong and W. X. Zheng. Convergence of bias-eliminating least squares methods for identification of errors-in-variables dynamic systems. Proc. 44th IEEE CDC/ECC, Seville, Spain, December 12-15, 2005.

[14] T. Söderström. Accuracy analysis of the Frisch estimates for identifying errors-in-variables systems. 44th IEEE CDC/ECC 2005, Seville, Spain, December 12-15, 2005.

[15] M. Hong, T. Söderström and W. X. Zheng. Accuracy analysis of bias-eliminating least squares estimates for errors-in-variables identification. 14th IFAC Symposium on System Identification, Newcastle, Australia, March, 29-31, 2006.

[16] M. Ekman. A separable nonlinear least-squares approach for identification of linear systems with errors in variables. Proc. of 16th IFAC World Congress, Prague, Czech Republic, July 4-8, 2005.

[17] M. Ekman, M. Hong and T. Söderström: A separable nonlinear least-squares approach for identification of linear systems with errors in variables. 14th IFAC Symposium on System Identification, Newcastle, Australia, March, 29-31, 2006.

[18] T. Söderström, E. K. Larsson, K. Mahata and M. Mossberg, Using continuous-time modeling for errors-in-variables identification. 14th IFAC Symposium on System Identification, Newcastle, Australia, March, 29-31, 2006.

[19] M. Hong, On Two Methods for Identifying Dynamic Errors-in-Variables Systems. Department of Information Technology, Licentiate thesis 2005-007, November 2005.

[20] T. Söderström, Errors-in-variables methods in system identification. 14th IFAC Symposium on System Identification, Newcastle, Australia, March, 29-31, 2006. Invited plenary paper.

[21] T. Söderström: Errors-in-variables methods in system identification. Automatica, survey paper, vol 43, no 6, pp 939-958, June 2007.

[22] T. Söderström: Accuracy analysis of the Frisch estimates for identifying errors-in-variables systems. IEEE Transactions on Automatic Control, vol 52, no 6, pp 985-997, June 2007.

[23] T. Söderström: A Frisch scheme for correlated output noise errors-in-variables identification. European Control Conference, ECC'07, Kos, Greece, July 2-5, 2007.

[24] M. Hong, T. Söderström and W. X. Zheng: Accuracy analysis of bias-eliminating least squares estimates for errors-in-variables systems. Automatica, vol 43, no 9, pp 1590-1596, September 2007.

[25] M. Hong, T. Söderström and W. X. Zheng: A simplified form of the bias-eliminating least squares method for errors-in-variables identification. IEEE Transactions on Automatic Control, vol 52, no 9, pp 1754-1756, September 2007.

[26] M. Hong, T. Söderström: Relations between bias-eliminating least squares, the Frisch scheme and extended compensated least squares methods for identifying errors-in-variables systems. Automatica, vol 45, no 1, pp 277-288, January 2009.

[27] T. Söderström: Extending the Frisch scheme for errors-in-variables identification to correlated output noise. International Journal of Adaptive Control and Signal Processing, vol 22, no 1, pp 55-73, February 2008.

[28] M. Hong, T. Söderström, U. Soverini and R. Diversi: Comparison of three Frisch methods for errors-in-variables identification. Proc. of the 17th IFAC World Congress, Seoul, Korea, July 6-11, 2008.

[29] M. Hong, T. Söderström, J. Schoukens and Rik Pintelon: Accuracy analysis of time domain likelihood method and sample maximum likelihood method in Errors-in-variables Identification. Proc. of the 17th IFAC World Congress, Seoul, Korea, July 6-11, 2008.

[30] S. Thil, M. Hong, T. Söderström, M. Gilson and H. Garnier: Statistical analysis of a third-order cumulants based algorithm for discrete errors-in-variables identification. Proc. of the 17th IFAC World Congress, Seoul, Korea, July 6-11, 2008.

[31] T. Söderström, M. Mossberg and M. Hong: A covariance matching approach for identifying errors-in-variables systems. Automatica, vol 45, no 9, pp 2018--2031, September 2009.

[32] T. Söderström, M. Hong, J. Schoukens and R. Pintelon: Accuracy Analysis of Time-domain Maximum Likelihood Method and Sample Maximum Likelihood Method for Errors-in-Variables and Output Error Identification. Automatica, vol 46, no 4, pp 721-727, April 2010.

[33] T. Söderström and M. Mossberg: Accuracy analysis of a covariance matching approach for identifying errors-in-variables systems. Automatica, vol 47, no 1, pp 272-282, February 2011.

[34] M. Mossberg and T. Söderström: Approximative weighting for covariance matching approach for identifying errors-in-variables systems. International Journal of Adaptive Control and Signal Processing, vol 25, no 6, pp 535-543, June 2011.

[35] M. Mossberg and T. Söderström: Continuous-time errors-in-variables system identification through covariance matching without input signal modeling. American Control Conference, St. Louis, Missouri, USA, June 10-12, 2009.

[36] T. Söderström, M. Mossberg and M. Hong: Identifying errors-in-variables systems by using a covariance matching approach. SYSID 2009, IFAC 15th Symposium on System Identification, Saint-Malo, France, July 6-8, 2009.

[37] M. Hong, T. Söderström, J. Schoukens and R. Pintelon: Accuracy analysis of time domain maximum likelihood method and sample maximum likelihood method in errors-in-variables identification. IFAC 17th World Congress, Seoul, Korea, July 06-11, 2008.

[38] T. Söderström and M. Mossberg: Asymptotic accuracy of covariance function based identifying errors-in-variables system parameter estimates. 49th IEEE Conference on Decision and Control, December 15-17, 2010, Atlanta, GA, USA.

[39] T. Söderström: Errors-in-variables identification using a generalized instrumental variable estimation method. 49th IEEE Conference on Decision and Control, December 15-17, 2010, Atlanta, GA, USA.

[40] T. Söderström: A generalized instrumental variable estimation method for errors-in-variables identification problems. Automatica, vol 47, no 8, pp 1656-166, August 2011.

[41] M. Mossberg and T. Söderström: Covariance matching for continuous-time errors-in-variables problems. IEEE Transactions on Automatic Control, vol 56, no 6, pp 1478-1483, June 2011.

[42] K. van Heusden, A. Karimi and T. Söderström: On identification methods for direct data-driven controller . International Journal of Adaptive Control and Signal Processing. vol 25, no 5, pp 448-465, May 2011.

[43] H. Hjalmarsson, J. Mårtensson, C. Rojas and T. Söderström: On the accuracy in errors-in-variables identification compared to predition-error identification. Automatica, vol 47, no 12, pp 2704-2712, December 2011.

[44] T. Söderström: System identification for the errors-in-variables problem. Transactions of the Institute of Measurement and Control, vol 34, no 7, pp 780-792, October 2012.

[45] T. Söderström: A generalized instrumental variable method for multivariable errors-in-variables identification problems. International Journal of Control, vol 85, no 3, pp 287-303. March 2012.

[46] T. Söderström and L. Wang: On model order determination for errors-in-variables estimation. 16th IFAC Symposium on System Identification, Brussels, Belgium, July 11-13 2012.

[47] T. Söderström: Model order determination based on rank properties of almost singular covariance matrices. 16th IFAC Symposium on System Identification, Brussels, Belgium, July 11-13 2012.

[48] M. Mossberg and T. Söderström: On covariance matching for multiple input multiple output errors-in-variables systems. 16th IFAC Symposium on System Identification, Brussels, Belgium, July 11-13 2012.

[49] T. Söderström, Y. Irshad, M. Mossberg and W. X. Zheng: Accuracy Analysis of a Covariance Matching Method for Continuous-time Errors-in-variables System Identification. 16th IFAC Symposium on System Identification, Brussels, Belgium, July 11-13 2012.

[50] T. Söderström, L. Wang, R. Pintelon and J. Schoukens: Can Errors-in-variables Systems Be Identified from Closed-loop Experiments? Automatica, vol 49, no 2, pp 681-684, February 2013.

[51] T. Söderström: Comparing some classes of bias-compensating least squares methods. Automatica, vol 49, no 3, pp 840-845, March 2013.

[52] T. Söderström, Y. Irshad, M. Mossberg and W. X. Zheng: Accuracy analysis of a covariance matching method for continuous-time errors-in-variables system identification. Automatica, vol 40, no 10, pp 2982-2993, October 2013.

[53] D. Kreiberg, T. Söderström and F. Wallentin: Errors-in-variables identification using covariance matching and structural equation modelling. 52nd IEEE Conference on Decision and Control, December 10-13 2013, Florence, Italy.

[54] T. Söderström and J. Yuz: Model validation methods for errors-in-variables estimation. 52nd IEEE Conference on Decision and Control, December 10-13, 2013, Florence, Italy.

Updated  2013-10-10 15:00:22 by Torsten Söderström.