System identification using continuous-time models

At the Department of Systems and Control, this topic has been an active research area for some years, and a considerable amount of papers have appeared, partly summarized in the theses [7],[14],[15],[16],[24],[25],[26]. To be more specific, we have so far mainly focused on estimating the continuous-time equivalent form of an AR process from discrete-time data. It turns out that discretizing the differential operator, thus obtaining a linear regression model and then applying least squares, will in most cases lead to a severe bias in the parameter estimates. Several simple means to modify the estimators, thus keeping the computational complexity low, have been developed [1],[18],[13].

The Cramer-Rao lower bound for this problem has been derived [19] and extended to uneven sampling in [8],[6]. The accuracy of the above estimators have been analyzed and is shown to be very close to the CRB, see [20]. Recently, a general recipe for calculating the CRB for parameter estimation of irregularly sampled continuous-time ARMA processes using a state-space approach and Bang's formula has been presented [9].

The sensitivity to additional measurement noise has been treated [2]. Moreover, the methods have been extended to integrated instead of simultaneous sampling [3], and to handle unevenly sampled data [11], [5].

The effects of sampling a stochastic AR process have been examined in detail, and more detailed expressions than available in the literature, have been derived for how zeros of a stochastic process are transformed under sampling [12]. These results have also been extended to include continuous-time ARMA processes [4], [14].

At the moment the following projects are being finalized:

• A very simple and computationally efficient method for estimation of continuous-time AR processes is considered; the method introduced in [12] is here refined to yield estimates with much lower bias, see [14].

• Is it possible to extend the techniques presented above to handle the more general case of CARMA parameter estimation? Some early experiments indicate that this is the case. Some results are presented in [17]. Furthermore, some of the inherent difficulties with estimating CARMA models are discussed in [10].

• Some limiting results (when the sampling interval goes towards zero) for continuous-time ARMA systems are presented in [22]. The value lies in the analysis tools, which facilitate examination of various identification questions under fast sampling schemes.

• For errors-in-variables identification problems, the noise-free input is not known. In [23] we examined several approaches where this signal was modelled using continuous-time techniques.

• Effects of sampling and approximations when modelling continuous-time processes from discrete-time measured data are treated in [27], [28].

• When identifying networked control systems, time-delays in the data recording are often present. Furthermore, the delays may vary due to some random pattern. This, of course, causes additional difficulties. How to cope with such problems is discussed in [29], where the underlying (time-invariant) model is set up in continuous-time.

There are several interesting possibilities for further research that can be attempted within the framework provided by our previous research. Some of the possible research questions are shortly described below:

• Can the methods be modified to handle measurement noise in a general, sound way? It is well known that differentiation of process data amplifies process measurement noise.

• Can optimal sampling strategies be developed for irregular sampling?

Researcher in the project: Torsten Söderström.

Current and earlier partners: Juan Aguero, Stefano Bigi, Bengt Carlsson, Howard Fan, Graham Goodwin, Yasir Irshad, Erik G. Larsson, Erik K. Larsson, Magnus Mossberg, Juan Yuz and Y. Zou.

The project was supported in a previous phase by the Swedish Research Council for Engineering Sciences as a part of a multi-project on Statistical Signal Processing.

[1] H. Fan, T. Söderström, M. Mossberg, B. Carlsson, and Y.Zou. Estimation of continuous-time AR process parameters from discrete-time data. IEEE Transactions on Signal Processing, 47(5):1232-1244, May 1999.

[2] H. Fan, T. Söderström, and Y. Zou. Continuous-time AR process parameter estimation in presence of additive white noise. IEEE Transactions on Signal Processing, 47(12):3392-3398, December 1999.

[3] H. Fan and T. Söderström. Parameter estimation of continuous-time ar processes using integrated
sampling. In Proc. 36th IEEE Conference on Decision and Control, San Diego, December 10-12 1997.

[4] E. K. Larsson. Limiting Properties of Sampled Stochastic Systems. Technical Report 2003-028, Department of Information Technology, Uppsala University, 2003.

[5] E. K. Larsson and T. Söderström. Identification of continuous-time AR processes from unevenly sampled data. Automatica, 38(4):709-718, April 2002.

[6] E. G. Larsson and E. K. Larsson.The Cramer-Rao bound for continuous-time AR parameter estimation with irregular sampling. Circuits, Systems and Signal Processing, 21(6):581--601, 2002.

[7] E. K. Larsson. On Identification of Continuous-Time Systems and Irregular Sampling. Licentiate thesis, Uppsala University, Systems and Control Group, Uppsala, Sweden, 2001.

[8] E. K. Larsson and E. G. Larsson. Cramer-Rao bounds for continuous-time AR parameter estimation with irregular sampling. In Proc. 26th ICASSP, Salt Lake City, Utah, May 2001.

[9] E. K. Larsson and E. G. Larsson. The CRB for Parameter Estimation in Irregularly Sampled Continuous-Time ARMA Systems. IEEE Signal Processing Letters, 11(2):197-200, February 2004.

[10] E. K. Larsson and M. Mossberg. On Possibilities for Estimating Continuous-Time ARMA Parameters. In Proc. of IFAC SYSID 2003 (Symposium on System Identification), Rotterdam, The Netherlands, August 27 - 29 2003.

[11] E. K. Larsson and T. Söderström. Approaches for identifying continuous-time AR processes from unevenly sampled data. In Proc. of IFAC SYSID 2000 (Symposium on System Identification), Santa Barbara, California, June 21 - 23 2000.

[12] E. K. Larsson and T. Söderström. Identification of continuous-time AR processes by using limiting properties of sampled systems. Technical Report 2001-006, Department of Information Technology, Uppsala University, 2001.

[13] E. K. Larsson and T. Söderström. Continuous-time AR parameter estimation by using properties of sampled systems. In Proc. of the 15th IFAC World Congress, Barcelona, Spain, July 21 - 26 2002.

[14] E. K. Larsson. Identification of Stochastic Continuous-time Systems: Algorithms, Irregular Sampling and Cramér-Rao Bounds. PhD thesis, Uppsala University, Department of Information Technology, Systems and Control Group, Uppsala, Sweden, 2004.

[15] M. Mossberg. On Identification of Continuous-Time Systems Using a Direct Approach. Licentiate thesis, Uppsala University, Systems and Control Group, Uppsala, Sweden, 1998.

[16] M. Mossberg. Identification of Viscoelastic Materials and Continuous-Time Stochastic Systems. PhD thesis, Uppsala University, Systems and Control Group, Uppsala, Sweden, 2000.

[17] M. Mossberg and E. K. Larsson. Fast and Approximative Estimation of Continuous-time Stochastic Signals from Discrete-time Data. Proc. ICASSP, Montreal, Quebec, Canada, May 2004.

[18] T. Söderström, H. Fan, B. Carlsson, and S. Bigi. Least squares parameter estimation of continuous-time ARX models from discrete-time data. IEEE Trans.on Automatic Control, 42:659-673, May 1997.

[19] T. Söderström. On the Cramer-Rao lower bound for estimating continuous-time autoregressive parameters. In Proc. 14th World Congress of IFAC, volume H, pages 175-180, Beijing, P.R. China, July 5-9 1999.

[20] T. Söderström and M. Mossberg. Performance evaluation of methods for identifying continuous-time autoregressive processes. Automatica, 36:53-59, 2000.

[21] E. K. Larsson. M. Mossberg and T. Söderström: The Cramer-Rao bound for estimation of continuous-time ARX parameters form irregularly sampled data. 16th IFAC World Congress, Prague, July 4-8, 2005.

[22] E. K. Larsson: Limiting sampling results for continuous-time ARMA systems. International Journal of Control, vol 78(7):461-473, May 2005.

[23] T. Söderström, E. K. Larsson, K. Mahata and M. Mossberg: Using continuous-time modeling for errors-in-variables identification, 14th IFAC Symposium on System Identification, Newcastle, Australia, March, 29-31, 2006.

[24] E. K. Larsson, M. Mossberg and T. Söderström: An overview of important practical aspects of continuous-time ARMA system identification. Circuits, Systems and Signal Processing, vol 25, No 1, pp 17-46, 2006.

[25] E. K. Larsson, M. Mossberg and T. Söderström: Identification of continuous-time ARX models from irregularly sampled data. IEEE Transactions on Automatic Control. vol 52, no 3, pp 417-427, March 2007.

[26] E. K. Larsson and M. Mossberg and T. Söderström: Estimation of Continuous-time Stochastic System Parameters. In H. Garnier and L. Wang, eds.: Identification of Continuous-time Models from Sampled Data, Springer-Verlag, 2008.

[27] J. C. Aguero, G. C. Goodwin, T. Söderström and J. I. Yuz: Sampled Data Errors in Variables Systems. SYSID 2009, IFAC 15th Symposium on System Identification, Saint-Malo, France, July 6-8, 2009.

[28] T. Söderström: Sampling approximations for continuous-time identification. SYSID 2009, IFAC 15th Symposium on System Identification, Saint-Malo, France, July 6-8, 2009.

[29] M. Mossberg, Y. Irshad and T. Söderström: Sub-optimal networked system identification based on covariance functions. NecSys '10, 2nd IFAC Workshop on Distributed Estimation and Control in Networked Systems, Annecy, France, September 13-14, 2010.