Non-linear system identification in systems biology

Background

Systems biology is evolving rapidly today, resulting in an increasing need for dynamical models of biological systems. Very often the underlying chemistry and biology is complicated, which leads to complicated models in case modeling from first principles is performed. In addition to this, the models needed to understand the biological processes are often nonlinear, a fact that complicates the situation further.

The present project therefore aims to apply non-linear system identification and non-linear machine learning techniques to model biological systems of different kinds. Another target is to study fundamental limitations for this, and to develop new and better algorithms for identification of non-linear dynamic biological systems. Here algorithms based on particle filtering ideas will be of prime interest, as will new techniques from the field of machine learning.

Background work of the department of information technology related to the present project appear in 7-12. The references 9, 11 and 12 describe algorithms for recursive identification of non-linear dynamic autonomous systems, i.e. systems without an input signal. Such identified models can be used to describe oscillation and chaotic behaviour in biological systems. The algorithms of 11 and 12 are based on the result of 9, which states that a periodic signal can be modeled by a non-linear ordinary differential equation (ODE), provided that the phase plane plot of the signal does not intersect itself. The references 7, 8 and 10 describe corresponding algorithms for the situation where an input signal is present.

Both the algorithms with and without input signal dependence mentioned above are based on modeling the right hand side of the non-linear ODE using only one non-trivial right hand side component, with the rest of the right hand side components creating a chain of integrators. The non-trivial right hand side function is parameterized using a polynomial function in the states (and inputs). A discretization with the Euler method allows for the implementation of different identification algorithms, as outlined in 7, 8, 11, and 12. The advantage is that it is possible to obtain highly accurate models by only estimating a few parameters.The material of 7-12 hence provide initial tools that can be used in a variety of cases relevant for identification in systems biology.

Identification of autonomous biological systems

Recently, a software package for identification of non-linear autonomous dynamical systems of order two was completed, see 6. The algorithms of that software package include the ones of 11 and 12, in addition to the algorithm used for modeling in the paper 1. MATLAB scrips for generation of simulated data, as well as for evaluation, are included. The software package is available for free download, please refer to the list of references for a link.

The report 5 and the paper 1 builds on 9 and presents a theoretical result that explains when it is at all possible to uniquely determine the non-linear dynamic equations of a biological system in stable oscillation, from measured data. As hinted about above the minimal order needed for this is tied to the minimal dimension in which the stable orbit of the system does not intersect itself. This is illustrated with a simulated fourth order Hodgkin-Huxley spiking neuron model, which is identified using a non-linear second order differential equation model. The algorithms of 6 and 12 were applied. The simulated results indicate that the underlying fourth order model of the spiking neuron cannot be uniquely determined given only the measured data. This is illustrated in Fig. 1.

Figure 1: The figure illustrates modeling of the fourth order Hodgkin-Huxley neuron model of 1. Simulation generated the data (blue), which was then identified with a second order model with 16 parameters used to model a polynomial right hand side of the second order ODE (red).

In order to improve the performance and accuracy of the algorithms of 8-10 particle filtering based indentification was applied to the Hodgkin-Huxley model in the conference paper 3. The paper shows that significantly better accuracy can be obtained, at the expense of a dramatically increased computational complexity. To avoid the complexity increase the least squares algorithm of 6 was used for modeling of the same Hodgkin-Huxley model, as described in the conference paper 4. The result obtained for polynomial degrees of 5 and 4 in the states of the second order model is very, very good and comparable to the particle filtering result of 3. The phase plane plots of the data and the identified model appears in Fig. 2. The time evolution of the data and the simulated output of the identified model appears in Fig. 3 and Fig. 4.

Figure 1: The phase plane plot of the data generated by the Hodgkin-Huxley model (blue), and the identified model (red) obtained with the least squares algorithm using polynomial degrees of 5 and 4 in the states.

Figure 3: The simulated Hodgkin-Huxley membrane potential, used as data.

Figure 4: The simulated model output corresponding to the estimated 2:nd order model of Fig. 2.

The techniques of least squares identification have been further developed to handle the case where both states are measureable. The algorithm is available in 6. The paper 2 applied this modified scheme for estimation of biological clock dynamics, more precisely a circadian clock in the paper 2. This type of clock controls the daily rhythm in the cells of a variety of organisms. The problem turned out to be difficult, and the best results were obtained with polynomial degrees of 3 and 9 in the states of the second order model. The phase plane plot of the result appears in Fig. 5.

Figure 5: The phase portrait of the identified circadian clock. Data (red) and estimated model (blue). The data is challenging as can be seen from the very sharp corners.

References

1. T. Wigren, "Model order and identifiability of non-linear biological systems in stable oscillation", ACM/IEEE Trans. on Computational Biology and Bioinformatics, vol. 12, no. 6, pp. 1479-1484, 2015. DOI: 10.1109/TCBB.2015.2404799.

2. T. Wigren and P. Lötstedt, "Nonlinear identification of biological clock dynamics", to appear in Proc. IEEE CDC 2015, Osaka, Japan, pp. 602-608, December 15-18, 2015.

3. J. Nordh, T. Wigren, T. Schön and B. Bernhardsson, "Particle filtering based identification for autonomous nonlinear ODE models", Proc. SYSID 2015, Beijing, PRC, pp. 415 - 420, October 19-21, 2015.

4. T. Wigren, "Nonlinear identification of neuron models", Proc. IEEE CCA 2015, Sydney, Australia, pp. 1340-1346, Sep. 21-23, 2015. DOI: 10.1109/CCA.2015.7320798.

5. T. Wigren, "Identifiability and limit cycles", Technical Reports from the Department of Information Technology, 2014-016, Uppsala University, Uppsala, Sweden, May, 2014.

6. T. Wigren, "Software for recursive identification of second order autonomous systems", Technical Reports from the Department of Information Technology, 2014-014, Uppsala University, Uppsala, Sweden, April, 2014.

7. L. Brus, T. Wigren and B. Carlsson, "Initialization of a nonlinear identification algorithm applied to laboratory plant data", IEEE Trans. Contr. Sys. Tech., vol. 16, no. 4, pp. 708-716, 2008.

8. T. Wigren, "Recursive prediction error identification and scaling of nonlinear state space models using a restricted black box parameterization", Automatica, vol. 42, no. 1, pp. 159-168, 2006.

9. T. Wigren and T. Söderström, A second order ODE is sufficient for modeling of many periodic signals, Int. J. Contr. , vol. 78, no. 13, pp. 982-996, 2005.

10. T. Wigren, "MATLAB software for recursive identification and scaling using a structured nonlinear black-box model - Revision 2", Technical Reports from the department of Information Technology 2005-022, Uppsala University, Uppsala, Sweden, August, 2005.

11. T. Wigren, E. Abd-Elrady and T. Söderström. Least squares harmonic signal analysis using periodic orbits of ODEs, Proc. of the IFAC Symposium on System Identification, Rotterdam, The Netherlands, August 27-29, 2003.

12. T. Wigren, E. Abd-Elrady and T. Söderström. Harmonic signal analysis with Kalman filters using periodic orbits of nonlinear ODEs, Proc. of the IEEE International Conference on Acoustics, Speech and Signal Processing, Honkong, China, April 6-10, 2003.