Model selection

The parametric (or model-based) methods of signal processing require often not only the estimation of a vector of real-valued parameters but also the selection of one or several integer-valued parameters that are equally important for the specification of a data model. Examples of these integer-valued parameters of the model include the orders of an autoregressive moving average model, the number of sinusoidal components in a sinusoids-in-noise signal, and the number of source signals impinging on a sensor array. In each of these cases, the integer-valued parameters determine the dimension of the parameter vector of the data model, and they must be estimated from the data.

The task of selecting the above mentioned integer parameters is known as model (order) selection. Our research group has in the last few years published several papers on this topic in well-known journals. The papers concern the following:

Information Criteria
A review on order selection algorithms like Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC). [1], [3]
Cross Validation Rules for Model Order Selection
Two previously presented order selection rules based on cross-validation are revisited. [2]
The Multi-model Approach
By considering several models instead of only one and weighing them in a specific manner, better performance for e.g. prediction can be obtained. [4], [5]
Estimation of sparse models
A sparse model, by definition, comprises many parameters equal to zero. We consider methods to select which parameters of a model should be included (i.e. given non-zero values), and which should not. See Figure 1. [6], [7]

sparse_channel2.GIF

Figure 1: Example of a sparse channel with L = 17 taps where we want to estimate n = 8 taps. A non-sparse model would consist of the 8 initial taps, whereas a sparse model could consist, e.g., of the 8 largest taps.

Empirical Bayesian model selection
Following a Bayesian framework it is possible to satisfy certain optimality criteria, such as selecting the model with the highest posterior probability, and so on. However, the use of Bayesian methods requires the specification of certain a priori parameters, which may not be available. We have studied empirical Bayesian solutions to the model selection problem: In this type of approach the values of the a priori parameters are first estimated from the data, and then used as if they were the true parameter values.

Selected publications

  1. Linear Regression With a Sparse Parameter Vector. Erik G. Larsson and Yngve Selén. In volume 55, issue 2 of IEEE Transactions on Signal Processing, IEEE, pp 451-460, 2007.
  2. RAKE Receiver for Channels with a Sparse Impulse Response. Yngve Selén and Erik G. Larsson. In volume 6, issue 9 of IEEE Transactions on Wireless Communications, pp 3175-3180, 2007.
  3. Adaptive equalization for frequency-selective channels of unknown length. Erik G. Larsson, Yngve Selén, and Peter Stoica. In volume 54, issue 2 of IEEE Transactions on Vehicular Technology, pp 568-579, 2005.
  4. Cross-Validation Rules for Order Estimation. Peter Stoica and Yngve Selén. In volume 14, issue 4 of Digital Signal Processing, pp 355-371, 2004.
  5. Model-Order Selection: A review of information criterion rules. Peter Stoica and Yngve Selén. In volume 21, issue 4 of IEEE Signal Processing Magazine, pp 36-47, 2004.
  6. On information criteria and the generalized likelihood ratio test of model order selection. Peter Stoica, Yngve Selén, and Jian Li. In volume 11, issue 10 of IEEE Signal Processing Letters, pp 794-797, 2004.
  7. Multi-model approach to model selection. Peter Stoica, Yngve Selén, and Jian Li. In volume 14, issue 5 of Digital Signal Processing, pp 399-412, 2004.