[ Home | GAUSS | GAUSS Engine | GAUSS Apps | 3rd Party Apps | Keywords Index ]

Constrained Maximum Likelihood

Constrained Maximum Likelihood (CML) solves the general maximum likelihood problem subject to linear or nonlinear and equality or inequality parameters constraints..


Key Features

  • Fast procedures : fastCML, fastCMLBoot, fastCMLBayes, fastCMLProfile, fastCMLPflClimits

  • "Kiss-Monster" random numbers used in the bootsrap and random line search procedures

  • Multiple Point Numerical Gradients

  • Grid Search Method

  • Trust Region Method


Major Features of CML

  • fastCML, fastCMLBoot, fastCMLBayes, fastCMLProfile, and fastCMLPflClimits can speed convergence times from 10 to 180 % over earlier versions of CML, depending on the type of problem.

  • CML includes built-in models for estimating numerous limited dependent variable models, including exponential, exponential gamma, and Pareto duration models with or without censoring, Poisson, hurdle Poisson, seemingly unrelated regression Poisson, and latent variable Poisson models.

CML uses the Sequential Quadratic Programming method in combination with a number of user-selectable descent methods and several selectable line search methods. Choice includes :

  • Newton-Raphson
  • quasi-Newton (DFP and BFGS)
  • scaled quasi-Newton
  • BHHH
  • PCRG
  • steepest descent

  • Confidence limits may be computed using bootstrap or Bayesian methods (using a weighted likelihood bootstrap) or by inverting Wald or likelihood ratio statistics. Confidence limits from inverting the likelihood ratio statistics are profile likelihood confidence limits.

  • A trust region method constrains the direction at each iteration to an interval. This prevents poor starting values from pushing current estimates into far off regions. It also aids in resisting convergence at saddle points.

  • A grid search method keeps CML working when it would otherwise halt without convergence. In most cases convergence is eventually achieved.

  • Gradients can be numerically calculated or provided by the user. Accuracy is considerably improved by adding points to the usual numerical gradient calculation. Greater accuracy is gained by adding more points.

  • The boostrap and Bayesian procedures and the random line search algorithm implement the new "Kiss-Monster" random number generator introduced in GAUSS 3.6. This generator has a period of approximately 10^8859, long enough for any serious Monte Carlos work.

Several examples are included with CML, including tobit, nonlinear curve fitting, simultaneous equations, nonlinear simultaneous equations, and factor analysis models.

Example

CML is especially suited for models with complex constraints on parameters. Because CML provides for general nonlinear constraints, it is possible to enforce any type of constraint. The GARCH model requires a number of inequality constraints to ensure the stationarity of the model.

Here a TGARCH(2,2) model is estimated for a well-know stock index, measured monthly. The residuals are assumed to have a Student's t distribution in order to measure the "fatness" or platykurtosis of the tails of the observed distribution. The extend to which the 'NU" parameters (the "degrees of freedom" parameters in the t distribution) is greater than 2 indicates the amount of platykustosis. In this case, the index is clearly platykurtosis.

The "delta2" parameters is on the constraint floor. A Lagrange multiplier is available for testing that the constraint is the same as the gradient, both equalling .0011. This result, plus the fact that the lower confidence limits of the "alpha" parameters are on the constraint boundary, suggest taht a TGARCH(1,1) model might be better model. Here are the TGARCH(1,1) model estimates :

The likelihood ratio statistic for testing the equivalence of the TGARCH(2,2) and TGARCH(1,1) models is .4478 (=265*(2.91808-2.91639)). It is statistically significant at the .05 level. The likelihood ratio of the TGARCH(1,1) over the GARCH(1,1) model, in which the errors are assumed to have a Normal distribution, is 9.9665 with 1 degree of freedom. We thus accept the TGARCH(1,1) model under the rule of parsimony over both the TGARCH(2,2) and GARCH(1,1) models.

The likelihood ratio statistic for the GARCH(1,1) model over an ordianry least squares model is 75.2043 with 4 degrees of freedom, which is highly significant and is strong evidence for the GARCH specification of the stock index.

Here are kernel density plots of the distribution of the coefficients of the GARCH(1,1) model from a bootstrap :

CML provides for a variety of methods for statistical inference. Among them are the usual standard errors and t-statistics, confidence limits by inversion of the Wald statistics or the likelihood ratio statistic, Bayesian limits by the method of weighted likelihood bootstrap, as well as the usual bootstrap method.

Platform : Windows / Linux / Solaris / Mac
Requirements : GAUSS / GAUSS Light version 10 or higher. Linux requires version 10.0.4 or higher.