Recall that for the general ARIMA() model we have

where and , where is the symmetric Toeplitz covariance matrix for the draws from the ARMA/ARFIMA process for the unconditional residuals (Doornik and Ooms 2003). Note that direct evaluation of this function requires the inversion of a large matrix which is impractical for large for both storage and computational reasons.

The ARIMA model restricts to be a known integer. The ARFIMA model treats as an estimable parameter.

It is well-known that for ARIMA models where is a known integer, we may employ the Kalman filter to efficiently evaluate the likelihood. The Kalman filter works with the state space prediction error decomposition form of the likelihood, which eliminates the need to invert the large matrix .

First, we must compute the autocovariances of the ARFIMA process that appear in the which an involve an infinite order MA representation. Fortunately, Hosking (1981) and Sowell (1992) describe closed-form alternatives and Sowell (1992) derives efficient recursive algorithms using hypergeometric functions.

Second, we must compute the determinant of the variance matrix and generalized (inverse variance weighted) residuals in a manner that is computationally and storage efficient. Doornik and Ooms (2003) describe a Levinson-Durbin algorithm for efficiently performing this operation with minimal operation count while eliminating the need to store the full matrix .

Third, where possible we follow Doornik and Ooms (2003) in concentrate the likelihood with respect to the regression coefficients and the scale parameter .

and the ARFIMA estimates may be obtained by minimizing .

The recursive innovation equation in Equation (24.54) is easy to evaluate given parameter values, lagged values of the differenced , , and estimates of the lagged innovations. Note, however that neither the nor the can be substituted in the first period as they are not available until we start up the difference equation.

We discuss below methods for starting up the recursion by specifying presample values of and . Given these presample values, the conditional likelihood function for normally distributed innovations is given by

Notice that the conditional likelihood function depends on the data and the mean and ARMA parameters only through the conditional least squares function , so that the conditional likelihood may be maximized by minimizing .

Consider an AR() regression model of the form:

for . Estimation of this model using conditional least squares requires computation of the innovations for each period in the estimation sample.

so we can see that we require pre-sample values to evaluate the AR process at

into a nonlinear model by substituting the second equation into the first, writing in terms of observables and rearranging terms:

Notice that we require observation on the and in the period before the start of the recursion. If these values are not available, we must adjust the period of interest to begin at so that the values of the observed data in may be substituted into the equation to obtain an expression for .

It is important to note that textbooks often describe techniques for estimating linear AR models like Equation (24.58). The most widely discussed approaches, the Cochrane-Orcutt, Prais-Winsten, Hatanaka, and Hildreth-Lu procedures, are multi-step approaches designed so that estimation can be performed using standard linear regression. These approaches proceed by obtaining an initial consistent estimate of the AR coefficients and then estimating the remaining coefficients via a second-stage linear regression.

In contrast, the EViews conditional least squares estimates the coefficients and are estimated simultaneously by minimizing the nonlinear sum-of-squares function (which maximizes the conditional likelihood). The nonlinear least squares approach has the advantage of being easy-to-understand, generally applicable, and easily extended to models that contain endogenous right-hand side variables and to nonlinear mean specifications.

Consider an MA() regression model of the form:

for . Estimation of this model using conditional least squares requires computation of the innovations for each period in the estimation sample.

Computing the innovations is a straightforward process. Suppose we have an initial estimate of the coefficients, , and estimates of the pre-estimation sample values of :

Then, after first computing the unconditional residuals , we may use forward recursion to solve for the remaining values of the innovations:

for .

All that remains is to specify a method of obtaining estimates of the pre-sample values of :

One may employ backcasting to obtain the pre-sample innovations (Box and Jenkins, 1976). As the name suggests, backcasting uses a backward recursion method to obtain estimates of for this period.

To start the recursion, the values for the innovations beyond the estimation sample are set to zero:

for . The final values, , which we use as our estimates, may be termed the backcast estimates of the pre-sample innovations. (Note that if your model also includes AR terms, EViews will -difference the to eliminate the serial correlation prior to performing the backcast.)

Alternately, one obvious method is to turn backcasting off and to set the pre-sample to their unconditional expected values of 0:

Whichever methods is used to initialize the presample values, the sum-of-squared residuals (SSR) is formed recursively as a function of the and , using the fitted values of the lagged innovations:

and the expression is minimized with respect to and .

The backcast step, forward recursion, and minimization procedures are repeated until the estimates of and converge.