A common occurrence in time series regression is the presence of correlation between residuals and their lagged values. This serial correlation violates the standard assumption of regression theory which requires uncorrelated regression disturbances. Among the problems associated with unaccounted for serial correlation in a regression framework are:

A popular framework for modeling serial dependence is the Autoregressive-Moving Average (ARMA) and Autoregressive-Integrated-Moving Average (ARIMA) models popularized by Box and Jenkins (1976) and generalized to Autoregressive-Fractionally Integrated-Moving Average (ARFIMA) specifications.

(Note that ARMA and ARIMA models which allow for explanatory variables in the mean are sometimes termed ARIMAX and ARIMAX. We will generally use ARMA to refer to models both with and without explanatory variables unless there is a specific reason to distinguish between the two types.)

where

is a lag polynomial that characterizes the AR process.If we add a mean to the model, we obtain:

Substituting the second equation into the first, we obtain the regression form

In the representation it is easy to see that the AR(1) model incorporates the residual from the previous observation into the regression model for the current observation.

Rearranging terms and using the lag operator, we have the polynomial form

Substituting and rearranging terms, we get the regression

and the polynomial form

Adding a mean to the model, we get the mean adjusted form:

and

We may combine the AR and the MA specifications to define an autoregressive model moving average (ARMA) model:

The simplest ARMA model is first-order autoregressive with a first-order moving average error:

or equivalently,

Box and Jenkins (1976) recommend the use of seasonal autoregressive (SAR) and seasonal moving average (SMA) terms for monthly or quarterly data with systematic seasonal movements. Processes with SAR and SMA terms are ARMA models constructed using products of lag polynomials. These products produce higher order ARMA models with nonlinear restrictions on the coefficients.

The SAR is not intended to be used alone. The SAR allows you to form the product of lag polynomials, with the resulting lag structure defined by the product of the AR and SAR lag polynomials.

For example, a second-order AR process without seasonality is given by,

For quarterly data, we might wish to add a SAR(4) term because we believe that there is correlation between a quarter and the quarter the year previous. Then the resulting process would be:

Expanded terms, we see that the process is equivalent to:

For example, second-order MA process without seasonality may be written as

or using lag operators:

To take account of seasonality in a quarterly workfile you may wish to add an SMA(4). Then the resulting process is:

The process is equivalent to:

or

Stationary processes are said to have long memory when autocorrelations are persistent, decaying more slowly than the rate associated with ARMA models. Modeling long term dependence is difficult for standard ARMA specifications as it requires non-parsimonious, large-order ARMA representations that are generally accompanied by undesirable short-run dynamics (Sowell, 1992).

In Box-Jenkins ARIMA modeling and forecasting, you assemble a complete forecasting model by using combinations of the three ARIMA building blocks described above. The first step in forming an ARIMA model for a series of residuals is to look at its autocorrelation properties. You can use the correlogram view of a series for this purpose, as outlined in “Correlogram” .

This phase of the ARIMA modeling procedure is called identification (not to be confused with the same term used in the simultaneous equations literature). The nature of the correlation between current values of residuals and their past values provides guidance in selecting an ARIMA specification.

If you suspect that there is a distributed lag relationship between your dependent (left-hand) variable and some other predictor, you may want to look at their cross correlations before carrying out estimation.

The next step is to decide what kind of ARIMA model to use. If the autocorrelation function dies off smoothly at a geometric rate, and the partial autocorrelations were zero after one lag, then a first-order autoregressive model is appropriate. Alternatively, if the autocorrelations were zero after one lag and the partial autocorrelations declined geometrically, a first-order moving average process would seem appropriate. If the autocorrelations appear to have a seasonal pattern, this would suggest the presence of a seasonal ARMA structure. Along these lines, Box and Jenkins (1976) recommend the use of seasonal autoregressive (SAR) and seasonal moving average (SMA) terms for monthly or quarterly data with systematic seasonal movements.

For example, we can examine the correlogram of the DRI Basics housing series in the “Hs.WF1” workfile by setting the sample to “1959m01 1984m12” then selecting View/Correlogram… from the HS series toolbar. Click on OK to accept the default settings and display the result.

The “wavy” cyclical correlogram with a seasonal frequency suggests fitting a seasonal ARMA model to HS.

The goal of ARIMA analysis is a parsimonious representation of the process governing the residual. You should use only enough AR and MA terms to fit the properties of the residuals. The Akaike information criterion and Schwarz criterion provided with each set of estimates may also be used as a guide for the appropriate lag order selection.

After fitting a candidate ARIMA specification, you should verify that there are no remaining autocorrelations that your model has not accounted for. Examine the autocorrelations and the partial autocorrelations of the innovations (the residuals from the ARIMA model) to see if any important forecasting power has been overlooked. EViews provides several views for diagnostic checks after estimation.