Econometrıcs II (ENG) — Ünite 4 Soru-Cevap

Univariate time series models allow to be explained by past, or lagged, values of y itself and stochastic error terms. For this reason, these models are sometimes called non-theoretical models because they are not derived from any economic theory. In stationary autoregressive-moving average (ARMA) models, the model that matches with data is determined without economic theory.

Stationary process implies statistical properties remain constant over time. Stationarity refers to a statistical equilibrium based on the basic idea that the laws of probability prevailing in the process do not change with time. For all choices of time points t1,t2,…,tn and time lag k, the stochastic process is called strong stationary if the joint-distribution of y(t1),y(t2),…,y(tn) is the same as the joint-distribution of y(t1+k),y(t2+k),…,y(tn+k). In time series analysis, it is sufficient for the series to be weakly stationary (covariance stationary) to make statistical inferences about the structure of the observed series.

The correlation between yt and y(t+k) is defined as follows 


ρk = Cov (yt, yt+k ) / Var (yt) = γk/ γ0

As functions of γk and ρk are called autocovariance and autocorrelation functions, respectively

The partial autocorrelation between yt and yt+k is defined as the correlation between these two variables after the effect of the variables yt+1,yt+2,…,yt+k-1 is removed.

Except at lag zero, a white noise process has zero autocovariance and constant mean and variance.

ARMA combines both autoregressive and moving average components, offering a comprehensive view of time series dynamics. ARIMA, an extension of ARMA, introduces differencing to achieve stationarity.

The partial autocorrelation function of the AR(p) process will be zero for k>p, while it takes a nonzero value for k≤p. Partial autocorrelations for this process can be found by using the Yule-Walker system of equations and the process autocorrelation function.

First order autoregressive, AR (1) can be expressed as

yt = ϕ1 yt-1 + εt

or

(1 - ϕ1 L)yt = εt

Second-order autoregressive, AR (2) can be expressed as

yt = ϕ1 yt-1 + ϕ2 yt-2 + εt

or

(1 - ϕ1 L - ϕ2 L2)yt = εt

A moving average model of order q, denoted as MA(q), can be expressed as;

yt = εt - θ1 εt-1 - θ2 ϵt-2 - … - θq εt-q 

where εt is a white noise disturbance term.

First order moving average, MA(1), can be expressed as

yt=εt - θ1 εt-1 

or

yt =(1 - θ1L)εt

This process is always stationary

Second-order moving average, MA(2), can be expressed as

y = εt - θ1 εt-1 - θ2 εt-2

or

yt = (1 - θ1 L - θ2 L2)εt

The MA(2) process is always stationary.

The partial autocorrelation function of the MA(2) process will have a tendency to sinus fluctuations if the roots of (1 - θ1 L - θ2 L2)=0 are complex, and exponentially decrease if the roots are real.

A stationary and invertible process can be represented in either moving average or autoregressive form. However, it is possible to encounter the problem of containing too many parameters in both notations. This problem exists when the optimal model for the finite order MA and the finite order AR models is a high order model. Too many parameters will reduce the efficiency in estimating parameters. Therefore, it is necessary to establish a model to include both autoregressive and moving average terms. These models are called autoregressive-moving average (ARMA) processes.

The ARMA(1,1) process is defined as follows 

(1 - ϕ1 L)yt = (1 - θ1 L)εt

or

yt = ϕ1 yt-1 + εt - θ1 εt-1

In order for the ARMA(1,1) process to be stationary, it must be |ϕ1|<1 restriction, and for invertibility, |θ1|<1 its restriction must be provided.

1) AR(p): 

ACF: Tails off exponential decay or damped sine wave
PACF: Cuts off after lag p.

2) MA(q)

ACF: Cuts off after lag q.
PACF: Tails off exponential decay or damped sine wave

3) ARMA (p,q) 

ACF: Tails off after lag (q-p)
PACF: Tails off after lag (p-q)

Stage 1: It is the stage of plotting the time series and selecting the proper transformation.


The first stage of time series analysis is to plot the data. Through such an examination, it is possible to get an idea of the trend, seasonal change, sudden change, non-constant variance characteristics of the series and therefore whether it is stationary or not. Such a review provides a basic idea of proper transformations for data. Commonly used transformations in time series analysis are variance stability or differencing. Since there are negative values as a result of the differencing, it is appropriate to perform the variance stability  transformation first

Step 2: Determination of the appropriate p and q sequences for the ARMA(p, q) model

The sample autocorrelation and sample partial autocorrelation functions are calculated from a stationary time series. For sample autocorrelation and partial autocorrelation functions, it is suggested to take the maximum lag length of approximately n/4. In addition, in the identification of the ARMA model, the sample size must be at least 50 (Wei, 1990). The way to determine the order p and q in the ARMA(p, q) model is to determine which of the sample autocorrelation and partial autocorrelation functions match to the theoretical autocorrelation and partial autocorrelation functions of the stationary ARMA models.

Consider the first-order autoregressive model 

Yt-μ=ϕ1 (Yt-1 - μ) + εt

This model is a regression model where the current period of Y is dependent and the one-previous period lagged of Y is the independent variable. In this case, the LS method will be based on the minimization of the sum of error squares.

Residuals that are estimates of error terms can be calculated by means of the model which is estimated of parameters. By finding sample autocorrelation and sample partial autocorrelation functions for residuals, the equality of autocorrelation coefficients between errors to zero can be tested individually. In addition, by selecting a given maximum lag length, the autocorrelation coefficients between error terms can be tested for equality to zero jointly. The joint null hypothesis for this test is as follows

H0: ρ1 = ρ2 = … = ρm = 0


Test statistic is the modified Q statistic originally proposed Box and Pierce (1970). Under the null hypothesis of model adequacy, Ljung and Box (1978) show that the Q statistic approximately follows the χ2 m-k distribution.

In the time series analysis, several models may adequately represent a given data set. In some cases, it is easy to choose a model that is adequately represented with the data, but sometimes it can be difficult to determine the appropriate model. Another technique, which removes some of the subjectivity involved in interpreting the autocorrelation function and partial autocorrelation function, is to use what are known as information criteria. In this section, we introduce some model selection criteria based on residuals.

The most commonly used information criteria in practice are those proposed by Akaike (1973) and Schwartz (1978).

Model selection criteria for alternative ARMA(p,q) models are calculated and selected as the optimal model that have minimum AIC or minimum SBC value from among them.

For ARMA(1,1) model, 

Ŷt(1) = μ + ϕ1 (Yt - μ) - θ1 εt 

with 

Ŷt(2) = μ + ϕ1(Ŷt(1) - μ)

and, more generally,

Ŷt(l) = μ + ϕ1(Ŷt(l - 1) - μ)