Nội dung

Chapter 48 ASPECTS OF MODELLING TIM0 NONLINEAR TIME SERIES* TERASVIRTA Copenhagen Business School and Bank of Norway DAG TJ@STHEIM University ofBergen CLIVE W.J. GRANGER University of California Contents Abstract 1. Introduction 2. Types of nonlinear 3. 4. 5. 2919 2919 2921 models 2.1. Models from economic 2.2. Models from time series theory 2.3. Flexible statistical 2.4. State-dependent, 2.5. Nonparametric Testing 2921 theory parametric time-varying 2922 2923 models parameter and long-memory models 2924 2925 models 2926 linearity 3.1. Tests against a specific alternative 2921 3.2. Tests without a specific alternative 2930 3.3. Constancy of conditional 2933 variance 2934 2937 Specification of nonlinear models Estimation in nonlinear time series 5.1. Estimation of parameters in parametric models 2937 *The work for this paper originated when TT and DT were visiting the University of California, San Diego. They wish to thank the economics and mathematics departments, respectively, of UCSD for their hospitality and John Rice and Murray Rosenblatt, in particular. The research of TT was also supported by the University of Giiteborg, Bank of Norway and a grant from the YrjG Jahnsson Foundation. DT acknowledges financial support from the Norwegian Council for Research and CWJG from NSF, Grant SES 9023037. Handbook of Econometrics, Volume IV, Edited by R.F. Engle and D.L. McFadden 0 1994 Elsevier Science B. V. All rights reserved T.Teriisvirta 2918 5.2. Estimation of nonparametric 5.3. Estimation of restricted 6. Evaluation of estimated 7. Example 8. Conclusions References functions nonparametric models et al. 2938 and semiparametric models 2942 2945 2946 2952 2953 Ch. 48: Aspects ofModellingNonlinear Time Series 2919 Abstract This paper surveys some of the recent developments in nonlinear analysis of economic time series. The emphasis lies on stochastic models. Various classes of nonlinear models appearing in the economics and time series literature are presented and discussed. Linearity testing and estimation of nonlinear models, both parametric and nonparametric, are considered as well as post-estimation model evaluation. Data-based nonlinear model building is illustrated with an empirical example. 1. Introduction It is common practice for economic theories to postulate nonlinear relationships between economic variables, production functions being an example. If a theory suggests a specific functional form, econometricians can propose estimation techniques for the parameters, and asymptotic results about normality and consistency, under given conditions, are known for these estimates, see, e.g. Judge et al. (1983, White (1984) and Gallant (1987, Chapter 7). However, in many cases the theory does not provide a single specification, or specifications are incomplete and may not capture the major features of the actual data, such as trends, seasonality or the dynamics. When this occurs, econometricians can try to propose more general specifications and tests of them. There are clearly an immense number of possible parametric nonlinear models and there are also many nonparametric techniques for approximating them. Given the limited amount of data that is usually available in economics it would not be appropriate to consider many alternative models or to use many techniques. Because of the wide possibilities, the methods and models available for analysing nonlinearities are usually very flexible so that they can provide good approximations to many different generating mechanisms. A consequence is that, with fairly small samples, the methods are inclined to over-fit, so that if the true mechanism is linear, say, with residual variance 02, the fitted model may appear to find nonlinearity and an estimated residual variance less than u2. The estimated model will then be inclined to forecast badly in the post-sample period. It is therefore necessary to have a specific research strategy for modelling nonlinear relationships between time series. In this chapter the modelling process concentrates on a particular situation, where there is a single dependent variable y, to be explained and X, is a vector of exogenous variables. Let I, be the information set (1.1) T. TerSisvirtaet al. 2920 and denote all of the variables (and lags) used in I, by w,. The modelling will then attempt to find a satisfactory approximation for f(wJ such that process (1.2) If the error is E, = Y, - f(wJ then in some cases a more parsimonious lagged E’S in f(s). representation will specifically include The strategy proposed is as follows. (i) Test y, for linearity, using the information I,. As there are many possible forms of nonlinearity it is likely that no one test will be powerful against them all, so several tests may be needed. (ii) If linearity is rejected, consider a small number of alternative parametric models and/or nonparametric estimates. Linearity tests may give guidance as to which kind of nonlinear models to consider. (iii) These models should be estimated in-sample and compared out-of-sample. The properties of the estimated models should be checked. If a single model is required, the one that is best out-of-sample may be selected and reestimated over all available data. The strategy is by no means guaranteed to be successful. For example, if the nonlinearity is associated with a particular feature of the data, but if this feature does not occur in the post-sample evaluation period, then the nonlinear model may not perform any better than a linear model. Section 2 of the chapter briefly considers some parametric models, Section 3 discusses tests of linearity, Section 4 reviews specification of nonlinear models, Section 5 considers estimation and Section 6 evaluation of estimated models. Section 7 contains an example and Section 8 concludes. This survey largely deals with linearity in the conditional mean, which occurs if f(wJ in (1.2) can be well approximated by some linear combination cp’w, of the components of w,. It will generally be assumed that w, contains lagged values of y, plus, possibly, present and lagged values of X, including 1. This definition avoids the difficulty of deciding whether or not processes having forms of heteroskedasticity that involve explanatory or lagged variables, such as ARCH, are nonlinear. It is clear that some tests of linearity will be confused by these types of heteroskedasticity. Recent surveys of some of the topics considered here include Tong (1990) for univariate time series, Delgado and Robinson (1992) Hardle (1990) and Tjsstheim (1994) for semi- and nonparametric techniques, Brock and Potter (1993) for linearity testing and Granger and Terasvirta (1993). There has recently been a lot of interest, particularly by economic theorists, in Ch. 48: Aspects of Modelliny Nonlinear Time Series 2921 chaotic processes, which are deterministic series which have some of the linear properties of familiar stochastic processes. A well known example is the “tent-map” y, = 4y,_ 1 (1 - y,- i), which, with a suitable starting value in (0, l), generates a series with all autocorrelations equal to zero and thus a flat spectrum, and so may be called a “white chaos”, as a stochastic white noise also has these properties. Economic theories can be constructed which produce such processes as discussed in Chen and Day (1992). Econometricians are unlikely to expect such models to be relevant in economics having a strong affiliation with stochastic models and, so far, there is no evidence of actual economic data having been generated by a deterministic mechanism. A difficulty is that there is no statistical test which has chaos as a null hypothesis, so that non-rejection of the null could be claimed to be evidence in favour of chaos. For a discussion and illustrations, see Liu et al. (1992). However, a much-used linearity test has been proposed by Brock et al. (1987), based on chaos theory, whose properties are discussed in Section 3.2. The hope in using nonlinear models is that better explanations can be provided of economic events and consequently better forecasts. If the economy were found to be chaotic, and if the generating mechanism could be discovered using some learning model, say, then forecasts would be effectively exact, without any error. 2. 2.1. Types of nonlinear models Models from economic theory Theory can both be used to suggest possibly sensible nonlinear models or to take into account some optimizing behaviour, with arbitrary assumed cost or utility functions, to produce a model. An example is a relationship of the form y, = min (#Wt, t9’wJ + s,, (2.1) so that y, is the smaller of a pair of alternative linear combinations of the vector of variables used to model y,. This model arises from a disequilibrium analysis of some simple markets, with the linear combinations representing supply and demand curves; for more discussion see Quandt (1982) and Maddala (1986). If we replace the “min condition” by another variable z,_~ which may also be one of the elements of W, but not 1, we may have y, = qo’w,+ B’w,F(z,_J + “*, (2.2) where F(z,_,) = 0, z,_~ < c and F(z,_,) = 1, z,_~ > c. This is a switching regression model with switching variable z~_~ where d is the delay parameter; see Quandt (1982). In univariate time series analysis (2.2) is called a two-regime threshold autoregressive model; see, e.g. Tong (1990). Model (2.2) may be generalized by T. Teriisuirta et al. 2922 assuming a continuum of regimes instead of only two. This can be done for instance by defining F(z,_,)= (1 +expC-y(z,-,-Cc)l~-‘~ Y>O (2.3) in (2.2). Maddala (1977, p. 396) [see also Bacon and Watts (1971)] has already proposed such a generalization which is here called a logistic smooth transition regression (LSTR) model. F may also have the form of a probability density rather than a cumulative distribution function. In the univariate case this would correspond to the exponential smooth transition autoregressive (ESTAR) model (Terasvirta, 1994) or its well-known special case, the exponential autoregressive model (Haggan and Ozaki, 1981). The transition variable may represent changing political or policy regimes, high versus low inflation, upswings versus downswings of the business cycle and so forth. These switching models or their smooth transition counterparts occur frequently in theory which, for example, suggests changes in relationships when there is idle production capacity versus otherwise or when unemployment is low versus high. Aggregation considerations suggest that a smooth transition regression model may often be more sensible than the abrupt change in (2.2). Some theories lead to models that have also been suggested by time series statisticians. An example is the bivariate nonlinear autoregressive model described as a “prey-predator” model by Desai (1984) taking the form Aylt = - a + b exp(y,,), Ayzt = c + b ew(y,,)~ where y, is the logarithm of the share of wages in national income and y, is the logarithm of the employment rate. Other examples can be found (Chen and Day, 1992). The fact that some models do arise from theory justifies their consideration but it does not imply that they are necessarily superior to other models that currently do not arise from economic theory. 2.2. Models from time series theory The linear autoregressive, moving average and transfer function models have been popular in the time series literature following the work by Box and Jenkins (1970) and there are a variety of natural generalizations to nonlinear forms. If the information set being considered is I,= {y,_j, j= l,...,q,X,_i,i=O,...,q}, 4< co, denote by E, the residual from yt explained by I, and let ekt be the residual from xkt explained by I, (excluding xkt itself). The components of the models considered in this section are nonlinear functions of components such as g(y, _j), h(x,,, _ i), G(E,_ j), Ch. 48: Aspects ofModelling Nonlinear Time Series 2923 H(e,,,_i) plus cross-products such as y,_j~k,t_i,yt_jst-i,~,,t_jeb,,-i or E,_jek,t_i. A model would string together several such components, each with a parameter. For a given specification, the model is linear in the parameters so they can be easily estimated by OLS. The big questions are about the specification of the model; what components, functions and lags to use. There are so many possible components and combinations that the “curse of dimensionality” soon becomes apparent, so that choices of specification have to be made. Several classes of models have been considered. They include (9 nonlinear autoregressive, involving only functions of the dependent variable. Typically only simple mathematical functions have been considered (such as sine or cosine, sign, modulus, integer powers, logarithm of modulus or ratios of low order polynomials); (ii) nonlinear transfer function models, using functions of the lagged dependent variable and current and lagged explanatory variables, usually separately; (iii) bilinear models, y, = Cj,Jjkyr_ j~,_k + similar terms involving products of a component of X, and a lagged residual of some kind. This can be thought of as one equation of a multivariate bilinear system, as considered by Stensholt and Tjostheim (1987); (iv) nonlinear moving averages, being sums of functions of lagged residuals E,,e,; (v) doubly stochastic models which contain the cross-products between lagged y, and current and lagged components of xkt or a random parameter process and are discussed in Tjostheim (1986). Most of the models are augmented by a linear autoregressive term. There has been little consideration of mixtures of these models. Because of difficulty of analysis, lags are often taken to be small. Specifying the lag structure in nonlinear models is discussed in Section 4. A number of results are available for some of these models, such as stability for simple nonlinear autoregressive models (Lasota and Mackey, 1989), stationarity and invertibility of bilinear models or the autocorrelation properties of certain bilinear systems, but are often too complicated to be used in practice. To study stability or invertibility of a specific model it is recommended that a long simulation be formed and the properties of the resulting series be studied. There is not a lot of experience with these models in a multivariate setting and little success in their use has been reported. At present they cannot be recommended for use in preference to the smooth transition regression model of the previous section or the more structured models of the next section. A simple nonlinear autoregressive or bilinear model with just a few terms may be worth considering from this group. 2.3. Flexible statistical parametric models A number of important modelling procedures concentrate on models of the form (2.4) T. Teriisuirta et al. 2924 where w, is a vector of past y, values and past and present values of a vector of explanatory variables x, plus a constant. The first component of the model is linear and the cpj(x) are a set of specific functions in x, examples being: (i) power series, cpj(x) = xj (x is generally not a lag of y); (ii) trigonometric, q(x) = sinx or cosx, (2.4) augmented by a quadratic term w,’ Aw, gives the flexible function forms discussed by Gallant (1981); (iii) cpj(x) = q(x) for all j, where q(x) is a “squashing function” such as a probability density function or the logistic function q(x) = [ 1 + exp( - x)] - i. This is a neural network model, which has been used successfully in various fields, especially as a learning model, see, e.g. White (1989) or Kuan and White (1994); (iv) if cpj(x) is estimated nonparametrically, by a “super-smoother”, say, the method is that of “projection-pursuit”, as briefly described in the next section. The first three models are dense, in the sense that theorems exist suggesting that any well-behaved function can be approximated arbitrarily well by a high enough choice of p, the number of terms in the sum, for example Stinchcombe and White (1989). In practice, the small sample sizes available in economics limit p to a small number, say one or two, to keep the number of parameters to be estimated at a reasonable level. In theory p should be chosen using some stopping criterion or goodness-of-fit measure. In practice, a small, arbitrary value is usually chosen, or some simple experimentation is undertaken. These models are sufficiently structured to provide interesting and probably useful classes of nonlinear relationships in practice. They are natural alternatives to nonparametric and semiparametric models. A nonparametric model, as discussed in Section 2.5, produces an estimate of a function at every point in the space of explanatory variables by using some smoother, but not a specific parametric function. The distinction between parametric and nonparametric estimators is not sharp, as methods using splines or neural nets with an undetermined cut-off value indicate. This is the case, in particular, for the res’tricted nonparametric models in Section 6. 2.4. State-dependent, Priestley form time-varying parameter (1988) has discussed and long-memory models a very general class of models for a system taking the (moving average terms can also be included) where Y, is a k x 1 stochastic vector and x, is a “state-variable” consisting of x, = (Y,, Y,_ i, . . . , Y, _k + J and which is updated by a Markov system X 1+1=h(x,)+F(x,)xt+v,+1. Ch. 48: Asprcts oJ’Modelling Nonlinear Time Series 2925 Here the cp’s and the components of the matrix Fare general functions, which in practice will be approximated by linear or low-order polynomials. Many of the models discussed in Section 2.2 can be embedded in this form. It is clearly related to the extended Kalman filter [see Anderson and Moore (1979)] and to time-varying parametric ARMA models, where the parameters evolve according to some simple AR model; see Granger and Newbold (1986, Chapter 10). For practical use various approximations can be applied, but so far there is little actual use of these models with multivariate economic series. For most of the models considered in Section 2.2, the series are assumed to be stationary, but this is not always a reasonable assumption in economics. In a linear context many actual series are I(l), in that they need to be differenced in order to become stationary, and some pairs of variables are cointegrated, in that they are both I(1) but there exists a linear combination that is stationary. A start to generalizing these concepts to nonlinear cases has been made by Granger and Hallman (1991a,b). I( 1) is replaced by a long-memory concept and cointegration by a possibly nonlinear attractor, so that yt, .x, are each long-memory but there is a function g(x) such that y, - g(x,) is stationary. A nonparametric estimator for gp) is proposed and an example provided. 2.5. Nonparametric models Nonparametric modelling of time series does not require an explicit model but for reference purposes it is assumed that there is the following model y,=f(Y*-1,x,-,)+g(y,-,,x,-,)&, (2.5) with (x,) being exogenous, and where y, _ r = (y, _ i,, . . . , are vectors of lagged variables, and {.st}is a sequence Yt_i,)andx,-,=(x,-j,,...,x,-j,) of martingale differences with respect to the information set I, = {y, _ i, i > 0; x, _ i, i > O}. The joint process {y,,x,} is assumed to be stationary and strongly mixing [cf. Robinson (1983)]. The model formulation can be generalized to several variables and the instantaneous transformation of exogenous variables. There has recently been a surge of interest in nonparametric modelling; for references see, for instance, Ullah (1989), Barnett et al. (1991) and Hardle (1990). The motivation is to approach the data with as much flexibility as possible, not being restricted by the straitjacket of a particular class of parametric models. However, more observations are needed to obtain estimates of comparable variability. In econometric applications the two primary quantities of interest are the conditional mean where {y,, x,> are observed WJ?4=M(y, ,..., yp;xl )...) XJ =E(Y~lY~-i,=Y~,~~~~Yf-i,=Yp;Xt_j,=X~,...,Xf_jq=X4) (2.6) T. Teriisvirta and the conditional et al. variance (2.7) The conditional mean gives the optimal least squares predictor of y, given lagged values y, _ i, ,..., y,_ip;Xt-j ,,...., Xt-jq. Derivatives of M(x;y) can also have economic interpretations (Ullah, 1989) and can be estimated nonparametrically. The conditional variance can be used to study volatility. For (2.5) M(y,x) =f(y,x) and V(y, x) = a2g2(y, x), where 0’ = E($). As pointed out in the introduction, this survey mainly concentrates on M(y; x) while it is assumed that g(y; x) = 1. A problem of nonparametric modelling in several dimensions is the curse of dimensionality. As the number of lags and regressors increases, the number of observations in a unit volume element of regressor space can become very small, and it is difficult to obtain meaningful nonparametric estimates of (2.6) and (2.7). Special methods have been designed to overcome this obstacle, and they will be considered in Sections 4 and 5.3. Applying these methods often results in a model which is an end product in that no further parametric modelling is necessary. Another remedy to dimension difficulties is to apply semiparametric models. These models usually assume linear and parametric dependence in some variables, and nonparametric functional dependence in the rest. The estimation of such models as well as restricted nonparametric ones will*be considered in Section 5.3. 3. Testing linearity When parametric nonlinear models are used for modelling economic relationships, model specification is a crucial issue. Economic theory is often too vague to allow complete specification of even a linear, let alone a nonlinear model. Usually at least the specification of the lag structure has to be carried out using the available data. As discussed in the introduction, the type of nonlinearity best suited for describing the data may not be clear at the outset either. The first step of a specification strategy for any type of nonlinear model should therefore consist of testing linearity. As mentioned above, it may not be difficult at all to fit a nonlinear model to data from a linear process, interpret the results and draw possibly erroneous conclusions. If the time series are short that may sometimes be successfully done even in situations in which the nonlinear model is not identified under the linearity hypothesis. There is more statistical theory available for linear than nonlinear models and the parameter estimation in the former models is generally simpler than in the latter. Finally, multi-step forecasting with nonlinear models is more complicated than with linear ones. Therefore the need for a nonlinear model should be considered before any attempt at nonlinear modelling.