Elsevier

Applied Soft Computing

Volume 11, Issue 2, March 2011, Pages 2664-2675
Applied Soft Computing

A novel hybridization of artificial neural networks and ARIMA models for time series forecasting

https://doi.org/10.1016/j.asoc.2010.10.015Get rights and content

Abstract

Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing decision makers in many areas. Both theoretical and empirical findings have indicated that integration of different models can be an effective way of improving upon their predictive performance, especially when the models in combination are quite different. Artificial neural networks (ANNs) are flexible computing frameworks and universal approximators that can be applied to a wide range of forecasting problems with a high degree of accuracy. However, using ANNs to model linear problems have yielded mixed results, and hence; it is not wise to apply ANNs blindly to any type of data. Autoregressive integrated moving average (ARIMA) models are one of the most popular linear models in time series forecasting, which have been widely applied in order to construct more accurate hybrid models during the past decade. Although, hybrid techniques, which decompose a time series into its linear and nonlinear components, have recently been shown to be successful for single models, these models have some disadvantages. In this paper, a novel hybridization of artificial neural networks and ARIMA model is proposed in order to overcome mentioned limitation of ANNs and yield more general and more accurate forecasting model than traditional hybrid ARIMA-ANNs models. In our proposed model, the unique advantages of ARIMA models in linear modeling are used in order to identify and magnify the existing linear structure in data, and then a neural network is used in order to determine a model to capture the underlying data generating process and predict, using preprocessed data. Empirical results with three well-known real data sets indicate that the proposed model can be an effective way to improve forecasting accuracy achieved by traditional hybrid models and also either of the components models used separately.

Introduction

Time series forecasting is an active research area that has drawn considerable attention for applications in variety of areas. With the time series approach to forecasting, historical observations of the same variable are analyzed to develop a model describing the underlying relationship. Then the established model is used in order to extrapolate the time series into the future. This modeling approach is particularly useful when little knowledge is available on the underlying data generating process or when there is no satisfactory explanatory model that relates the prediction variable to other explanatory variables. Over the past several decades, much effort has been devoted to the development and improvement of time series forecasting models [1].

Artificial neural networks (ANNs) are one of the most important types of nonparametric nonlinear time series models, which have been proposed and examined for time series forecasting. The basic structure of and operations performed by the ANN emulate those found in a biological neural systems [2]. ANNs have some advantages over other forecasting models, which make them attractive in forecasting tasks. First, artificial neural networks have flexible nonlinear function mapping capability, which can approximate any continuous measurable function with arbitrarily desired accuracy [3], [4]. Second, being nonparametric and data-driven models, artificial neural networks impose few prior assumptions on the underlying process from which data are generated [1]. Because of this property, artificial neural networks are less susceptible to model misspecification problem than most parametric nonlinear methods. Third, artificial neural networks are adaptive in nature. The adaptivity implies that the network's generalization capabilities remain accurate and robust in a nonstationary environment whose characteristics may change over time. Fourth, artificial neural networks models use only linearly many parameters, whereas traditional polynomial, spline, and trigonometric expansions use exponentially many parameters to achieve the same approximation rate [5].

Given the advantages of artificial neural networks, it is not surprising that this methodology has attracted overwhelming attention in time series forecasting. Artificial neural networks have been found to be a viable contender to various traditional time series models [6], [7], [8]. Lapedes and Farber [9] report the first attempt to model nonlinear time series with artificial neural networks. De Groot and Wurtz [10] present a detailed analysis of univariate time series forecasting using feedforward neural networks for two benchmark nonlinear time series. Chakraborty et al. [11] conduct an empirical study on multivariate time series forecasting with artificial neural networks. Poli and Jones [12] propose a stochastic neural network model based on Kalman filter for nonlinear time series prediction. Cottrell et al. [13] address the issue of network structure for forecasting real world time series. Berardi and Zhang [14] investigate the bias and variance issue in the time series forecasting context. In addition, several large forecasting competitions [15], [16] suggest that neural networks can be a very useful addition to the time series forecasting toolbox.

Although ANNs have the advantages of accurate forecasting, their performance in some specific situation is inconsistent. In the literature, several papers are devoted to comparing ANNs with the traditional methods [1]. Despite the numerous studies, which have shown ANNs are significantly better than the conventional linear models and their forecast considerably and consistently more accurately, some other studies have reported inconsistent results. Foster et al. [17] find that ANNs are significantly inferior to linear regression and a simple average of exponential smoothing methods. Brace et al. [18] also find that the performance of ANNs is not as good as many other statistical methods commonly used in the load forecasting. Denton [19] with generated data for several different experimental conditions shows that under ideal conditions, with all regression assumptions, there is little difference in the predictability between ANNs and linear regression, and only under less ideal conditions such as outliers, multicollinearity, and model misspecification, ANNs perform better. Hann and Steurer [20] make comparisons between the neural networks and the linear model in exchange rate forecasting. They report that if monthly data are used, neural networks do not show much improvement over linear models. Taskaya and Casey [21] compare the performance of linear models with neural networks. Their results show that linear autoregressive models can outperform neural networks in some cases.

Most other researchers also make comparisons between ANNs and the corresponding traditional methods in their particular applications. De Groot and Wurtz [10] compare ANNs with the linear (Box-Jenkins) and nonlinear (bilinear and TAR) statistical models in forecasting the sunspots data. Fishwick [22] reports that the performance of ANNs is worse than that of the simple linear regression. Tang et al. [23], and Tang and Fishwick try to answer the question: under what conditions ANN forecasters can perform better than the linear time series forecasting methods such as Box–Jenkins models [24]. Some researchers believe that in some specific situations where ANNs perform worse than linear statistical models, the reason may simply be that the data is linear without much disturbance, therefore; cannot be expected that ANNs to do better than linear models for linear relationships [1]. However, for any reason, using ANNs to model linear problems have yielded mixed results and hence; it is not wise to apply ANNs blindly to any type of data.

In the literature, several linear approaches have been proposed to time series forecasting. Autoregressive integrated moving average (ARIMA) models are one of the most popular linear models for time series forecasting over the past three decades that have enjoyed useful applications in forecasting social, economic, engineering, foreign exchange, and stock problems. ARIMA models have been originated from the autoregressive models (AR), the moving average models (MA) and the combination of the AR and MA, the ARMA models. ARIMA models can be used when the time series is stationary and there is no missing data in the within the time series [25]. In ARIMA analysis, an identified underlying process is generated based on observations to a time series for generating a good model that shows the process-generating mechanism precisely.

Box and Jenkins [26] provided a step-by-step procedure for ARMA analysis, which is a combination of AR coefficients, which are multiplied by past values of the time series data and MA coefficients, which are multiplied by past random shocks. The popularity of the ARIMA model is due to its statistical properties as well as the well-known Box–Jenkins methodology in the model building process. In addition, ARIMA models [22] can implement various exponential smoothing models. Although ARIMA models are quite flexible in that they can represent several different types of time series, their major limitation is the pre-assumed linear form of the model. ARIMA models assume that future values of a time series have a linear relationship with current and past values as well as with white noise, so approximations by ARIMA models may not be adequate for complex nonlinear real-world problems. However, real world systems are often nonlinear [1], thus, it is unreasonable to assume that a particular realization of a given time series is generated by a linear process.

Both ANNs and ARIMA models have achieved successes in their own linear or nonlinear domains. However, none of them is a universal model that is suitable for all circumstances. The approximation of ARIMA models to complex nonlinear problems as well as ANNs to model linear problems may be totally inappropriate, and also, in problems that consist both linear and nonlinear correlation structures. Using hybrid models or combining several models has become a common practice in order to overcome the limitations of components models and improve the forecasting accuracy. In addition, since it is difficult to completely know the characteristics of the data in a real problem, hybrid methodology that has both linear and nonlinear modeling capabilities can be a good strategy for practical use.

The hybrid techniques that decompose a time series into its linear and nonlinear form are one of the most popular hybrid models categories, which have been shown to be successful for single models. Zhang [27] presented a hybrid ARIMA and ANN approaches for time series forecasting using mentioned technique. In Zhang's hybrid model is jointly used the linear ARIMA and the nonlinear multilayer perceptrons models in order to capture different forms of relationship in the time series data. The motivation of Zhang's hybrid model comes from the following perspectives. First, it is often difficult in practice to determine whether a time series under study is generated from a linear or nonlinear underlying process; thus, the problem of model selection can be eased by combining linear ARIMA and nonlinear ANN models. Second, real-world time series are rarely pure linear or nonlinear and often contain both linear and nonlinear patterns, which neither ARIMA nor ANN models alone can be adequate for modeling in such cases; hence the problem of modeling the combined linear and nonlinear autocorrelation structures in time series can be solved by combining linear ARIMA and nonlinear ANN models. Third, it is almost universally agreed in the forecasting literature that no single model is the best in every situation, due to the fact that a real-world problem is often complex in nature and any single model may not be able to capture different patterns equally well. Therefore, the chance in order to capture different patterns in the data can be increased by combining different models. These hybrid models, despite the all their advantages, have two assumptions [21] that will degenerate their performance if the opposite situation occurs; therefore, they may be inadequate in some specific situations.

In this paper, ARIMA models are applied to construct a new hybrid model in order to overcome the above-mentioned limitation of artificial neural networks and to yield more general and more accurate model than traditional hybrid ARIMA and artificial neural networks models. In our proposed model, a time series is considered as function of a linear and a nonlinear component, so, in the fist phase, an autoregressive integrated moving average model is first used in order to identify and magnify the existing linear structures in data. In the second phase, a multilayer perceptron is used as a nonlinear neural network in order to model the preprocessed data, which the existing linear structures are identified and magnified by ARIMA, and to predict the future value of time series in the future. Three well-known real data sets – the Wolf's sunspot data, the Canadian lynx data, and the British pound/US dollar exchange rate data – are used in this paper in order to show the appropriateness and effectiveness of the proposed model to time series forecasting. The rest of the paper is organized as follows. In the next section, the literature survey of the hybrid models is briefly reviewed. The basic concepts of autoregressive integrated moving average (ARIMA) and artificial neural networks (ANNs) are presented in section 3. In Section 4, the formulation of the proposed model is introduced. In Section 5, the proposed model is applied to time series forecasting and its performance is compared with those of other models. Section 6 contains the concluding remarks.

Section snippets

The hybrid models

In the literature, different combination techniques have been proposed in order to overcome the deficiencies of single models. The basic idea of the model combination in forecasting is to use each model's unique feature in order to capture different patterns in the data. The difference between these combination techniques can be described using terminology developed for the classification and neural network literature [28]. Hybrid models can be homogeneous, such as using differently configured

Time series forecasting models

There are several different approaches to time series forecasting, which are generally categorized as follow. Traditional statistical models including moving average, exponential smoothing, and autoregressive integrated moving average (ARIMA) are linear in that predictions of the future values are constrained to be linear functions of past observations. Second category of time series models are nonlinear models. In the literature, several classes of nonlinear models have been proposed to

Formulation of the proposed model

Despite the numerous time series models available, the accuracy of time series forecasting currently is fundamental to many decision processes, and hence, never research into ways of improving the effectiveness of forecasting models been given up. Many researches in time series forecasting have argued that predictive performance improves in combined models [21]. In hybrid models, the aim is to reduce the risk of using an inappropriate model by combining several models to reduce the risk of

Application of the hybrid model to exchange rate forecasting

In this section, three well-known data sets – the Wolf's sunspot data, the Canadian lynx data, and the British pound/US dollar exchange rate data – are used in order to demonstrate the appropriateness and effectiveness of the proposed model. These time series come from different areas and have different statistical characteristics. They have been widely studied in the statistical as well as the neural network literature [27]. Both linear and nonlinear models have been applied to these data

Conclusions

Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing decision makers in many areas. Despite the numerous time series models available, the research for improving the effectiveness of forecasting models has never stopped. Several large-scale forecasting competitions with a large number of commonly used time series forecasting models conclude that combining forecasts from more than one model often leads to improved performance,

Acknowledgements

The authors wish to express their gratitude to anonymous referees and Seyed Reza Hejazi, assistant professor of industrial engineering, Isfahan University of Technology, for their insightful and constructive comments, which helped to improve the paper greatly.

References (71)

  • V. Ediger et al.

    ARIMA forecasting of primary energy demand by fuel in Turkey

    Energy Policy

    (2007)
  • G.P. Zhang

    Time series forecasting using a hybrid ARIMA and neural network model

    Neurocomputing

    (2003)
  • F.M Tseng et al.

    Combining neural network model with seasonal time series ARIMA model

    Technological Forecasting & Social Change

    (2002)
  • S. Makridakis

    Why combining works?

    International Journal of Forecasting

    (1989)
  • D.K. Wedding et al.

    Time series forecasting by combining RBF networks certainty factors, and the Box–Jenkins model

    Neurocomputing

    (1996)
  • J.T. Luxhoj et al.

    A hybrid econometric-neural network modeling approach for sales forecasting

    International Journal of Production Economics

    (1996)
  • P.F. Pai et al.

    A hybrid ARIMA and support vector machines model in stock price forecasting

    Omega

    (2005)
  • K.Y. Chen et al.

    A hybrid SARIMA and support vector machines in forecasting the production values of the machinery industry in Taiwan

    Expert Systems with Applications

    (2007)
  • Z.J. Zhou et al.

    An effective hybrid approach based on grey and ARMA for forecasting gyro drift

    Chaos, Solitons and Fractals

    (2008)
  • G. Armano et al.

    A hybrid genetic-neural architecture for stock indexes forecasting

    Information Sciences

    (2005)
  • L. Yu et al.

    A novel nonlinear ensemble forecasting model incorporating GLAR and ANN for foreign exchange rates

    Computers and Operations Research

    (2005)
  • M. Khashei et al.

    Improvement of auto-regressive integrated moving average models using fuzzy logic and artificial neural networks (ANNs)

    Neurocomputing

    (2009)
  • H. Kim et al.

    A hybrid approach based on neural networks and genetic algorithms for detecting temporal patterns in stock markets

    Applied Soft Computing

    (2007)
  • M. Khashei et al.

    A new hybrid artificial neural networks and fuzzy regression model for time series forecasting

    Fuzzy Sets and Systems

    (2008)
  • C.-S. Ong et al.

    Model identification of ARIMA family using genetic algorithms

    Applied Mathematics and Computation

    (2005)
  • M. Haseyama et al.

    An ARMA order selection method with fuzzy reasoning

    Signal Processing

    (2001)
  • M. Hibon et al.

    To combine or not to combine: selecting among forecasts and their combinations

    International Journal of Forecasting

    (2005)
  • N. Terui et al.

    Combined forecasts from linear and nonlinear time series models

    International Journal of Forecasting

    (2002)
  • M. Ghiassi et al.

    A dynamic architecture for artificial neural networks

    Neurocomputing

    (2005)
  • L. Stone et al.

    Chaotic oscillations and cycles in multi-trophic ecological systems

    Journal of Theoretical Biology

    (2007)
  • Y. Tang et al.

    A consistent nonparametric Bayesian procedure for estimating autoregressive conditional densities

    Computational Statistics & Data Analysis

    (2007)
  • P. Cornillon et al.

    Forecasting time series using principal component analysis with respect to instrumental variables

    Computational Statistics & Data Analysis

    (2008)
  • R.A. Meese et al.

    Empirical exchange rate models of the seventies: do they /t out of samples?

    Journal of International Economics

    (1983)
  • A. Timmermann et al.

    Efficient market hypothesis and forecasting

    International Journal of Forecasting

    (2004)
  • D. Rumelhart et al.

    Parallel Distributed Processing

    (1986)
  • Cited by (0)

    View full text