A hybrid GARCH, convolutional neural network and long short term memory methods for volatility prediction in stock market

Abstract

Forecasting the volatility of the stock market price is indispensable for managing the risks associated with market dynamics and provides valuable insights for financial decision in trading strategies. This study aims to enhance the accuracy of volatility prediction for stock market price using hybrid models combining econometric and deep learning approaches. Specifically, it introduces a novel GARCH-CNN-LSTM hybrid model for more precise volatility forecasting of stock market price. The GARCH model is efficient at capturing volatility clusters and kurtosis features, while the CNN excels in extracting spatial patterns from time series data, and LSTM effectively preserves essential information over extended periods. GARCH(1,1) model is selected based on AIC, maximum log-likelihood, and parameter significance. Subsequently, CNN and LSTM models are chosen for their complementary capabilities in volatility prediction. We evaluated the forecasting performance of the hybrid models from out-sample test data, employing Mean Square Error, Root Mean Square Error and Mean Absolute Error. The result indicates that the new model outperforms the existing models with an improvement of 8% to 13% accuracy. Furthermore, we conduct the Diebold-Mariano test to confirm significant differences in performance.