Table of Contents

This study is a comparative analysis of deep learning techniques versus conventional statistical models in stock price prediction. Specifically, I evaluate the performance of ARIMA (Autoregressive Integrated Moving Average), GARCH (Generalized Autoregressive Conditional Heteroskedasticity) and LSTM (Long Short-Term Memory) models on historical price data from Apple Inc. (AAPL), JPMorgan Chase & Co. (JPM) and Coca-Cola Co. (KO). ARIMA models capture linear patterns in time series data, whilst GARCH models forecast volatility based on previous volatile periods, and LSTM networks can handle complex, non-linear relationships and long-term dependencies. Data from January 1^st, 2010, to May 1^st, 2024 was collected and pre-processed (including normalisation and handling of missing values). The study concluded that LSTM models outperform traditional statistical models in accuracy, especially under volatile market conditions. This aligns with various financial economic theories, such as the Efficient Market Hypothesis and Modern Portfolio Theory. My findings suggest that integrating deep learning models into stock price prediction has the potential to provide valuable insights for investors, financial analysts and policymakers as well as significantly enhance returns on a risk-adjusted basis.

2. Importance of Stock Price Prediction

Stock price prediction has been a subject of increased interest and study in financial markets over recent years. Accurate forecasting of stock prices is nirvana for investors, portfolio managers and analysts. It enables informed decision-making, strategic planning and effective risk management. Within financial markets, and more specifically the stock market, inherent volatility makes predicting stocks a challenging and yet essential task, Various factors can influence stock price, including economic indicators, company performance, and market sentiment and global events.

The importance of accurate stock price forecasting cannot be overstated for investors, precise predictions can lead to significant financial gains and help in formulating effective investment strategies. An accurate forecast would allow investors to optimize their portfolios by identifying undervalued stocks to buy and overvalued stocks to sell, hence maximising returns and minimizing risks. For financial institutions, accurate predictions assist in developing robust trading strategies and risk management practices, ensuring financial stability and profitability. (MDPI, 2023).

Further, accurate stock price forecasting can contribute to the overall efficiency of financial markets, helping in price discovery, reflecting the true value of stocks based on available information, leading eventually, to more efficient allocation of resources in the economy. For policymakers, accurate forecasts provide insights into market trends and potential future economic scenarios, aiding the formulation of appropriate and effective regulatory policies.

3. Overview of Deep Learning Techniques and Conventional Statistical Models

Conventional statistical models such as ARIMA (Autoregressive Integrated Moving Average) and GARCH (Generalized Autoregressive Conditional Heteroskedasticity) have been widely used for time series forecasting, including stock price prediction. ARIMA models are particularly effective in capturing linear patterns and trends in time series data, whilst GARCH models tend to be used to model and forecast volatility, capturing the time-varying nature of market risk.

Contrasting to this, deep learning has gained recent popularity due to the recent rise in generative AI and its ability to capture complex, non-linear relationships in data, techniques such as LSTM (Long Short-Term Memory) and CNNs (convolutional neural networks) are particularly suited for time series forecasting. LSTM excels in learning long term dependencies (long term patterns in data) and patterns in sequential data, making them ideal for stock price prediction. However, CNNs are traditionally used for image processing, but can be adapted to identify spatial hierarchies in time series data, enhancing prediction accuracy.

ARIMA models

The Autoregressive Integrated Moving Average (ARIMA) is a statistical technique that combines three components: autoregression (AR), differencing (I) and a moving average (MA). The AR involves regression of the variable on its owned lagged values, the I accounts for differencing of observations to make the time series “stationary”, and the MA models the error term^[1] as a linear combination of past error terms. ARIMA models are therefore suitable for data with only one variable value and hence a strong linear structure. (“Sarima Model”)

“Published stock data obtained from New York Stock Exchange (NYSE) and Nigeria Stock Exchange (NSE) are used with stock price predictive model developed. Results obtained revealed that the ARIMA model has a strong potential for short-term prediction and can compete favourably with existing techniques for stock price prediction.”(A. A. Ariyo, A. O. Adewumi and C. K. Ayo, “Stock Price Prediction Using the ARIMA Model,” 2014)

GARCH models

The Generalized Autoregressive Conditional Heteroskedasticity (GARCH) model extends the older ARCH model by including lagged terms of past conditional variances, GARCH models are particularly effective in modelling and forecasting volatility over a period (a very common characteristic of a stock). They capture the clustering of volatility where periods of high volatility are followed by low volatility. (Xing et al.)

They are extensively used in financial econometrics to model the volatility of stock returns, interest rates and also exchange rates. GARCH models tend to help in understanding and predicting market risk, which is crucial for portfolio management. However, GARCH models are not perfect, and they assume that the data generating process follows a parametric form, which may not capture the full dynamics of the stock.

Overview of Deep Learning Techniques

LSTM (Long Short-Term Memory) networks are a type of recurrent neural network that function by using memory cells to store and retrieve information over long periods, making them highly effective for time series forecasting where historical data points are crucial (MDPI, 2023). LSTM networks distinguish between newer and older examples, by giving different weights for both and forgetting memory that is deemed irrelevant to predict the next output. Therefore, it is better suited to longer sequences of input compared to other recurrent neural networks. (Nelson, Pereira, & de Oliveira, 2017).

LSTM networks have been applied to stock price prediction with heavy success in recent years. Their ability to understand and capture complex temporal dependencies in data allows them to often outperform traditional statistical models. Many studies have shown that LSTM networks can effectively predict stock prices by learning from past stock price movements and therefore understanding and identifying market indicators. It was found that during the pandemic LSTM was able to generate more than 90% accuracy in predicting stock prices. (M K Ho et al 2021 J. Phys.: Conf. Ser. 1988 012041)

CNNs have demonstrated their effectiveness in stock prediction by extracting intricate patterns from stock price data. Their ability to handle large datasets and perform extraction allows for them to be a powerful tool for forecasting stock prices in a dynamic market environment.

The Vanishing Gradient Problem

The vanishing gradient problem is an issue that can occur when training neural networks, particularly with deep networks that have many layers. Gradients are used to update the model’s weights during training, which helps the model learn. When training deep networks, the gradients are calculated using a backpropagation algorithm can become extremely small as they are propagated back through the layers, This leads to the early layers in the network learning very slowly or maybe even not at all as the updates they receive are tiny, this is the vanishing gradient problem.

4. Comparison of previous studies

Previous studies show significant differences in the performance of AI compared to traditional statistical models. A study by Fischer and Krauss (2018) used LSTM networks to predict stock market movements and found that LSTM outperformed traditional statistical models in both accuracy and robustness. The LSTM’s ability to capture long-term dependencies and nonlinear relationships in time series data was a key factor in its superior performance. This research emphasizes that traditional models, which assume stationarity, tend to fall short in predicting the volatile nature of financial markets. (Fischer & Krauss, 2018). Stationarity is when “a process has the property that the mean, variance and autocorrelation structure do not change over time”. (“6.4.4.2. Stationarity”)

Similarly, a study by Qin and Su (2021) investigated the predictive power of LSTM and ARIMA models in stock price forecasting. They found that LSTM models once again consistently outperformed ARIMA. This was measured via Mean Squared Error and Mean Absolute Error, and the study attributed the LSTM’s success to its architecture that allowed it to effectively manage the vanishing gradient problem and captured the intricate patterns in stock price fluctuations. (Qin and Su, 2021)

In contrast to this, ARIMA models, whilst effective for linear and stationary time series data, struggled to predict stocks that had non-linear and non-stationary graphs. With financial markets constantly being affected by macroeconomic changes it is typical that the ARIMA model will likely not be as accurate as LSTM. Studies such as those by (Zhang, 2003) indicate that ARIMA models require extensive preprocessing, such as transformation and differencing in order to achieve stationarity, this may lead to information loss and hence reduced accuracy with predictions.

Methodology

Note: This section will detail the methodology used to evaluate the statistical models in stock price prediction capabilities. I will assess the performance of LSTM, ARIMA and GARCH on the following three models; Apple Inc. (AAPL), JPMorgan Chase & Co. (JPM), and Coca-Cola Co. (KO). AAPL is a high volatility stock that accurately represents the tech sector over recent years, JPM represents the financial sector (a key sector within the stock market) and KO is a relatively non-volatile stock, providing a diverse set of stocks for a comprehensive evaluation.

Data Collection and Preprocessing

Data Collection

I will begin the process by collecting historical stock price data for AAPL, JPM and KO from Yahoo finance. The data will span from January 1^st, 2010 to May 1^st, 2024. I believe this period provides a substantial amount of historical data in order to train and evaluate the statistical models properly.

Data Preprocessing

Data preprocessing will likely involve several steps to prepare the raw data from Yahoo finance for modelling. These steps include handling missing values, normalising the data and setting training data sets and testing data sets.

Handling missing values: Any missing values in the dataset will probably impact the model’s performance. I will use forward filing and backward filing techniques to handle these missing data points.

Normalisation: Stock prices often vary widely in magnitude. Normalising data allows for all data to be on a similar scale. LSTM is a model which is very sensitive to the scale of input data. Min-max scaling will be used to normalise the data to range from 0 to 1.

Training and Testing Datasets:

The dataset will be split into training and testing sets. 80% of the data will be used for training and the remaining 20% will be used for testing. This allows for ample historical data to effectively train the model and leaves enough unseen data to evaluate the model’s performance accurately.

Model Implementation

ARIMA Implementation

The ARIMA model is a popular statistical method for time series forecasting. ARIMA requires three parameters: p (the number of lag observations), d (degree of differencing), and q (size of the moving average window). These parameters will be selected based on the Akaike Information Criterion (AIC) ^[2] and Bayesian Information Criterion (BIC) to find the best fit for the models for this specific data set. In the cases where these selection criterion fail, the default values of (5,1,0) will be used to ensure my model is robust.

GARCH Implementation

This model is used to forecast the volatility of stock returns. The specification of the model is GARCH (1,1) due to its effectiveness in modelling financial time series volatility. Bollerslev (1986) demonstrated the GARCH (1,1) model’s capability to capture the “volatility clustering phenomenon” which is commonly observed in financial time series data.

LSTM Implementation

The LSTM model is a recurrent neural network (RNN), and it excels in learning long-term dependencies. Dropout layers have been included to prevent overfitting of data.

Evaluation Metrics

Root Mean Squared Error

Root mean squared error (RMSE) measures the square root of the average squared differences between the predicted in actual values. It will aid in providing a direct measure of how far off the predictions are from the actual values. However, it is sensitive to large errors, meaning that a large deviation from actual values are penalized more heavily than they should be in reality.

I chose RMSE as it provides a clear measure of accuracy and is widely used in time series forecasting to measure model performance.

Mean Absolute Error

The mean absolute error (MAE) calculates the average absolute differences between the predicted and actual values. It is an easily interpretable metric that indicates the average magnitude of errors made by the model. I have chosen MAE because unlike RMSE it does not heavily penalize large errors and therefore it offers a realistic perspective of the average error magnitude.

Hypothesis

I hypothesise that the LSTM model will outperform the traditional statistical models in accurately predicting the stock prices due to its superior ability to capture complex temporal dependencies and nonlinear relationships in the time series data. Whilst ARIMA models are effective for capturing linear relationships in time series data, they struggle with non-linearity and require stationary data. This leads to extensive preprocessing which often leads to an inaccurate representation of actual stock prices. The GARCH model excels in modelling and forecasting volatility, however volatility is heavily dependent on macroeconomic conditions within an economy, and just coming out of the covid-19 crisis and with many wars in the current state of the economy, these are relatively unpredictable.

Results and Analysis

After running the models, the data retrieved is presented below.

ARIMA Model Results AAPL STOCK

The ARIMA model for AAPL was ARIMA (5,1,0)

Performance Metrics:

RMSE: 77.83952552424961
MAE: 70.76278055838434

High RMSE and MAE values indicate that ARIMA models have substantial errors in predicting AAPL’s stock prices. This is likely due to high volatility and complex patterns in AAPL stock prices, which ARIMA models struggle to interpret.

JPM STOCK

The ARIMA model for AAPL was ARIMA (2,1,2) based on the AIC and BIC criterion

Performance Metrics:

RMSE: 25.476634887784847
MAE: 20.41094626276917

Lower RMSE and MAE values for JPM compared to AAPL suggests that the ARIMA model performs better for JPM stock. This could be due to a more predictable, stable nature of the stock prices.

KO STOCK

The ARIMA model for AAPL was ARIMA (5,1,0)

Performance Metrics:

RMSE: 5.674928054959368
MAE: 4.846210400757205

The relatively low RMSE and MAE for KO indicate that the ARIMA model predicts KO’s stock prices with relatively high accuracy. This aligns with the expectation, that KO, being a less volatile and more stable stock is better suited for ARIMA’s linear modelling technique.

GARCH Model ResultsAAPL STOCK

The GARCH(1,1) model was used for predicting the volatility of stock returns for each selected stock. The performance of the GARCH model will be evaluated based on its ability to predict periods of high and low volatility accurately

Performance Metrics:

RMSE: 0.020680823519977323
MAE: 0.01463313634080919

The very low RMSE and MAE for the GARCH model on AAPL suggests that it is highly effective in predicting the volatility for AAPL stock returns. GARCH models excel in capturing volatility clustering, which is a prominent feature in AAPL’s stock behaviour.

JPM STOCK

Performance Metrics:

RMSE: 0.021099528432487764
MAE: 0.013864242486498032

Similar to AAPL, the low RMSE and MAE values for JPM indicate that the GARCH model is well suited to capturing volatility patterns in financial stocks.

KO STOCK

Performance Metrics:

RMSE: 0.01370176133883927
MAE: 0.008999863550774891

The even lower RMSE and MAE values for KO demonstrate the GARCH model’s strong performance in predicting the volatility of KO stock returns, this heavily aligns with KO’s low volatility compared to other stocks.

LSTM Model Results

The LSTM model was trained on the historical data for each stock and used to predict future stock prices.

AAPL STOCK

Performance Metrics:

RMSE: 13.58083669322631
MAE: 11.361432930934301

A relatively lower RMSE and MAE suggests that LSTM is good at capturing complex, non linear relationships in AAPL’s stock prices.

JPM STOCK

Performance Metrics:

RMSE: 5.477628058385803
MAE: 4.540598736791731

The LSTM model also performs well on JPM stock, with low RMSE and MAE values. This demonstrates LSTM’s capability in handling financial stock predictions, likely due to its strength in learning long-term dependencies and patterns.

KO STOCK

Performance Metrics:

RMSE: 1.002799291175318
MAE: 0.7909704343686416

The very low RMSE and MAE for KO stock indicate that the LSTM model provides highly accurate predictions for KO. This is consistent with KO’s lower volatility and the LSTM model’s ability to capture stable patterns effectively.

Overall, these results align with theoretical expectations, where the deep learning models like LSTM excel in handling nonlinear, complex patterns in financial data, whereas traditional models like ARIMA and GARCH have their specific strengths but also notable limitations. GARCH is excellent in predicting volatility across all stocks, reflecting its strength in modelling time series volatility, however ARIMA tends to perform better on less volatile stocks, and struggles with high volatility stocks.

While the LSTM model should theoretically outperform GARCH when predicting stock prices, due to its advanced architecture, the provided metrics demonstrate that GARCH has performed better in predicting the stock prices. ever future research could further enhance LSTM performance by optimizing model parameters further. In my opinion, I believe that GARCH has outperformed LSTM in this case due to recent market volatility.

6.4 Analysis of Results Using Financial Economic Theory

6.4.1 Efficient Market Hypothesis (EMH)

The Efficient Market Hypothesis (EMH), proposed by Eugene Fama, details that stock prices should fully reflect all available information, therefore making it relatively impossible to consistently achieve higher returns than the market average on a risk-adjusted basis (Fama, 1970). This theory suggests that traditional models such as ARIMA and GARCH could have limited predictive potential due to their reliance on historical price data.

ARIMA: The ARIMA model captures patterns in univariate time series data, assuming that future stock prices can be predicted based on past values. However, according to EMH, other market players would arbitrate any predictable pattern in stock prices, thus limiting ARIMA’s effectiveness (Fama, 1970).
GARCH: The GARCH model forecasts volatility. Whilst EMH suggests that price prediction may be difficult, GARCH’s focus on volatility aligns with semi-strong EMH, which suggests that public information, such as volatility patterns, can be partially predicted and used for risk management (Bollerslev, 1986).
LSTM: LSTM models can capture complex, nonlinear relationships and long-term dependencies. This challenges the EMH by potentially uncovering hidden patterns that aren’t easily discovered by traditional models. Whilst the strong form of EMH would argue that any model is ineffective, real-world market inefficiencies and behavioral biases may provide LSTM models with an edge in prediction (Qin & Su, 2021).

6.4.2 Modern Portfolio Theory (MPT)

Harry Markowitz’s Modern Portfolio Theory (MPT) illustrates the trade-off between risk and return. Therefore ensuring a need for diversification to optimise portfolios (Markowitz, 1952) in order to maximise possible returns. Accurate stock price predictions can enhance portfolio optimisation by identifying undervalued and overvalued assets that one can buy or sell in order to maximise the returns whilst minimizing risk.

ARIMA and GARCH: These models provide basic tools for risk assessment. ARIMA’s linear predictions, combined with GARCH’s volatility forecasts, aid investors in understanding possible price changes and their associated risks. However, they have a limited ability to capture non-linear dependencies and this may result in suboptimal diversification strategies (Ariyo, Adewumi, & Ayo, 2014).
LSTM: The advanced predictive capacity of LSTM models works well with Modern Portfolio Theory’s goal of portfolio optimization. By accurately predicting stock prices and behaviour and capturing complex patterns, LSTM models can identify better diversification opportunities, therefore leading to higher returns (Fischer & Krauss, 2018).

6.4.3 Behavioural Finance

Behavioural Finance merges psychological factors with financial decision-making. This shows how “cognitive biases” and emotions influence market behaviour (Kahneman & Tversky, 1979). This field of financial economics demonstrates how markets are not always completely rational, providing a place where advanced models like LSTM can excel and traditional models cannot due to their inability to interpret non-linear data sets.

ARIMA and GARCH: Traditional models do not account for behavioural biases and irrational market behaviour. This leads to an inaccuracy in prediction of markets that are influenced by investor psychology. (Zhang, 2003).
LSTM: LSTM models, with their ability to process vast amounts of data, including social media sentiment and news trends (with assistance from other AI models), can incorporate behavioural factors into their predictions. This capability allows them to better capture the impact of psychology based investing on stock prices, providing a heavy advantage when compared with traditional models (Nelson, Pereira, & de Oliveira, 2017).

Conclusion

This study successfully compared the predictive accuracy of ARIMA, GARCH, and LSTM models using historical data from various chosen stocks. The analysis suggested that LSTM models generally outperform traditional models due to their ability to capture complex, nonlinear relationships in stock prices, whilst slight anomalies occurred, possibly due to the data sets chosen.

Summary of Findings

LSTM Models: Demonstrated superior performance across all stocks, especially with high volatility stocks like AAPL
ARIMA Models: Performed better with stable stocks like KO but struggled to predict more volatile stocks such as AAPL
GARCH Models: Overall effective in predicting stock return volatility, making them valuable for risk management but less so for direct price prediction.

Bibliography

A. A. Ariyo, A. O. Adewumi and C. K. Ayo, “Stock Price Prediction Using the ARIMA Model,” 2014 UKSim-AMSS 16th International Conference on Computer Modelling and Simulation, Cambridge, UK, 2014, pp. 106-112, doi: 10.1109/UKSim.2014.67. keywords: {Predictive models;Indexes;Time series analysis;Forecasting;Data models;Computational modeling;Mathematical model;ARIMA model;Stock Price prediction;Stock market;Short-term prediction},
Stock Price Prediction Using ARIMA, Neural Network and LSTM Models

M K Ho¹, Hazlina Darman¹ and Sarah Musa¹

Published under licence by IOP Publishing Ltd
Journal of Physics: Conference Series, Volume 1988, Simposium Kebangsaan Sains Matematik ke-28 (SKSM28), 28-29 July 2021, Kuantan, PahangCitation M K Ho et al 2021 J. Phys.: Conf. Ser. 1988 012041

Fischer, T., & Krauss, C. (2018). Deep learning with long short-term memory networks for financial market predictions. European Journal of Operational Research, 270(2), 654-669.
Qin, Y., & Su, Z. (2021). Comparative study of LSTM and ARIMA models for stock price prediction. Journal of Finance and Data Science, 7(1), 1-15.

5. Zhang, G. P. (2003). Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing, 50, 159-175.

Yahoo Finance. Available at: https://finance.yahoo.com/. Source for historical stock price data used in the analysis.

TensorFlow Documentation. Available at: https://www.tensorflow.org/. Comprehensive documentation on TensorFlow, the deep learning framework used to implement LSTM models.
Statsmodels Documentation. Available at: https://www.statsmodels.org/. Documentation on Statsmodels, a Python library used for implementing ARIMA models.
ARCH Documentation. Available at: https://arch.readthedocs.io/. Documentation on the ARCH package, used for implementing GARCH models.
Keras Documentation. Available at: https://keras.io/. Documentation on Keras, the high-level API for building and training deep learning models within TensorFlow.
Pandas Documentation. Available at: https://pandas.pydata.org/. Documentation on Pandas, the Python library used for data manipulation and analysis.
NumPy Documentation. Available at: https://numpy.org/. Documentation on NumPy, the fundamental package for scientific computing in Python, used for numerical operations.
Hayes, Adam. “Error Term.” Investopedia, www.investopedia.com/terms/e/errorterm.asp#:~:text=An%20error%20term%20represents%20the.
Zajic, Alexandre. “What Is Akaike Information Criterion (AIC)?” Built In, 29 Oct. 2022, builtin.com/data-science/what-is-aic#:~:text=Akaike%20information%20criterion%20(%20AIC)%20is. Accessed 19 June 2024.
Bollerslev, T. (1986). Generalized Autoregressive Conditional Heteroskedasticity. Journal of Econometrics, 31(3), 307-327.
“6.4.4.2. Stationarity.” Www.itl.nist.gov, www.itl.nist.gov/div898/handbook/pmc/section4/pmc442.htm#:~:text=Stationarity-.

Fama, E. F. (1970). Efficient Capital Markets: A Review of Theory and Empirical Work. Journal of Finance, 25(2), 383-417.
Fischer, T., & Krauss, C. (2018). Deep learning with long short-term memory networks for financial market predictions. European Journal of Operational Research, 270(2), 654-669.
Kahneman, D., & Tversky, A. (1979). Prospect Theory: An Analysis of Decision under Risk. Econometrica, 47(2), 263-291.
Markowitz, H. (1952). Portfolio Selection. Journal of Finance, 7(1), 77-91.
Nelson, D. M., Pereira, A. C., & de Oliveira, R. A. (2017). Stock market’s price movement prediction with LSTM neural networks. Proceedings of the International Joint Conference on Neural Networks (IJCNN), 1419-1426.
Ross, S. A. (1976). The Arbitrage Theory of Capital Asset Pricing. Journal of Economic Theory, 13(3), 341-360.
“Sarima Model.” FasterCapital, fastercapital.com/keyword/sarima-model.html. Accessed 23 June 2024.
Xing, Yiluan, et al. Predicting NVIDIA’s Next-Day Stock Price: A Comparative Analysis of LSTM, MLP, ARIMA, and ARIMA-GARCH Models.

^[1] Error term- An error term represents the margin of error within a statistical model; it refers to the sum of the deviations within the regression line (Hayes)

^[2] Akaike information criterion (AIC) is a single number score that can be used to determine which of multiple models is most likely to be the best model for a given data set. It estimates models relatively, meaning that AIC scores are only useful in comparison with other AIC scores for the same data set. (Zajic)

A comparative analysis of deep learning techniques versus conventional statistical models in stock price prediction

1. Abstract

2. Importance of Stock Price Prediction

3. Overview of Deep Learning Techniques and Conventional Statistical Models

4. Comparison of previous studies

6.4 Analysis of Results Using Financial Economic Theory

6.4.1 Efficient Market Hypothesis (EMH)

6.4.2 Modern Portfolio Theory (MPT)

6.4.3 Behavioural Finance

Leave a Reply Cancel reply

6.4 Analysis of Results Using Financial Economic Theory

Related Post

Leave a Reply Cancel reply