AI-Driven Stock Trend Prediction with Market Regime Stratification: A Technical Indicator-Based Study on the KSE-100 Index
Contributors
- Naveen Shah Research Scholar, Department of Management Sciences, COMSATS University Islamabad, Abbottabad, Pakistan.
- Dr. Muhammad Naveed Jan Assistant Professor, Department of Management Sciences, COMSATS University Islamabad, Abbottabad, Pakistan.
- Dr. Muhammad Shariq Assistant Professor, School of Management Sciences, Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Pakistan.
Abstract
This research analyses the predictive capability of Long Short-Term Memory (LSTM) networks in comparison to Random Forest (RF) algorithms in forecasting the directional trends of the KSE-100 index at the Pakistan Stock Exchange (PSX). Utilizing an expansive longitudinal dataset of 15 years from 2010 to 2025, the research incorporates 33 technical indicators along with stratification/categorization of market conditions into bullish and bearish regimes to address the non-linearities of an emerging market. The models’ performance is evaluated using a multi-metric framework comprising of ROC-AUC, F1-score, and Root Mean Square (RMSE) with the statistical significance supported with paired t-test and bootstrapped confidence intervals. Empirical results show the invariable outperformance of Random Forest (RF) over the LSTM architecture with RF consistently showing exceptional classification accuracy and lower rates of error across all regimes. The feature importance approach demonstrates that momentum and trend-following models like the Return Strength Index (RSI) and Moving Average Convergence Divergence (MACD) play critical roles in determining the performance of the model. One contribution of this research is its demonstration of the regime-dependent predictability in turbulent markets through the examination of how algorithmic performance varies depending on different market regimes. The findings of this paper are valuable not only for institutions but also for policy makers because it evaluates the effectiveness of machine learning techniques in decision-making processes.
References
Amin, S., & Shahzad, M. (2023). Modeling volatility in the Islamic equity market: A case of Pakistan. Elementary Education Online, 22(1), 7325–7338. https://ilkogretim-online.org/index.php/pub/article/view/7325
Bayar, Y., Toader, V., Gavriletea, M. D., & Yelkesen, O. (2024). Exploring the Stock Market Development and Sustainable Development Nexus: Insights from emerging market economies. International Journal of Islamic and Middle Eastern Finance and Management, 17(6), 1213 1231. https://doi.org/10.1108/IMEFM-03-2024-0136
Cambria, E, & White, B. (2014). Jumping NLP curves. A review of Natural Language Processing research. IEEE Computational Intelligence Magazine, 9(2), 48-57. https://doi.org/10.1109/MCI.2014.2307227
Chikwira, C., & Mohammed, J. I. (2023). The impact of the stock market on liquidity and economic growth: Evidence of volatile market. Economies, 11(6), 155. https://doi.org/10.3390/economies11060155
Fama, E. F. (1970). Efficient Capital Markets: A review of theory and Empirical work. The Journal of Finance, 25(2), 383. https://doi.org/10.2307/2325486
Fischer, T., & Krauss, C. (2018). Deep learning with long short-term memory networks for financial market predictions. European Journal of Operational Research, 270(2), 654–669. https://doi.org/10.1016/j.ejor.2017.11.054
Henrique, B. M., Sobreiro, V. A., & Kimura, H. (2018). Stock price prediction using support vector regression on daily and up to the minute prices. The Journal of Finance and Data Science, 4(3), 183–201. https://doi.org/10.1016/j.jfds.2018.04.003
Jadoon, Mahmood, T., Sarwar, A., Maria Faiq Javaid, & Iqbal, M. (2024). Prediction of Stock Market Movement Using Long Short-Term Memory (LSTM) Artificial Neural Network: Analysis of KSE 100 Index. Pakistan Journal of Life and Social Sciences (PJLSS), 22(1). https://doi.org/10.57239/pjlss-2024-22.1.009
Naveed Jan, M., & Ayub, U. (2019). Do the fama and French five-factor model forecast well using Ann? Journal of Business Economics and Management, 20(1), 168-191. https://doi.org/10.3846/jbem.2019.8250
Javid, I., Ghazali, R., Syed, I., Zulqarnain, M., & Husaini, N. A. (2022). Study on the Pakistan stock market using a new stock crisis prediction method. PLoS ONE, 17(10), e0275022. https://doi.org/10.1371/journal.pone.0275022
Kulkarni, M. S., Vijayakumar Bharathi, S., Perdana, A., & Kilari, D. (2025). A Quest for Context Specific Stock Price Prediction: A Comparison Between Time Series, Machine Learning and Deep Learning Models. SN Computer Science, 6(4). https://doi.org/10.1007/s42979-025 03848-y
Lane, G. (1984). Lane’s stochastics. Technical Analysis of Stocks & Commodities, 3(2), 87–90.
Lau, T. (2024). Application of sentiment analysis to explore the connection between public sentiment and stock price movement by python. Advances in Economics, Management and Political Sciences, 102(1), 89-95. https://doi.org/10.54254/2754-1169/102/2024ed0081
Lo, A. W. (2004). The adaptive markets hypothesis: market efficiency from an evolutionary perspective. Journal of Portfolio Management. https://papers.ssrn.com/
Ma, Y., & Yan, X. (2022). Stock market prediction based on optimized CNN and multi-feature fusion. Expert Systems with Applications, 205, 117683. https://doi.org/10.1016/j.eswa.2022.117683.
Mabu, S., Obayashi, M., & Kuremoto, T. (2015). Ensemble learning of rule-based evolutionary algorithm using multi-layer perceptron for supporting decisions in stock trading problems. Applied Soft Computing, 36, 357-367. https://doi.org/10.1016/j.asoc.2015.07.020
Manogna, R. L., & Kulkarni, N. (2025). Portfolio Optimization Model for Stock Price Prediction Using Machine Learning. Journal of Statistical Theory and Applications. https://doi.org/10.1007/s44199-025-00140-z
Meher, B. K., Singh, M., Birau, R., & Anand, A. (2024). Forecasting stock prices of fintech companies of India using random forest with high-frequency data. Journal of Open Innovation: Technology, Market, and Complexity. https://doi.org/10.1016/j.joitmc.2023.100180
Murphy, J. J. (1999). Technical analysis of the financial markets: a comprehensive guide to trading methods applications. Choice Reviews Online, 36(07), 36–4016. https://doi.org/10.5860/choice.36-4016
Nabipour, M., Nayyeri, P., Jabani, H., Mosavi, A., Salwana, E., & Shahab, S. (2020). Deep learning for stock market prediction. Entropy, 22(8), 840. https://doi.org/10.3390/e22080840
Nikou, M., Mansourfar, G., & Bagherzadeh, J. (2019). Stock price prediction using deep learning algorithm and its comparison with machine learning algorithms. Intelligent Systems in Accounting, Finance and Management, 26(4), 164-174. https://doi.org/10.1002/isaf.1459
Patel, A., Patel, D., & Yadav, S. (2021). Prediction of stock market using artificial intelligence. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3871022
Phuoc, T., Anh, P. T. K., Tam, P. H., & Nguyen, C. V. (2024). Applying machine learning algorithms to predict the stock price trend in the stock market– The case of Vietnam. Humanities and Social Sciences Communications, 11(1). https://doi.org/10.1057/s41599-024 02807-x
Pilla, P., & Mekonen, R. (2025). Forecasting S&P 500 using LSTM models [Preprint]. arXiv. https://arxiv.org/abs/2501.17366
Provost, F., & Fawcett, T. (2013). Data science and its relationship to big Data and data driven decision making. Big Data, 1(1), 51–59. https://doi.org/10.1089/big.2013.1508
Raza, H., & Akhtar, N. Z. (2024). Predicting stock prices in the Pakistan market using machine learning and technical indicators. Modern Finance, 2(2), 46–63. https://doi.org/10.61351/mf.v2i2.167
Sadorsky, P. (2021). A random forests approach to predicting clean energy stock prices. Journal of Risk and Financial Management, 14(2), 48. https://doi.org/10.3390/jrfm14020048
Selvin, S., Gopalakrishnan, E. A., Ravi, V., & Menon, V. K. (2017). Stock market prediction using LSTM, RNN and CNN-sliding window model [Conference paper]. https://doi.org/10.1109/ICACCI.2017.8126078
Shen, J., & Shafiq, M. O. (2020). Short-term stock market price trend prediction using a comprehensive deep learning system. Journal of Big Data, 7(1). https://doi.org/10.1186/s40537-020-00333-6
Siami-Namini, S., & Namin, A. S. (2018). Forecasting Economics and Financial Time Series: ARIMA vs. LSTM. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1803.06386
Sizan, M. M. H., Das, B. C., Shawon, R. E. R., Rana, MD. S., Al Montaser, M. A., Chouksey, A., & Pant, L. (2023). AI-enhanced stock market prediction: evaluating machine learning models for financial forecasting in the USA. Journal of Business and Management Studies. https://doi.org/10.32996/jbms
Sonkavde, G., Dharrao, D., Bongale, A., Deokate, S., Doreswamy, D., & Bhat, S. (2023). Forecasting stock market prices using machine learning and deep learning models: a systematic review, performance analysis and discussion of implications. International Journal of Financial Studies, 11(3), 94. https://doi.org/10.3390/ijfs11030094
Wang, X. (2024). Stock price prediction: a comparative study of random forest and lstm models. Highlights in Science, Engineering and Technology, 107, 117-123. https://doi.org/10.54097/70a8b947
Widiputra, H. and Juwono, E. (2024). Parallel multivariate deep learning models for time-series prediction: a comparative analysis in asian stock markets. IAES International Journal of Artificial Intelligence (IJ-AI), 13(1), 475. https://doi.org/10.11591/ijai.v13.i1.pp475-486
Wilder, J. W. (1978). New concepts in technical trading systems. https://agris.fao.org/agris search/search.do?recordID=US201300554903
Wu, H. (2024). Comparison of random forest and lstm in stock prediction. Advances in Economics, Management and Political Sciences, 86(1), 28-33. https://doi.org/10.54254/2754-1169/86/20240936
Zahid, S., & Saleem, H. M. N. (2025). A comparison of machine learning algorithms to predict stock price during COVID-19 outbreaks: An empirical investigation of pakistan stock exchange. Global Business Review. https://doi.org/10.1177/09721509251316121
Zhang, Z. (2023). Comparison of lstm and arima in price forecasting: evidence from five indexes. Advances in Economics, Business and Management Research, 40-46. https://doi.org/10.2991/978-94-6463-268-2_6
Zhou, H., Xu, K., Bao, Q., Lou, Y., & Qian, W. (2024). Application of conversational intelligent reporting system based on artificial intelligence and large language models. JTPES, 4(03), 176-182. https://doi.org/10.53369/jtpes.2024.04(03).16
Corresponding Author
Dr. Muhammad Naveed Jan
Assistant Professor, Department of Management Sciences, COMSATS University Islamabad, Abbottabad, Pakistan.
Corresponding Author: [email protected]
