Stock Prediction Based on Transformer Model

Abstract

Stock price forecasting is a critical tool for investors and traders to manage risks and make informed financial decisions in volatile markets. Traditional models often struggle with the non-linear and unpredictable nature of stock prices, leading to the adoption of advanced deep-learning techniques. In this study, we propose a LASSO-regularized Transformer model for stock price prediction, focusing on the encoder component to capture long-term dependencies in historical stock data. We evaluate the model's performance using MAPE across five Nepalese commercial banks and analyze the impact of different look-back periods on prediction accuracy. Our results show that the LASSO-Transformer achieves low MAPE values, with the best performance observed for shorter look-back periods (2-5 days). The model demonstrates robustness across different stocks, with the lowest MAPE of 1.4140% for SCB. This study highlights the effectiveness of combining feature selection (LASSO) with self-attention mechanisms (Transformer) for accurate stock price forecasting, offering practical insights for future applications in financial markets.