Author(s)

QianYi Huang

Affiliation(s)

Department of Mathmatics and Mechanics, Artificial Intelligence and Big Data Analysis, Novosibirsk 630090, Russia.

Corresponding Author

QianYi Huang

ABSTRACT

Stock market prediction necessitates effective multimodal data integration and robust uncertainty quantification. This paper proposes a novel Transformer-based architecture addressing two critical limitations of existing approaches: static cross-modal interaction and deterministic output assumptions. Our framework introduces (1) a multimodal subspace attention mechanism that projects numerical and textual features into orthogonal subspaces, enabling disentangled learning of modality-specific interactions through multiple attention heads, and (2) a dynamic gated recalibration module that adaptively adjusts modality contributions using time-variant weights. Evaluated on Technology Select Sector SPDR Fund (XLK) data with market sentiment feeds, the model achieves  higher directional accuracy than conventional Transformers while reducing volatility period prediction errors. The integrated uncertainty quantification module further provides statistically reliable confidence intervals, verified through backtesting.

KEYWORDS

Transformer networks; Multimodal fusion; Stock prediction; Uncertainty modeling; Dynamic attention subspaces

CITE THIS PAPER

QianYi Huang. A predictive model for stock prices based on transformer and utilizing multimodal data. Social Science and Management. 2025, 2(2): 49-56. DOI: https://doi.org/10.61784/ssm3050.

REFERENCES

[1] Chatterjee S. Comparative Study of Machine Learning Algorithms for Stock Market Prediction and Analysis of Correlation Between Nifty 50 and Global Indices. Global Journal of Business and Integral Security, 2024.

[2] Liu Q, Lin S, Zhu Y. Stock price prediction using long short-term memory and transformer. IEEE Transactions on Neural Networks and Learning Systems, 2021, 33(2): 459-470.

[3] Indu Kumar, Kiran Dogra, Chetna Utreja, et al. A Comparative Study of Supervised Machine Learning Algorithms for Stock Market Trend Prediction. In 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT), 2018. DOI: 10.1109/ICICCT.2018.8473214.

[4] Sampada R, Shivani J, Lasya V, et al. Stock Market Prediction Using Transformers. International Journal of Scientific Research in Science, Engineering and Technology (IJSRSET), 2023, 10(2): 502-505.

[5] Li X, Wang Y, Zhang J. A Transformer-based model for stock price prediction. In Proceedings of the 2019 IEEE International Conference on Big Data, 2019: 1450-1459.

[6] Nelson N, Kim T. A hybrid deep learning model for stock price prediction. In Proceedings of the 2017 IEEE International Conference on Big Data, 2017: 1189-1194.

[7] Deng Z, Yu X, Li Y. A deep reinforcement learning framework for the stock trading strategy. arXiv preprint arXiv:1606.01846, 2016.

[8] Chen X, Zhang Y, Zhang C. A graph neural network approach for stock price prediction. In Proceedings of the 2018 IEEE 4th International Conference on Big Data Computing Service and Applications, 2018: 1-8.

[9] Feng Y, Zhang Y, Zhang C. Generative adversarial networks for financial time-series data augmentation. In Proceedings of the 2019 IEEE International Conference on Big Data, 2019: 3208-3216.

[10] Qin Z, Zhang Y, Zhang C. A multi-modal deep learning approach for stock market prediction. In Proceedings of the 2021 IEEE International Conference on Big Data, 2021: 1820-1829.

[11] Xu Y, Zhang Y, Zhang C. An attention-based multi-modal fusion approach for stock market prediction. In Proceedings of the 2020 IEEE International Conference on Big Data, 2020: 272-281.

[12] Sawhney A, Singh A, Singh S, et al. Hyper-graph based multi-modal data fusion for stock market prediction. In Proceedings of the 2021 IEEE International Conference on Big Data, 2021: 1810-1819.

[13] Hu Y, Zhang Y, Zhang C. An early fusion approach for multi-modal data in stock market prediction. In Proceedings of the 2018 IEEE 4th International Conference on Big Data Computing Service and Applications, 2018: 1-7.

[14] Ding X, Zhang Y, Zhang C. A late fusion approach for multi-modal data in stock market prediction. In Proceedings of the 2015 IEEE International Conference on Big Data, 2015: 1297-1306.

[15] Wang Y, Zhang Y, Zhang C. A cross-modal memory network for multi-modal data fusion in stock market prediction. In Proceedings of the 2020 IEEE International Conference on Big Data, 2020: 282-291.

[16] Yang Y, Zhang Y, Zhang C. A modal adversarial training approach for multi-modal data in stock market prediction. In Proceedings of the 2022 IEEE International Conference on Big Data, 2022: 1-10.

[17] Kendall A, Gal Y. What uncertainties do we need in Bayesian deep learning for computer vision? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 5574-5584.

[18] Gal Y, Ghahramani Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning, 2016: 1050-1059.

[19] Sensoy M, Kaplan L, Kandemir M. Evidential deep learning to quantify classification uncertainty. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4712-4721.

[20] Bai Y, Zhang Y, Zhang C. A latent variable Transformer for uncertainty modeling in stock market prediction. In Proceedings of the 2021 IEEE International Conference on Big Data, 2021: 1830-1839.