|
Vol.15, No.2, May 2026. ISSN: 2217-8309 eISSN: 2217-8333
TEM Journal
TECHNOLOGY, EDUCATION, MANAGEMENT, INFORMATICS Association for Information Communication Technology Education and Science |
Evaluating Transformer-Based Foundation Models for Time-Series Forecasting Across Multiple Horizons
Miranda Harizaj, Alfons Harizaj, Olgerta Idrizi
© 2026 Miranda Harizaj, published by UIKTEN. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. (CC BY-NC-ND 4.0)
Citation Information: TEM Journal. Volume 15, Issue 2, Pages 1226-1245, ISSN 2217-8309, DOI: 10.18421/TEM152-24, May 2026.
Received: 09 October 2025.
Abstract:
Recent studies show that transformer-based architectures and foundation models have achieved promising results in time-series forecasting, yet their advantages over traditional machine-learning methods remain inconsistent across domains and forecasting horizons. This study investigates whether modern transformer-based models provide systematic performance gains compared to classical regression approaches and examines four representative models as SAMFormer, TimesFM, Time-MoE, and TimeGPT and benchmark them against established baselines including Linear Regression, MLPRegressor and ensemble methods. In the paper it is hypothesized that transformer-based models outperform traditional methods, particularly for longer forecasting horizons and datasets with strong temporal dependencies. An experimental evaluation is conducted across four real-world datasets from weather, finance, energy, and healthcare domains, using multiple context and prediction length settings. Model performance is assessed using standard error-based and distribution-based metrics. The results show that transformer-based models generally outperform regression baselines, with SAMFormer demonstrating the most stable and consistent performance across datasets and horizons. TimeGPT excels in short-term forecasting, while TimesFM exhibits limited robustness, especially for longer horizons. Fine-tuning yields mixed benefits, depending on the dataset and model architecture. Overall, the findings provide an evidence-based assessment of when transformer-based forecasting is advantageous and when simpler models remain competitive, offering practical guidance for model selection in real-world time-series applications.
Keywords – Time-series forecasting, transformer models, foundation models, fine-tuning, predictive analytics. |
|
----------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------- |