Time Series Forecasting Using Linear Models:

Abstract

Transformer models, renowned for their exceptional performance in natural language pro- cessing and computer vision, encounter challenges when applied to time series forecasting. This is attributed to the permutation invariance of their self-attention mechanism, which hinders their ability to capture temporal dependencies effectively. Consequently, researchers have explored adapting simpler models, such as those based on convolutional neural networks (CNNs), recurrent neural networks (RNNs), and multi- layer perceptrons (MLPs), to time series forecasting. MLP-based models, in particular, have demonstrated success in capturing moving averages and seasonal patterns in data. However, they often struggle to accurately forecast trends and sudden changes, limiting their overall forecasting performance. To address this limitation, we propose a novel Univariate Multi-Scale Linear (UMS- Linear) model that leverages timestamps, multi-scale decomposition, and a new loss func- tion to enhance the forecasting ability of MLP-based models. UMS-Linear decomposes the input time series into multiple scales, capturing intricate dynamics and patterns that may be missed by traditional MLP models. By incorporating timestamps, UMS-Linear explicitlymodels the temporal relationships within the data, further improving forecasting accuracy. Empirical results across multiple benchmark datasets demonstrate that UMS-Linear outperforms existing methods, including transformer-based models, CNNs, RNNs, and MLPs. This superior performance highlights the effectiveness of our approach in captur- ing the complex dynamics of time series data, paving the way for improved forecasting accuracy in various applications. important field.