Adapting Random Simple Recurrent Network for Online Forecasting Problems.

Abstract

Random Simple Recurrent Network (RSRN) is a forecasting model based on the Random Neural Network (RaNN) and Recurrent Neural Network (RNN). RSRN has demonstrated energy-efficient and effective forecasting capabilities in offline mode, making it suitable for various applications. However, offline training faces challenges, such as limited storage capacity, computational power, and evolving datasets. To address these limitations, this paper introduces an online learning approach to the RSRN model. We present adaptations of two online learning algorithms, Projected Online Gradient Descent (POGD) and Follow-The-Proximally-Regularized-Leader (FTRL-Proximal), for training RSRN in real-Time. POGD leverages Back Propagation Through Time (BPTT) for handling dependencies with a sliding window, while FTRL-Proximal offers a balance between adaptability and stability, especially for sparse data. Our approach is the first to introduce RSRN's forecasting capabilities in a dynamic environment, demonstrating its potential in real-world applications where data availability is not guaranteed. The effectiveness of the online RSRN with both approaches is demonstrated through experimental results on benchmark datasets, showcasing competitive performance that surpasses offline mode computation and result.