Comparative Analysis of Online and Offline Learning Algorithms with Data Drift Detectors in Multi-Target Time Series

Abstract

In machine learning, addressing data drift is crucial due to its profound impact on model accuracy over time. This study investigates the performance of online and offline learning algorithms, alongside black-box and white-box models, integrated with data drift detectors, focusing on multitarget time series problems using real-world datasets in an online setting. We systematically compare the efficacy of these algorithms within a unified experimental framework, evaluating their ability to manage data drift while sustaining predictive accuracy. Our findings reveal that offline algorithms generally outperform their online counterparts, albeit at the expense of higher computational costs when implemented in an online environment. Notably, among the algorithms examined, a single-stack Random Forest model demonstrates superior performance even without explicitly considering correlations between targets. Additionally, black-box models consistently outperform white-box models. For data drift detection, the Kolmogorov-Smirnov Windowing detector emerges as the most effective method. Furthermore, we enhance model interpretability by leveraging rules derived from the RuleFit white-box model and SHapley Additive exPlanations values, illustrating their efficacy in enhancing transparency and understanding of model decisions in the context of drift events. This comprehensive analysis offers insights into optimizing model selection and deployment strategies for dynamic data environments.