Dynamic load balancing of short videos in heterogeneous storage environment

Abstract

Performance of storage system has great impact on overall performance of I/O intensive systems. Data distribution across storage devices affects storage performance directly. Data migration can degrade system performance. This paper focuses on data placement issue in heterogeneous tiers of storage devices storing YouTube videos. In addition to storage capability and current workload, our data placement algorithm also takes into account future workload. Future workload is estimated from video's characteristics. Workload in all storage tiers is dynamically adjusted to achieve balance at all times. Four data placement algorithms were used to distribute 16,314 videos across 3-tier storage system in the experiment. The proposed algorithm resulted in a more balance workload distribution compared to round robin and random algorithms.