KMITL

Permanent URI for this communityhttps://dspace.kmitl.ac.th/handle/123456789/1

Browse

Search Results

Now showing 1 - 10 of 12

Some of the metrics are blocked by your
consent settings
Self-attention hierarchical kernel reservoir state network for inland water level prediction
(2026-01-15)
Liu, Zongying
;
Xu, Xiaohan
;
Pasupa, Kitsuchart
;
Loo, Chu Kiong
;
Wei, Yang
Waterway transportation sustainably facilitates global trade through eco-efficient cargo movement, where accurate water level forecasting is critical for ensuring navigational safety and operational continuity. To develop a highly accurate prediction model, it is essential to consider the periodic characteristics of water level data, which often emerge in real-world datasets. This study introduces a novel reservoir state structure based on reservoir computing theory, the Self-attention Hierarchical Kernel Reservoir State Network (SHK-RSN). It employs three primary mechanisms. First, a hierarchical feature extraction method groups training data and extracts high-dimensional features from these groups using the kernel trick in a hierarchical manner. Second, a self-attention weight selection approach is introduced to replace the random weights in the Hierarchical Kernel Reservoir State Network (HK-RSN), improving the rationale for hidden neuron connections and enhancing the interpretability of weight selection. Third, a novel reservoir state structure is proposed to capture periodic information and extract temporal features across periods, enabling the model to capture richer temporal information and identify relationships among periods. Experiments are conducted on one artificial and five real-world time series datasets, with forecast performance evaluated over 1–7 steps. Our proposed model, SHK-RSN, is compared with models based on randomization, the kernel trick, and deep learning. The experimental results demonstrate that SHK-RSN exhibits superior forecasting ability relative to the baselines. It achieves the best Symmetric Mean Absolute Percentage Error (SMAPE) across all datasets in the 1–7 period average among baseline methods, demonstrating a relative improvement of 25.7% to 46.9% over the conventional Echo State Network.
Some of the metrics are blocked by your
consent settings
MS-PatchTST: Leveraging Multi-Scale Temporal Features for Water Level Forecasting
(2025-01-01)
Zhang, Dong
;
Pasupa, Kitsuchart
;
Liu, Zongying
;
Pan, Mingyang
Accurate water level forecasting is essential for navigation, enabling safe sailing, effective drought management, optimized route planning, and efficient port operations. However, traditional statistical approaches and conventional machine learning models often struggle to capture adaptive, multi-scale temporal features, thereby limiting forecasting accuracy. In recent years, patch-based forecasting methods have demonstrated strong capabilities in modeling consecutive temporal features. Building on this foundation, we propose Multi-Scale PatchTST (MS-PatchTST), a framework designed to enhance the perception of multi-scale information. The model incorporates a newly developed multi-scale parallel convolutional network (Multi-Scale ConvNet) to extract interaction features across different time scales. These features are then fused through a Transformer Encoder with relative positional encoding to capture temporal dependencies more effectively. Finally, the kernel mean squared error loss function is employed in place of the conventional mean squared error loss, improving the optimization process and enhancing overall training performance. Experiments on four real-world water level datasets demonstrate that MS-PatchTST consistently outperforms state-of-the-art baselines, achieving an average reduction of approximately 13% in both MAE and SMAPE compared with PatchTST.
Some of the metrics are blocked by your
consent settings
Weighted error-output recurrent Xavier echo state network for concept drift handling in water level prediction
(2024-11-01)
Liu, Zongying
;
Zhang, Wenru
;
Pan, Mingyang
;
Loo, Chu Kiong
;
Pasupa, Kitsuchart
Water level holds utmost significance in maritime domains. Precise water level predictions furnish indispensable insights for safe maritime navigation, guiding ships and vessels through passages, harbors, and waterways. This paper introduces a novel approach: the Weighted Error-Output Recurrent Xavier Echo State Network with Adaptive Forgetting Factor (WER-XESN-AFF). One of the contributions of this study is the introduction of the Xavier weights selection method, which replaces random weight selection from the Echo State Network (ESN). This method not only enhances forecasting performance but also reduces uncertainty in predictions. Additionally, two modified concept drift detectors, the Early Drift Detection Method and the Adaptive Forgetting Factor, are employed to address concept drift challenges. Another notable contribution is the introduction of a novel weighted error-output recurrent multi-step algorithm. This algorithm successfully overcomes the error accumulation problem by using past forecast errors to update current output weights. This study performs extensive experiments to evaluate the effectiveness of our approach in multi-step prediction in synthetic and real datasets. It compares the performance between the conventional randomization-based models and the ESN with the new weights selection approach and also tests the ability of concept drift detectors and the weighted error-output multi-step algorithm. Empirical findings and statistical analyses demonstrate that our proposed methods achieve expected effects, and the proposed model has better prediction ability than baselines. A significant improvement rate of 75.39% in Mean Squared Error is evident within the Jiujiang water level dataset when contrasting the performance of WER-XESN-AFF against the baseline model R-ESN across the 1–5 period.
Some of the metrics are blocked by your
consent settings
Error-output recurrent multi-layer Kernel Reservoir Network for electricity load time series forecasting
(2023-01-01)
Liu, Zongying
;
Tahir, Ghalib Ahmed
;
Masuyama, Naoki
;
Kakudi, Habeebah Adamu
;
Fu, Zhongyu
Electricity is one of the most consumed commodities in the modern world. Electricity load prediction models are used to plan distribution operations to balance the equilibrium of demand and supply. This necessity has increased the number of recent research works. They employed several learning algorithms, such as support vector regression, to predict demands. However, these algorithms have high computational cost and too many user-defined parameters that directly impact their performance. Recently, randomization-based learning algorithms have been widely tested because they performed well at a lower cost. However, still, there was a main drawback: uncertainty in approximation and learning. This work employed a kernel trick to solve the uncertainty problem. A kernel with reservoir-state layers was used to solve the problem. The kernel reservoir-state layers from the echo state network not only transformed features into high-dimensional space, but also enhanced the forecasting ability by learning temporal information. Additionally, the proposed model also had a multi-step prediction ability that used previous forecasting errors to update the output weights in the current step to prevent an accumulated error problem. We compared our proposed model with single-layer and multi-layer variants of Extreme Learning Machine, Echo State Network, and Random Vector Functional Link on ten electrical load data sets. The proposed model showed the best performance on 9/10 data sets in terms of Mean Square Error or Symmetric Mean Absolute Percentage Error. These findings implied that the proposed algorithm was superior in forecasting long-term electricity load.
Some of the metrics are blocked by your
consent settings
A novel error-output recurrent two-layer extreme learning machine for multi-step time series prediction
(2021-03-01)
Liu, Zongying
;
Loo, Chu Kiong
;
Pasupa, Kitsuchart
With the development of industry and technology, the development of the environment and cities has drawn lots of attention. Time series prediction plays a vital role in protecting the environment and improving the level of intelligence and technology in cities, for example prediction of air pollution, water levels, palm oil prices, financial data and grid security. We describe a new algorithm, “Error-output Recurrent Two-layer Extreme Learning Machine” or ERT-ELM: it applied a new recurrent technique, that not only removed the restriction of the prediction horizon problem, but it also used a mean squared error of the current step to update the output weights for the next step. This technique avoided error accumulation in the original recurrent algorithm for multi-step time series prediction. Moreover, the new two-layer structure network improved forecasting compared to conventional single-layer or two-layer ELM models. Quantum behaved Particle Swarm Optimization was used to find suitable ERT-ELM parameters. The ability of our model was assessed on ten data sets—two artificial and eight real-world data sets and performed significantly better than the baselines. Especially for the synthetic data sets, in 1–18 prediction periods, our model achieved mean square errors of 2.64 ×10<sup>−3</sup> on the Mackey-Glass data set and 1.49 × 10<sup>−4</sup> on the Lorenz data sets.
Some of the metrics are blocked by your
consent settings
Online Sequential Extreme Learning Machine based Instinct Plasticity for Classification
(2020-10-06)
Liu, Zongying
;
Pasupa, Kitsuchart
Random determination of input weights leads to unstable performance in Online Sequential Extreme Learning Machines (OS-ELM), so obtaining reliable input weights was expected to improve the model performance. We designed a new model-the OS-ELM based Instinct Plasticity with a new weight selection scheme (NOS-ELM-IP) to enhance the forecast stability and accuracy for classification. In this model, the input weights were selected by a new weight selection method, which replaced the original random selection part in OS-ELM. Moreover, the Instinct Plasticity idea was used to find the gain and bias, used in the sequential training part of OS-ELM. It maximized the information of hidden neurons and enlarged the memory. The experimental results show that the proposed new weight selection method and Instinct Plasticity rule enhanced the overall performance in classification tasks for binary and multi-class data sets.
Some of the metrics are blocked by your
consent settings
Meta-cognitive recurrent kernel online sequential extreme learning machine with kernel adaptive filter for concept drift handling
(2020-02-01)
Liu, Zongying
;
Loo, Chu Kiong
;
Pasupa, Kitsuchart
;
Seera, Manjeevan
This paper proposes a multi-step prediction model for time series prediction, i.e. Meta-cognitive Recurrent Kernel Online Sequential Extreme Learning Machine with Drift Detector Mechanism (Meta-RKOS-ELM<inf>ALD</inf>). Recurrent multi-step algorithm is applied to release the limitation in the number of prediction steps, and Drift Detector Mechanism (DDM) is used to overcome the problem of concept drift in the prediction model. The new meta-cognitive strategy decides the way of the incoming data during training, which decreases the training computation of prediction model and solves the parameter dependency. In our evaluation, we use a total of six artificial data sets and three real-world data sets (Standard & Poor's 500 Index, Shanghai Stock Exchange Composite Index, and Ozone Concentration in Toronto) to prove the ability of kernel filters, the detecting ability of concept drift detector, and situation of applying meta-cognitive strategy in our proposed model. Experiments results indicate that the Meta-KOS-ELM<inf>ALD</inf> with DDM has better forecasting ability in various predicting periods with the shortest learning time, as compared with other algorithms.
Some of the metrics are blocked by your
consent settings
Real-time financial data prediction using meta-cognitive recurrent kernel online sequential extreme learning machine
(2019-01-01)
Liu, Zongying
;
Loo, Chu Kiong
;
Pasupa, Kitsuchart
This paper proposes a novel algorithm called Meta-cognitive Recurrent Kernel Online Sequential Extreme Learning Machine with a kernel filter and a modified Drift Detector Mechanism (Meta-RKOS-ELM<inf>ALD</inf>-DDM). The algorithm aims to tackle a well-known concept drift problem in time series prediction by utilising the modified concept drift detector mechanism. Moreover, the new meta-cognitive learning strategy is employed to solve parameter dependency and reduce learning time. The experimental results show that the proposed method can achieve better performance than the conventional algorithm in a set of financial datasets.
Some of the metrics are blocked by your
consent settings
Recurrent kernel extreme reservoir machine for time series prediction
(2018-04-04)
Liu, Zongying
;
Loo, Chu Kiong
;
Masuyama, Naoki
;
Pasupa, Kitsuchart
This paper proposes a novel recurrent multi-step-ahead prediction model called recurrent kernel extreme reservoir machine (RKERM) with quantum particle swarm optimization (QPSO). This model combines the strengths of recurrent kernel extreme learning machine (RKELM) and modified reservoir computing to overcome the limitations of prediction horizon with increased prediction accuracy based on reservoir computing theory. Furthermore, QPSO is used to optimize the parameters of kernel method and leaking rate of reservoir computing in the RKERM. In the experiment, we apply two synthetic benchmark data sets and five real-world time series data sets, including Malaysia palm oil price, ozone concentration in Toronto, sunspots, Standard Poor's 500, and water level at Phra Chulachomklao Fort in Thailand to evaluate the echo state network, recurrent support vector regression, recurrent extreme learning machine, RKELM, and RKERM. The experimental results show that the RKERM with QPSO has superior abilities in the different predicting horizons than others.
Some of the metrics are blocked by your
consent settings
Handling concept drift in time-series data: Meta-cognitive recurrent recursive-kernel OS-ELM
(2018-01-01)
Liu, Zongying
;
Loo, Chu Kiong
;
Pasupa, Kitsuchart
This paper proposes a meta-cognitive recurrent multi-step-prediction model called Meta-cognitive Recurrent Recursive Kernel Online Sequential Extreme Learning Machine with a new modified Drift Detector Mechanism (Meta-RRKOS-ELM-DDM). This model combines the strengths of Recurrent Kernel Online Sequential Extreme Learning Machine (RKOS-ELM) with the recursive kernel method and a new meta-cognitive learning strategy. We apply Drift Detector Mechanism to solve concept drift problem. Recursive kernel method successfully replaces the normal kernel method in RKOS-ELM and generates a fixed reservoir with optimised information. The new meta-cognitive learning strategy can reduce the computational complexity. The experimental results show that Meta-RRKOS-ELM-DDM has a superior prediction ability in different predicting horizons than the others.

KMITL

Browse

Filters

Settings

Sort By

Results per page

Search Results