KMITL

Permanent URI for this communityhttps://dspace.kmitl.ac.th/handle/123456789/1

Browse

Search Results

Now showing 1 - 7 of 7

Some of the metrics are blocked by your
consent settings
n-LIPO: Framework for Diverse Cooperative Agent Generation Using Policy Compatibility
(2025-01-01)
Charakorn, Rujikorn
;
Manoonpong, Poramate
;
Dilokthanakul, Nat
Diverse training partners in multiagent tasks are crucial for training a robust and adaptable cooperative agent. Prior methods often rely on state-action information to diversify partners’ behaviors, but this can lead to minor variations instead of diverse behaviors and solutions. We address this limitation by introducing a novel training objective based on “policy compatibility.” Our method learns diverse behaviors by encouraging agents within a team to be compatible with each other while being incompatible with agents from other teams. We theoretically prove that incompatible policies are inherently dissimilar, allowing us to use policy compatibility as a proxy for diversity. We call this method learning incompatible policies for n -player cooperative games (n-LIPO). We propose to further diversify individual policies by incorporating a mutual information objective using state-action information. We empirically demonstrate that n-LIPO effectively generates diverse joint policies in various two-player and multi-player cooperative environments. In a complex cooperative task, two-player multi-recipe Overcooked, we find that n-LIPO generates a population of behaviorally diverse partners. These populations are then used to train robust generalist agents that can generalize better than using baseline populations. Finally, we demonstrate that n-LIPO can be applied to a high-dimensional StarCraft multiagent challenge (SMAC) multiplayer cooperative environment to discover diverse winning strategies when only a single goal exists.
Some of the metrics are blocked by your
consent settings
Diversity Is Not All You Need: Training A Robust Cooperative Agent Needs Specialist Partners
(2024-01-01)
Charakorn, Rujikorn
;
Manoonpong, Poramate
;
Dilokthanakul, Nat
Partner diversity is known to be crucial for training a robust generalist cooperative agent. In this paper, we show that partner specialization, in addition to diversity, is crucial for the robustness of a downstream generalist agent. We propose a principled method for quantifying both the diversity and specialization of a partner population based on the concept of mutual information. Then, we observe that the recently proposed cross-play minimization (XP-min) technique produces diverse and specialized partners. However, the generated partners are overfit, reducing their usefulness as training partners. To address this, we propose simple methods, based on reinforcement learning and supervised learning, for extracting the diverse and specialized behaviors of XP-min generated partners but not their overfitness. We demonstrate empirically that the proposed method effectively removes overfitness, and extracted populations produce more robust generalist agents compared to the source XP-min populations. This result highlights the importance of considering both the diversity and specialization of training partners while carefully managing their overfitness for training robust cooperative generalists.
Some of the metrics are blocked by your
consent settings
Impact of Rhythm, Tempo, and Rest Variations on Pitch Detection in Deep Learning-Based Piano Transcription Models
(2024-01-01)
Pangwapee, Priyakorn
;
Mekkoktanphira, Juthakan
;
Dilokthanakul, Nat
;
Lochanachit, Sirasit
;
Kanungsukkasem, Nont
This paper investigates the impact of rhythm, tempo, and rest variations on pitch detection in deep learning-based models for piano transcription. We conducted a series of experiments using GRU and Transformer architectures, manipulating note lengths, rhythmic patterns, and rest intervals to assess their effect on pitch transcription accuracy. Our findings indicate that model performance is significantly influenced by these musical factors. The experiment with GRU shows notable sensitivity to rhythmic and rest changes. However, the Transformer model handles varied conditions more robustly. These findings help refine our approach to music transcription software, particularly in improving pitch recognition across varied rhythmic patterns, tempos and rests.
Some of the metrics are blocked by your
consent settings
Hybrid learning mechanisms under a neural control network for various walking speed generation of a quadruped robot
(2023-10-01)
Zhang, Yanbin
;
Thor, Mathias
;
Dilokthanakul, Nat
;
Dai, Zhendong
;
Manoonpong, Poramate
Legged robots that can instantly change motor patterns at different walking speeds are useful and can accomplish various tasks efficiently. However, state-of-the-art control methods either are difficult to develop or require long training times. In this study, we present a comprehensible neural control framework to integrate probability-based black-box optimization (PI<sup>BB</sup>) and supervised learning for robot motor pattern generation at various walking speeds. The control framework structure is based on a combination of a central pattern generator (CPG), a radial basis function (RBF) -based premotor network and a hypernetwork, resulting in a so-called neural CPG-RBF-hyper control network. First, the CPG-driven RBF network, acting as a complex motor pattern generator, was trained to learn policies (multiple motor patterns) for different speeds using PI<sup>BB</sup>. We also introduce an incremental learning strategy to avoid local optima. Second, the hypernetwork, which acts as a task/behavior to control parameter mapping, was trained using supervised learning. It creates a mapping between the internal CPG frequency (reflecting the walking speed) and motor behavior. This map represents the prior knowledge of the robot, which contains the optimal motor joint patterns at various CPG frequencies. Finally, when a user-defined robot walking frequency or speed is provided, the hypernetwork generates the corresponding policy for the CPG-RBF network. The result is a versatile locomotion controller which enables a quadruped robot to perform stable and robust walking at different speeds without sensory feedback. The policy of the controller was trained in the simulation (less than 1 h) and capable of transferring to a real robot. The generalization ability of the controller was demonstrated by testing the CPG frequencies that were not encountered during training.
Some of the metrics are blocked by your
consent settings
Semantic Bird'View Map Prediction Using Horizontally-Aware Pyramid Occupancy Network
(2023-01-01)
Teerarattanyu, Thanapat
;
Wongchai, Tunlaton
;
Pavarangkoon, Praphan
;
Dilokthanakul, Nat
Deep neural network has been used to predict the bird's-eye-view map from a frontal camera of an autonomous car. A state-of-the-art approach, namely pyramid occupancy network (PON), uses an encoder-decoder architecture to condense an image column into a context vector that describes the object occupancy along the radial direction. Our work, Horizontally-aware Pyramid Occupancy Network (H-PON), extends the PON model with a novel component that provides additional context information describing the relationships of the objects along the horizontal direction. This is done by also encoding the horizontal column of the image into an additional context vector using another encoder-decoder layer. This context vector is, then, expanded back providing improved features for semantic reasoning across the horizontal direction. We found that this simple extension significantly improves PON's semantic prediction performance in the nuScences dataset. Our experiment shows that the objects that are rarely seen and those that are further away from the center greatly benefit from this novel component.
Some of the metrics are blocked by your
consent settings
Energy-Efficient Continual Learning for Autonomous Driving
(2023-01-01)
Ng, Qi Ding
;
Loo, Chu Kiong
;
Pasupa, Kitsuchart
;
Dilokthanakul, Nat
;
Zhang, Jie
Our work highlighted the primary challenges of Autonomous Driving (AD), namely the Catastrophic Forgetting (CF) of previous knowledge by the AD system upon new scenario encounters. Considering the infeasible model retraining with past data given computational, power, and storage constraints on the embedded device, we proposed an experiment featuring Avalanche Continual Learning (CL) training strategies to investigate which strategies excel in this task and combine the promising ones in the hope for a more balanced and efficient trade-off between performance and energy consumption. Our experiment unprecedentedly validated the candidates against a new benchmark introducing natural distribution change and time correlation between input images. We found that although a synergy of CL strategies yields higher resistance towards CF, the slight accuracy gain is not worth the additional computation when we account for energy consumption, rendering a simple Replay strategy the best solution for the Continual Learning benchmark for Autonomous Driving: Online Continual Classification (CLAD-C). Our proposal delivers a 65.80% improvement over the baseline at our proposed accuracy-power ratio metric.
Some of the metrics are blocked by your
consent settings
GENERATING DIVERSE COOPERATIVE AGENTS BY LEARNING INCOMPATIBLE POLICIES
(2023-01-01)
Charakorn, Rujikorn
;
Manoonpong, Poramate
;
Dilokthanakul, Nat
Training a robust cooperative agent requires diverse partner agents. However, obtaining those agents is difficult. Previous works aim to learn diverse behaviors by changing the state-action distribution of agents. But, without information about the task's goal, the diversified agents are not guided to find other important, albeit sub-optimal, solutions: the agents might learn only variations of the same solution. In this work, we propose to learn diverse behaviors via policy compatibility. Conceptually, policy compatibility measures whether policies of interest can coordinate effectively. We theoretically show that incompatible policies are not similar. Thus, policy compatibility-which has been used exclusively as a measure of robustness-can be used as a proxy for learning diverse behaviors. Then, we incorporate the proposed objective into a population-based training scheme to allow concurrent training of multiple agents. Additionally, we use state-action information to induce local variations of each policy. Empirically, the proposed method consistently discovers more solutions than baseline methods across various multi-goal cooperative environments. Finally, in multi-recipe Overcooked, we show that our method produces populations of behaviorally diverse agents, which enables generalist agents trained with such a population to be more robust.

KMITL

Browse

Filters

Settings

Sort By

Results per page

Search Results