KMITL

Permanent URI for this communityhttps://dspace.kmitl.ac.th/handle/kmitl/1

Browse

Settings

Author: search.filters.author.Netisopakul, PonrudeeSubject: search.filters.subject.Natural language processing

Search Results

Now showing 1 - 2 of 2
  • Loading...
    Thumbnail Image
    Some of the metrics are blocked by your 
    consent settings
    Item type:Publication,
    Comparison of Evaluation Metrics for Short Story Generation
    (2023-01-01)
    Netisopakul, Ponrudee
    ;
    Taoto, Usanisa
    This study aimed to analyze the correlation among different automatic evaluation metrics for short story generation. In the study, texts were generated from short stories using different language models: the N-gram model, the Continuous Bag-of-Word (CBOW) model, the Gated recurrent unit (GRU) model and the Generative Pre-trained Transformer 2 (GPT-2) model. All models were trained on short Aesop's fables. The quality of the generated text was measured with various metrics: Perplexity, BLEU score, the number of grammatical errors, Self-BLEU score, ROUGE score, BERTScore, and Word Mover's Distance (WMD). The resulting correlation analysis of the evaluation metrics revealed four groups of correlated metrics. Firstly, perplexity and grammatical errors were moderately correlated. Secondly, BLEU, ROUGE and BERTScore were highly correlated. Next, WMD was negatively correlated with BLEU, ROUGE and BERTScore. On the other hand, Self-BLEU, which measures text diversity within the model, did not correlate with the other metrics. In conclusion, to evaluate text generation, a combination of various metrics should be used to measure different aspects of the generated text.
  • Loading...
    Thumbnail Image
    Some of the metrics are blocked by your 
    consent settings
    Item type:Publication,
    Thai stock news classification based on price changes and sentiments
    (2022-01-01)
    Netisopakul, Ponrudee
    ;
    Saewong, Woranun
    This research investigates the daily stock news influences toward a company's stock price direction in the Stock Exchange of Thailand. First, machine learning's text classification methods, namely, naïve Bayes, decision tree, random forest, support vector machine, and the three-layer and the five-layer backpropagation neural networks, are applied to predict the stock price directions using stock news collected during the year 2018. Then, the stock news sentiment is incorporated to help improve the prediction accuracy. Last, a meaningful grouping of stock news is carried out to further improve the direction prediction. The testing dataset collected from January to March 2019 stock news are used for model evaluations. The best accuracy obtained from the baseline dataset using stock news only is 78.6%. When dataset is augmented with sentiments and grouped, the best accuracy increases to 90.6%.