Text analytics of IMDB reviews using latent Dirichlet allocation

Abstract

This study utilizes a seven-step text analytics methodology in MATLAB to analyze IMDB review data. The methodology includes: 1) Data Loading and Preprocessing, involving loading the data and preprocessing text using functions such as converting text to lowercase, tokenization, punctuation removal, stop word removal, short and long word removal, and lemmatization. 2) Exploratory Data Analysis, using word clouds to visualize the most frequent words. 3) Bag-of-Words Model Creation, including splitting the data into training and validation sets and removing infrequent words and empty documents. 4) Topic Modeling with LDA, testing different solvers and evaluating performance using perplexity. 5) Optimal Topic Number Selection, optimizing the number of topics by comparing validation perplexities. 6) Final Topic Model Training, training a final LDA model and assessing its performance. and 7) Topic Interpretation and Analysis, involving visualizing word clouds and finding relevant reviews for specific words.