ANALISIS SENTIMEN PROGRAM MAKAN BERGIZI GRATIS DI X: KOMPARASI NAIVE BAYES DAN SVM

SENTIMENT ANALYSIS OF THE FREE NUTRITIOUS MEALS PROGRAM ON X: A COMPARISON OF NAIVE BAYES AND SUPPORT VECTOR MACHINE (SVM)

Abstract

Program Makan Bergizi Gratis (MBG) memicu diskursus publik masif di platform X (Twitter) dengan konteks linguistik yang kompleks dan distribusi kelas yang tidak seimbang. Penelitian ini bertujuan untuk menganalisis sentimen masyarakat sekaligus membandingkan performa algoritma Multinomial Naive Bayes (MNB) dan Support Vector Machine (SVM/LinearSVC) berbasis ekstraksi fitur Term Frequency-Inverse Document Frequency (TF-IDF) dan penanganan class imbalance menggunakan Synthetic Minority Over-sampling Technique (SMOTE). Dataset dilabeli secara otomatis menggunakan pendekatan leksikon dan divalidasi manual (N = 100) berbasis konteks pragmatik. Evaluasi reliabilitas pelabelan manual menghasilkan nilai Cohen's Kappa\kappa = 0,6425  (substantial agreement), yang mengungkap adanya bias asimetris pada leksikon otomatis dalam mengenali sentimen negatif (Recall  = 51,35% ). Untuk mencegah kebocoran data (data leakage), SMOTE diterapkan secara per-fold pada skema Stratified 5-Fold Cross-Validation dan train-set only pada skema hold-out split 80:20. Hasil pengujian Cross-Validation menunjukkan SVM secara konsisten mengungguli MNB dengan rata-rata akurasi  81,25% \pm 2,37%  berbanding  59,58% \pm 2,34% . Pada evaluasi data uji (80:20), SVM mencapai akurasi  83,26%  dan Macro F1-score  0,78 , sedangkan MNB hanya mencapai akurasi  60,93%  dan Macro F1-score  0,59 . Hasil ini membuktikan bahwa SVM lebih adaptif dalam memisahkan batas keputusan pada teks opini publik yang kaya akan konteks, frasa implisit, dan sarkasme.

The Free Nutritious Meal (MBG) program has triggered massive public discourse on platform X (Twitter), characterized by complex linguistic contexts and imbalanced class distributions. This study aims to analyze public sentiment and compare the performance of Multinomial Naive Bayes (MNB) and Support Vector Machine (SVM/LinearSVC) algorithms using Term Frequency-Inverse Document Frequency (TF-IDF) feature extraction and Synthetic Minority Over-sampling Technique (SMOTE) for class imbalance mitigation. The dataset was automatically labeled using a lexicon approach and manually validated ( N = 100 ) based on pragmatic context. Reliability evaluation of manual annotation yielded a Cohen’s Kappa of \kappa = 0.6425 (substantial agreement), revealing a systematic one-directional bias in the automated lexicon toward underestimating negative sentiment (Recall = 51.35% ). To prevent data leakage, SMOTE was strictly applied per-fold during Stratified 5-Fold Cross-Validation and exclusively on the training set in the 80:20 hold-out split scheme. Cross-validation results demonstrated that SVM consistently outperformed MNB, achieving a mean accuracy of 81.25% \pm 2.37% compared to 59.58% \pm 2.34% . On the 80:20 test evaluation, SVM achieved an accuracy of 83.26% and a Macro F1-score of 0.78 , whereas MNB achieved an accuracy of 60.93% and a Macro F1-score of 0.59 . These findings confirm that SVM is significantly more robust in establishing decision boundaries for public opinion text containing implicit phrasing, domain-specific terminology, and sarcasm.