Sentiment Analysis on Social Media Using Transformer-based Models: A Comparative Study
Review artikel NLP untuk sentiment analysis
Penulis: Dr. Siti Nurhaliza, Ahmad Prakoso, M.Kom.
Afiliasi: Universitas Padjadjaran
Negara: Indonesia
Tanggal Review: 25 September 2024
Publikasi: Applied Sciences (MDPI)
Putaran: Round 2
Ini adalah review round 2 setelah authors melakukan major revision. Paper melakukan comprehensive comparative study terhadap 8 transformer-based models untuk sentiment analysis pada social media data (Twitter, Facebook, Reddit, YouTube, Instagram).
Results menunjukkan RoBERTa dan XLNet memberikan accuracy tertinggi (91-93%), namun computational cost tinggi. DistilBERT menawarkan trade-off terbaik dengan accuracy 88% dan inference speed 2x lebih cepat. Statistical tests confirm significance of results.
Area Fokus Review
Comparative analysis methodology, experimental design, statistical analysis, contribution dan novelty
Evaluasi
Kelebihan
✓ Excellent Revision - All Concerns Addressed
Paper ini (Review Round 2) sudah mengakomodasi semua feedback dari review round 1 dengan sangat baik.
1. Comprehensive Comparative Study
Comparison terhadap 8 transformer models:
- BERT (Bidirectional Encoder Representations from Transformers)
- RoBERTa (Robustly Optimized BERT)
- ALBERT (A Lite BERT)
- XLNet
- DistilBERT
- ELECTRA
- DeBERTa
- T5 (Text-to-Text Transfer Transformer)
2. Multi-Domain Evaluation
Eksperimen dilakukan pada 5 datasets berbeda:
| Dataset | Domain | Size |
|---|---|---|
| General social media | 100K samples | |
| Product reviews | 85K samples | |
| Discussion forums | 75K samples | |
| YouTube | Video comments | 90K samples |
| Image captions | 65K samples |
3. Statistical Analysis
Statistical significance test dilakukan dengan proper methodology:
- Paired t-test untuk accuracy comparison
- Wilcoxon signed-rank test untuk non-parametric analysis
- P-value < 0.05 untuk significance level
4. Trade-off Analysis
Analysis mendalam terhadap computational efficiency vs accuracy trade-off:
- Inference speed (samples/second)
- Memory consumption
- Training time
- Model size
5. Visualization Quality
Results visualization clear dan informative dengan:
- Bar charts untuk accuracy comparison
- Scatter plots untuk speed vs accuracy trade-off
- Heatmaps untuk confusion matrices
- Box plots untuk statistical distribution
Kelemahan
✓ No Significant Weaknesses
All previous concerns from Round 1 have been addressed properly:
- ✓ Added 3 more transformer models for comparison
- ✓ Extended experiments to 5 datasets across different domains
- ✓ Included comprehensive statistical significance tests
- ✓ Added detailed computational efficiency analysis
- ✓ Improved visualization quality
- ✓ Enhanced discussion section with practical insights
Rekomendasi
✓ READY FOR PUBLICATION
Paper sudah excellent dan siap untuk publikasi.
Minor formatting adjustments:
- Adjust table 3 formatting untuk comply dengan journal style
- Fix figure 7 caption (minor typo)
- Update references format untuk consistency
Recommendation: ACCEPT
Catatan Reviewer
Round 2 Review Summary
Authors telah melakukan revisi excellent dan menjawab semua concerns dari round 1 dengan thoroughness yang sangat baik.
Key Improvements from Round 1:
| Aspect | Round 1 | Round 2 |
|---|---|---|
| Models compared | 5 models | 8 models (+60%) |
| Datasets | 2 datasets | 5 datasets (+150%) |
| Statistical tests | None | Complete (t-test, Wilcoxon) |
| Efficiency analysis | Basic | Comprehensive |
Key Findings:
- Highest Accuracy: RoBERTa & XLNet (91-93%)
- Best Trade-off: DistilBERT (88% accuracy, 2x faster inference)
- Most Efficient: DistilBERT & ALBERT (lower memory, faster)
- Statistical Confirmation: All differences statistically significant (p < 0.05)
Paper ini memberikan valuable insights tentang performance comparison transformer models untuk sentiment analysis yang akan sangat bermanfaat untuk researchers dalam memilih model yang sesuai, practitioners dalam production deployment, dan students sebagai learning material.
Strongly Recommended for Acceptance
Informasi Review
-
Kategori
Review Jurnal Internasional
-
Jenis Review
Review Jurnal Ilmiah
-
Level
Internasional
-
Metode Review
Double Blind
-
Tanggal Submit
10 Sep 2024
-
Tanggal Selesai
25 Sep 2024
-
Durasi Review
15 hari
Penerbit/Institusi
Beban Kerja
-
Skor Review
8.0/10
-
Waktu Review
4.5 jam
Kata Kunci
sentiment analysis natural language processing transformer BERT social media analytics