Implementation of Aspect-Based Sentiment Analysis on the Mitra Darat App User Reviews Using Machine Learning
DOI:
https://doi.org/10.30865/klik.v4i6.1889Keywords:
Aspect-based Sentiment Analysis; Machine Learning; Support Vector Machine; Naive Bayes; Mitra DaratAbstract
The Mitra Darat application is an Android-based application available on the Google Play Store, developed by the Directorate General of Land Transportation (Ministry of Transportation). User reviews on this platform enable direct communication with developers, offering valuable feedback for service enhancement and future development. For that reason, aspect-based sentiment analysis is needed to help organizations monitor product sentiment in user feedback, and understand user needs. This research aims to implement aspect-based sentiment analysis on Mitra Darat application user reviews to generate insights via a system dashboard. Comparing Naive Bayes (NB) and Support Vector Machines (SVM) for machine learning models with the addition of pre-trained Indobert as word embedding, SVM showed superior performance with an accuracy score of 94% for aspect classification and 90% for sentiment classification, compared to Naive Bayes with scores of 84% and 78% respectively. The trained Support Vector Machine model (SVM) was then utilized to analyze 967 reviews of the Mitra Darat application for 2023. The results of the analysis are presented on a dashboard page with summary information, which shows that the overall user sentiment is 52.3% positive, 10% neutral, and 37.7% negative. In terms of sentiment polarity by aspect, the system aspect is 29% positive, 8% neutral, and 63% negative, meaning that some bugs and issues have been found in the application, so it can be evaluated for future system development. The service aspect is 62% positive, 27% neutral, and 11% negative, which means that the free mudik service is quite well organized
Downloads
References
R. L. Wuri, “Biar Enggak Bingung Saat Mudik, Kemenhub Ajak Masyarakat Gunakan Aplikasi MitraDarat dan Ferizy,” wartaekonomi. Accessed: Dec. 08, 2023. [Online]. Available: https://wartaekonomi.co.id/read491399/biar-enggak-bingung-saat-mudik-kemenhub-ajak-masyarakat-gunakan-aplikasi-mitradarat-dan-ferizy
M. Arnani, L. A. Azanella, and I. D. Wedhaswary, “Mudik, ‘Mulih Dhisik’, Kembali ke Udik...,” Kompas.com, Jun. 07, 2018. Accessed: Dec. 08, 2023. [Online]. Available: https://nasional.kompas.com/read/2018/06/07/09311731/mudik-mulih-dhisik-kembali-ke-udik
S. Panichella, A. Di Sorbo, E. Guzman, C. A. Visaggio, G. Canfora, and H. C. Gall, “How can i improve my app? Classifying user reviews for software maintenance and evolution,” in 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE, 2015, pp. 281–290. doi: 10.1109/ICSM.2015.7332474.
S. Venkatakrishnan, A. Kaushik, and J. K. Verma, “Sentiment Analysis on Google Play Store Data Using Deep Learning,” in Algorithms for Intelligent Systems, Springer Nature Singapore Pte Ltd, 2020, pp. 15–30. doi: 10.1007/978-981-15-3357-0_2.
U. Pathak and Er. P. Rai, “Sentiment Analysis: Methods, Applications, and Future Directions,” Int J Res Appl Sci Eng Technol, vol. 11, no. 2, pp. 1453–1458, 2023, doi: 10.22214/ijraset.2023.49165.
S. S. Shah, “Opinion Mining For Text Data: An Overview,” Int J Res Appl Sci Eng Technol, vol. 10, no. 6, pp. 4301–4311, 2022, doi: 10.22214/ijraset.2022.44902.
M. Wankhade, A. C. S. Rao, and C. Kulkarni, “A survey on sentiment analysis methods, applications, and challenges,” Artif Intell Rev, vol. 55, no. 7, pp. 5731–5780, 2022, doi: 10.1007/s10462-022-10144-1.
S. N. Hakim, A. J. Putra, and A. U. Khasanah, “Sentiment analysis on myindihome user reviews using support vector machine and naïve bayes classifier method,” International Journal of Industrial Optimization, vol. 2, no. 2, pp. 151–164, 2021, doi: 10.12928/ijio.v2i2.4437.
E. Indrayuni, A. Nurhadi, and D. A. Kristiyanti, “Implementasi Algoritma Naive Bayes, Support Vector Machine, dan K-Nearest Neighbors untuk Analisa Sentimen Aplikasi Halodoc,” Faktor Exacta, vol. 14, no. 2, pp. 64–71, 2021, doi: 10.30998/faktorexacta.v14i2.9697.
Z. Nurthohari, D. I. Sensuse, and S. Lusa, “Sentiment Analysis of Jakarta Bus Rapid Transportation Services using Support Vector Machine,” in 2022 International Conference on Data Science and Its Applications (ICoDSA), IEEE, 2022, pp. 171–176. doi: 10.1109/ICoDSA55874.2022.9862903.
C. A. Bahri and L. H. Suadaa, “Aspect-Based Sentiment Analysis in Bromo Tengger Semeru National Park Indonesia Based on Google Maps User Reviews,” IJCCS (Indonesian Journal of Computing and Cybernetics Systems), vol. 17, no. 1, pp. 79–90, 2023, doi: 10.22146/ijccs.77354.
M. S. Mubarok, Adiwijaya, and M. D. Aldhi, “Aspect-based sentiment analysis to review products using Naïve Bayes,” in AIP conference proceedings, AIP Publishing, 2017, p. 020060. doi: 10.1063/1.4994463.
R. K. Singh and K. Ramalingam, “Amazon Product Review Sentiment Analysis with Machine Learning,” International Journal of Trend in Scientific Research and Development (ijtsrd), vol. 5, no. 4, pp. 720–723, 2021.
A. M. Rahat, A. Kahir, and A. K. M. Masum, “Comparison of Naive Bayes and SVM Algorithm based on Sentiment Analysis Using Review Dataset,” in 2019 8th International Conference System Modeling and Advancement in Research Trends (SMART), IEEE, 2019, pp. 266–270. doi: 10.1109/SMART46866.2019.9117512.
D. I. Af’idah, P. D. Anggraeni, M. Rizki, A. B. Setiawan, and S. F. Handayani, “Aspect-Based Sentiment Analysis for Indonesian Tourist Attraction Reviews Using Bidirectional Long Short-Term Memory,” JUITA: Jurnal Informatika, vol. 11, no. 1, pp. 27–36, 2023.
G. Dwyer, S. Aggarwal, and J. Stouffer, Flask: building python web services. Packt Publishing, 2017.
V. Gupta and G. S. Lehal, “A survey of text mining techniques and applications,” Journal of Emerging Technologies in Web Intelligence, vol. 1, no. 1, pp. 60–76, 2009.
A. K. Uysal and S. Gunal, “The impact of preprocessing on text classification,” Inf Process Manag, vol. 50, no. 1, pp. 104–112, 2014, doi: 10.1016/j.ipm.2013.08.006.
E. Yulianti, N. Pangestu, and M. A. Jiwanggi, “Enhanced TextRank using weighted word embedding for text summarization,” International Journal of Electrical and Computer Engineering (IJECE), vol. 13, no. 5, pp. 5472–5482, 2023, doi: 10.11591/ijece.v13i5.pp5472-5482.
B. Wilie et al., “IndoNLU: Benchmark and resources for evaluating Indonesian natural language understanding,” in Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, Association for Computational Linguistics, 2020.
J. Hartmann, M. Heitmann, C. Siebert, and C. Schamp, “More than a Feeling: Accuracy and Application of Sentiment Analysis,” International Journal of Research in Marketing, vol. 40, no. 1, pp. 75–87, 2023, doi: 10.1016/j.ijresmar.2022.05.005.
Scikit-learn, “Naive Bayes,” Scikit-learn. [Online]. Available: https://scikit-learn.org/stable/modules/naive_bayes.html
V. N. Vapnik, “An overview of statistical learning theory,” IEEE Trans Neural Netw, vol. 10, no. 5, pp. 988–999, 1999, doi: 10.1109/72.788640.
D. Gil and M. Johnsson, “Support vector machines in medical classification tasks,” in Support Vector Machines: Data Analysis, Machine Learning and Applications, Nova Science Publishers, Inc., 2011, pp. 81–102.
D. Ghimire, “Comparative study on Python web frameworks: Flask and Django,” Metropolia University of Applied Sciences, 2020.
A. M. R. Wajong, “Applying performance dashboard in hospitals,” International Journal Of Software Engineering and Its Applications, vol. 9, no. 1, pp. 213–220, 2015, doi: 10.14257/ijseia.2015.9.1.19.
P. Guilizzoni, “What are wireframes and why are they used?,” Balsamiq Studios, LLC. Accessed: Dec. 17, 2023. [Online]. Available: https://balsamiq.com/learn/articles/what-are-wireframes/
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Implementation of Aspect-Based Sentiment Analysis on the Mitra Darat App User Reviews Using Machine Learning
ARTICLE HISTORY
Issue
Section
Copyright (c) 2024 Maulaya Ishaq, Dewi Lestari; Michael Marchenko
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
