Comparative Analysis of DT and SVM Model Performance with SMOTE in Sentiment Classification
DOI:
https://doi.org/10.30865/klik.v4i5.1828Keywords:
DT; Sentiment Classification; SMOTE; SVM; Model PerformanceAbstract
This research investigates the efficacy of employing the Cross-Industry Standard Process for Data Mining (CRISP-DM) framework to analyze sentiment classification models. The study focuses on evaluating the performance of Decision Trees (DT) and Support Vector Machine (SVM) models integrated with the Synthetic Minority Over-sampling Technique (SMOTE) across various performance metrics, including accuracy, precision, recall, f-measure, and Area Under the Curve (AUC). Using CRISP-DM, the research ensures a systematic data preprocessing, modeling, and evaluation approach. The findings reveal that both DT and SVM models with SMOTE achieve high accuracy rates, with DT yielding an accuracy of 98.37% +/- 0.48% and SVM achieving 98.91% +/- 0.59%. These models effectively distinguish between positive and negative sentiments, as precision, recall, and f-measure scores indicate. Additionally, the AUC scores underscore the robustness of the models in sentiment analysis tasks. These results highlight the potential of CRISP-DM as a structured methodology for sentiment classification research, providing insights into the performance of different machine learning algorithms in handling imbalanced datasets. Based on these findings, it is recommended that future studies further explore the application of CRISP-DM in sentiment analysis tasks and investigate the scalability of DT and SVM models with SMOTE in larger datasets.
Downloads
References
W. Van Zoonen, J. W. Treem, and A. Sivunen, “An analysis of fear factors predicting enterprise social media use in an era of communication visibility,” Internet Res., vol. 32, no. 7, pp. 354–375, Jan. 2022, doi: 10.1108/INTR-05-2021-0341.
K. Abhari, M. Zarei, M. Parsons, and P. Estell, “Open innovation starts from home: the potentials of enterprise social media (ESM) in nurturing employee innovation,” Internet Res., vol. 33, no. 3, pp. 945–973, Jan. 2023, doi: 10.1108/INTR-08-2021-0556.
H. Li, Z. Yang, C. Jin, and J. Wang, “How an industrial internet platform empowers the digital transformation of SMEs: theoretical mechanism and business model,” J. Knowl. Manag., vol. 27, no. 1, pp. 105–120, Jan. 2023, doi: 10.1108/JKM-09-2022-0757.
M. T. Bui and D. J. F. Jeng, “Capture coproduction behavior in networking alumni communities: Progress from platform belongingness, knowledge sharing, and citizenship behavior,” J. Enterprising Communities, vol. 16, no. 1, pp. 46–73, Jan. 2022, doi: 10.1108/JEC-08-2021-0112.
E. Jütte and E. L. Olson, “A brand hegemony rejection explanation for digital piracy,” Eur. J. Mark., vol. 56, no. 5, pp. 1512–1531, Jan. 2022, doi: 10.1108/EJM-04-2020-0303.
M. Törhönen, M. Sjöblom, L. Hassan, and J. Hamari, “Fame and fortune, or just fun? A study on why people create content on video platforms,” Internet Res., vol. 30, no. 1, pp. 165–190, Jan. 2020, doi: 10.1108/INTR-06-2018-0270.
R. Casidy, C. Leckie, M. W. Nyadzayo, and L. W. Johnson, “Customer brand engagement and co-production: an examination of key boundary conditions in the sharing economy,” Eur. J. Mark., vol. 56, no. 10, pp. 2594–2621, Jan. 2022, doi: 10.1108/EJM-10-2021-0803.
Y. Hong, S. Sawang, and H. P. (Sophie) Yang, “How is entrepreneurial marketing shaped by E-commerce technology: a case study of Chinese pure-play e-retailers,” Int. J. Entrep. Behav. Res., vol. 30, no. 2–3, pp. 609–631, Jan. 2024, doi: 10.1108/IJEBR-10-2022-0951.
A. K. Olsson and I. Bernhard, “Keeping up the pace of digitalization in small businesses–Women entrepreneurs’ knowledge and use of social media,” Int. J. Entrep. Behav. Res., vol. 27, no. 2, pp. 378–396, Jan. 2021, doi: 10.1108/IJEBR-10-2019-0615.
B. Mastromartino and M. L. Naraine, “(Dis)Innovative digital strategy in professional sport: examining sponsor leveraging through social media,” Int. J. Sport. Mark. Spons., vol. 23, no. 5, pp. 934–949, Jan. 2022, doi: 10.1108/IJSMS-02-2021-0032.
N. Gryllakis and M. Matsiola, “Digital audiovisual content in marketing and distributing cultural products during the COVID-19 pandemic in Greece,” Arts Mark., vol. 13, no. 1, pp. 4–19, Jan. 2023, doi: 10.1108/AAM-09-2021-0053.
P. Tiwasing, Y. R. Kim, and S. Sawang, “The interplay between digital social capital and family-owned SME performance: a study of social media business networks,” J. Fam. Bus. Manag., vol. 13, no. 4, pp. 1026–1048, Jan. 2023, doi: 10.1108/JFBM-07-2022-0103.
A. Boukis, “Exploring the implications of blockchain technology for brand–consumer relationships: a future research agenda,” J. Prod. Brand Manag., vol. 29, no. 3, pp. 307–320, Jan. 2020, doi: 10.1108/JPBM-03-2018-1780.
K. K. Coker, R. L. Flight, and D. M. Baima, “Video storytelling ads vs argumentative ads: how hooking viewers enhances consumer engagement,” J. Res. Interact. Mark., vol. 15, no. 4, pp. 607–622, Jan. 2021, doi: 10.1108/JRIM-05-2020-0115.
J. Ho, C. Pang, and C. Choy, “Content marketing capability building: a conceptual framework,” J. Res. Interact. Mark., vol. 14, no. 1, pp. 133–151, Jan. 2020, doi: 10.1108/JRIM-06-2018-0082.
M. L. Cheung, W. K. S. Leung, M. X. Yang, K. Y. Koay, and M. K. Chang, “Exploring the nexus of social media influencers and consumer brand engagement,” Asia Pacific J. Mark. Logist., vol. 34, no. 10, pp. 2370–2385, Jan. 2022, doi: 10.1108/APJML-07-2021-0522.
S. Fready, P. Vel, and M. W. Nyadzayo, “Business customer virtual interaction: enhancing value creation in B2B markets in the post-COVID-19 era – an SME perspective,” J. Bus. Ind. Mark., vol. 37, no. 10, pp. 2075–2094, Jan. 2022, doi: 10.1108/JBIM-01-2021-0074.
Y. Wang, M. Zhang, and Y. Ming, “What contributes to online communities’ prosperity? Understanding value co-creation in product-experience-shared communities (PESCs) from the view of resource integration,” Inf. Technol. People, vol. 35, no. 7, pp. 2241–2262, Jan. 2022, doi: 10.1108/ITP-12-2020-0869.
S. L. Alam, “Many hands make light work: towards a framework of digital co-production to co-creation on social platforms,” Inf. Technol. People, vol. 34, no. 3, pp. 1087–1118, Jan. 2020, doi: 10.1108/ITP-05-2019-0231.
G. Rejikumar, A. Jose, S. Mathew, D. P. Chacko, and A. Asokan-Ajitha, “Towards a theory of well-being in digital sports viewing behavior,” J. Serv. Mark., vol. 36, no. 2, pp. 245–263, Jan. 2022, doi: 10.1108/JSM-06-2020-0247.
R. V. Kozinets, “Algorithmic branding through platform assemblages: core conceptions and research directions for a new era of marketing and service management,” J. Serv. Manag., vol. 33, no. 3, pp. 437–452, Jan. 2022, doi: 10.1108/JOSM-07-2021-0263.
B. Senanu, T. Anning-Dorson, and N. N. Tackie, “Social media insights for non-luxury fashion SMEs in emerging markets: evidence from young consumers,” J. Fash. Mark. Manag., vol. 27, no. 6, pp. 965–987, Jan. 2023, doi: 10.1108/JFMM-02-2022-0026.
A. Garrido-Moreno, V. García-Morales, S. King, and N. Lockett, “Social Media use and value creation in the digital landscape: a dynamic-capabilities perspective,” J. Serv. Manag., vol. 31, no. 3, pp. 313–343, Jan. 2020, doi: 10.1108/JOSM-09-2018-0286.
E. E. Vazquez, “Effects of enduring involvement and perceived content vividness on digital engagement,” J. Res. Interact. Mark., vol. 14, no. 1, pp. 1–16, Jan. 2020, doi: 10.1108/JRIM-05-2018-0071.
G. Oakley, “Developing pre-service teachers’ technological, pedagogical and content knowledge through the creation of digital storybooks for use in early years classrooms,” Technol. Pedagog. Educ., vol. 29, no. 2, pp. 163–175, 2020, doi: 10.1080/1475939X.2020.1729234.
E. Mora, N. Vila, and I. Küster, “Qualitative social media content analysis as teaching-learning method in higher education,” Interact. Learn. Environ., pp. 1–15, 2022, doi: 10.1080/10494820.2022.2150222.
N. Al Said, L. Vorona-Slivinskaya, and E. Gorozhanina, “Data mining in education: managing digital content with social media analytics in medical education,” Interact. Learn. Environ., pp. 1–13, 2023, doi: 10.1080/10494820.2023.2194330.
M. Lindfors and A. D. Olofsson, “The search for professional digital competence in Swedish teacher education policy—A content analysis of the prerequisites for teacher educators’ dual didactic task,” Cogent Educ., vol. 10, no. 2, 2023, doi: 10.1080/2331186X.2023.2272994.
Y. Zhou, B. J. Calder, E. C. Malthouse, and Y. K. Hessary, “Not all clicks are equal: detecting engagement with digital content,” J. Media Bus. Stud., vol. 19, no. 2, pp. 90–107, 2022, doi: 10.1080/16522354.2021.1924558.
H. Liang, U. Ganeshbabu, and T. Thorne, “A Dynamic Bayesian Network Approach for Analysing Topic-Sentiment Evolution,” IEEE Access, vol. 8, pp. 54164–54174, 2020, doi: 10.1109/ACCESS.2020.2979012.
M. Sohi, M. Pitesky, and J. Gendreau, “Analyzing public sentiment toward GMOs via social media between 2019-2021,” GM Crop. Food, vol. 14, no. 1, pp. 1–9, 2023, doi: 10.1080/21645698.2023.2190294.
R. Thomas and J. R. Jeba, “A novel framework for an intelligent deep learning based product recommendation system using sentiment analysis (SA),” Automatika, vol. 65, no. 2, pp. 410–424, 2024, doi: 10.1080/00051144.2023.2295148.
S. Sommariva, J. Beckstead, M. Khaliq, E. Daley, and D. Martinez Tyson, “An approach to targeted promotion of HPV vaccination based on parental preferences for social media content,” J. Soc. Mark., vol. 13, no. 3, pp. 341–360, Jan. 2023, doi: 10.1108/JSOCM-08-2022-0164.
R. Odoom, “Digital content marketing and consumer brand engagement on social media- do influencers’ brand content moderate the relationship?,” J. Mark. Commun., vol. 00, no. 00, pp. 1–24, 2023, doi: 10.1080/13527266.2023.2249013.
M. Arevalillo-Herraez, P. Arnau-Gonzalez, and N. Ramzan, “On Adapting the DIET Architecture and the Rasa Conversational Toolkit for the Sentiment Analysis Task,” IEEE Access, vol. 10, no. September, pp. 107477–107487, 2022, doi: 10.1109/ACCESS.2022.3213061.
U. Naqvi, A. Majid, and S. A. Abbas, “UTSA: Urdu Text Sentiment Analysis Using Deep Learning Methods,” IEEE Access, vol. 9, pp. 114085–114094, 2021, doi: 10.1109/ACCESS.2021.3104308.
A. P. Rodrigues, N. N. Chiplunkar, and R. Fernandes, “Aspect-based classification of product reviews using Hadoop framework,” Cogent Eng., vol. 7, no. 1, 2020, doi: 10.1080/23311916.2020.1810862.
I. Awajan, M. Mohamad, and A. Al-Quran, “Sentiment Analysis Technique and Neutrosophic Set Theory for Mining and Ranking Big Data from Online Reviews,” IEEE Access, vol. 9, pp. 47338–47353, 2021, doi: 10.1109/ACCESS.2021.3067844.
Y. Zheng, Y. Long, and H. Fan, “Identifying Labor Market Competitors with Machine Learning Based on Maimai Platform,” Appl. Artif. Intell., vol. 36, no. 1, 2022, doi: 10.1080/08839514.2022.2064047.
N. S. Mohd Nafis and S. Awang, “An Enhanced Hybrid Feature Selection Technique Using Term Frequency-Inverse Document Frequency and Support Vector Machine-Recursive Feature Elimination for Sentiment Classification,” IEEE Access, vol. 9, no. Ml, pp. 52177–52192, 2021, doi: 10.1109/ACCESS.2021.3069001.
J. L. Arroyo Barrigüete, L. Barcos, C. Bellón, and T. Corzo, “One year of European premiers leadership and empathy in times of global pandemic: a Twitter sentiment analysis,” Cogent Soc. Sci., vol. 8, no. 1, 2022, doi: 10.1080/23311886.2022.2115693.
W. Zhao, X. Yang, and N. Sun, “Do Digital City Policies Promote Corporate ESG Performance?? Evidence from Research on Textual Analysis of China Do Digital City Policies Promote Corporate ESG Performance?? Evidence from Research on Textual Analysis of China,” Emerg. Mark. Financ. Trade, vol. 00, no. 00, pp. 1–20, 2024, doi: 10.1080/1540496X.2024.2331013.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Comparative Analysis of DT and SVM Model Performance with SMOTE in Sentiment Classification
ARTICLE HISTORY
Issue
Section
Copyright (c) 2024 Yerik Afrianto Singgalen
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
