Sentimental Analysis of Legal Aid Services: A Machine Learning Approach

Abstract

Legal Aid services in South Africa, administered by Legal Aid South Africa (SA), aim to provide essential legal representation to vulnerable individuals lacking financial resources. Despite its significant role, there is a pervasive perception among the public that the quality of these state-funded services is substandard, often leading to negative attitudes towards the organization. This research employs sentiment analysis to evaluate client perceptions of Legal Aid SA's services, using a dataset of 5,246 entries from Twitter and the Internal client feedback system between 2019 and 2024. The study utilizes various machine learning algorithms, including Naive Bayes, Stochastic Gradient Descent (SGD), Random Forest, Support Vector Classification (SVC), Logistic Regression, and Extreme Gradient Boosting (XGBoost), to analyze sentiment polarity and classify feedback into positive, neutral, and negative sentiments. The accuracy, precision, recall, and F1 scores assessed model performance. The SVC and XGBoost models demonstrated superior performance, achieving testing accuracies of 90.10% and 90.00%, respectively. In contrast, Naive Bayes and Logistic Regression lagged, with test accuracies of 82.00% and 85.00%, respectively. The findings reveal that most responses are either neutral or positive, suggesting a predominantly favourable impression of Legal Aid services. This research not only aims to enhance Legal Aid SA's service offerings but may also provide valuable insights for similar organizations globally.