An Approach for Emotion Detection in Natural Arabic Audio Files Based on Acoustic and Lexical Features

Abstract

Emotion Detection is a crucial for enhancing human-machine interactions. This paper addresses the challenge of accurately recognizing emotional states from speech, particularly in distinguishing between emotions with similar acoustic characteristics, such as anger, happiness and surprise, which have high pitch and energy. While acoustic features convey significant information about emotional states, they are often inadequate for distinguishing between these emotions. This limitation highlights the need for improved performance in emotion detection systems. The main contribution of this work is the introduction of a multimodal approach that combines both acoustic and lexical features for emotion detection in natural Arabic audio files, focusing on four emotions anger, happiness, sadness and neutral. To the best of our knowledge, this is the first study that employ such a combination in this context, building on our previous work that utilized only acoustic features. Several Machine Learning (ML) classifiers were applied including Sequential Minimal Optimization (SMO), Random Forest (RF), K-Nearest Neighbors (KNN), and Simple Logistic (SL). Two types of experiments were executed: one using only lexical features and another combining various acoustic features sets with lexical features. This approach enhances our previous experiments that used only acoustic features. The experimental results show that SMO classifier achieved the highest performance, with an accuracy 96.11% when using all acoustic features combined with a unigram model, outperforming the other classifiers. These results suggest that combining acoustic and lexical features enhances the performance of emotion detection models, particularly for complex emotions in natural Arabic audio datasets.