Automated Brain Tumor Analysis with Multimodal Fusion and Augmented Intelligence

Abstract

Brain tumor segmentation and classification are critical tasks in medical imaging, having a major impact on spotting and treating brain tumors. In the medical field, augmented intelligence has garnered a lot of attention lately since it emphasizes how human knowledge and artificial intelligence can be combined to enhance efficiency and decision-making in applications like brain tumor identification. This research concentrates on developing a novel approach utilizing Attention U-Net and Multimodal Transformers to assist doctors with precise tumor segmentation and classification while maintaining their critical clinical judgment. Attention U-Net is used to segment brain tumor because it efficiently collects detailed spatial data while focusing on key locations compared with traditional U-Net models. Multimodal Transformers provide reliable as well as effective feature extraction when utilized for early fusion to merge data from many modalities, such as T1, T2, and FLAIR This work utilizes CycleGAN-based data augmentation to supplement limited training data, thus improving the variety and quality of the dataset. The fused multimodal features are then utilized for the segmentation of the tumor and further classified as benign and malignant using hybrid transformer. The performance of the proposed system is assessed using standard metrics like accuracy for classification and Dice Similarity Coefficient and Intersection Over Union for segmentation. The proposed approach demonstrates high effectiveness in both segmentation and classification tasks, achieving 98 % accuracy showcasing its potential as a process innovation for clinical applications.