Exploring Machine Learning to Support Software Managers' Insights on Software Developer Productivity

Abstract

Software developer productivity is a complex issue with no single, universally accepted definition or measurement. Emerging technologies like machine learning offer a promising opportunity for more accurate productivity measurement. Semi-structured interviews were conducted to gain qualitative insights into software managers’ perception of developer productivity to identify issues and inform the development of applied machine learning solutions. It was discovered that digital distractions significantly hinder developer productivity and conventional methods to monitor developer activity were often inefficient. Therefore, machine learning models were developed to monitor developer activity by classifying screenshots captured during activity, along with the URL and text content scraped from accessed URLs. Train and test data were obtained from a cooperating software house, supplemented with online sources. For screenshot classification, transfer learning using EfficientNetV2B0 outperformed InceptionV3, Resnet50V2, and VGG16, reaching 99.6% accuracy. This was achieved without fine-tuning, which resulted in the fastest training and lowest resource consumption. For content classification, SVC hyperparameter-tuned using grid search outperformed six other classifiers, reaching 88.5% accuracy. The design concept for a web application that utilizes the developed models to help managers measure developer productivity was well-received by the managers interviewed.