Benchmarking and comparison of software project human resource allocation optimization approaches

For the Staffing and Scheduling a Software Project (SSSP), one has to find an allocation of resources to tasks while considering parameters such skills and availability to identify the optimal delivery of the project. Many approaches have been proposed that solve SSSP tasks by representing them as optimization problems and applying optimization techniques and heuristics. However, these approaches tend to vary in the parameters they consider, such as skill and availability, as well as the optimization techniques, which means their accuracy, performance, and applicability can vastly differ, making it difficult to select the most suitable approach for the problem at hand. The fundamental reason for this lack of comparative material lies in the absence of a systematic evaluation method that uses a validation dataset to benchmark SSSP approaches. We introduce an evaluation process for SSSP approaches together with benchmark data to address this problem. In addition, we present the initial evaluation of five SSSP approaches. The results shows that SSSP approaches solving identical challenges can differ in their computational time, preciseness of results and that our approach is capable of quantifying these differences. In addition, the results highlight that focused approaches generally outperform more sophisticated approaches for identical SSSP problems.