Helping software engineering students analyzing their performance data: tool support in an educational environment

ABSTRACT

ProcessPAIR is a novel tool for automating the performance analysis of software developers. Based on a performance model calibrated from the performance data of many developers, it automatically identifies and ranks potential performance problems and root causes of individual developers. We present the results of a controlled experiment involving 61 software engineering master students, half of whom used ProcessPAIR in a performance analysis assignment. The results show significant benefits in terms of students' satisfaction (average score of 4.78 out of 5 for ProcessPAIR users, against 3.81 for other users), quality of the analysis outcomes (average grades achieved of 88.1 out of 100 for ProcessPAIR users, against 82.5 for other users), and time required to do the analysis (average of 252 min for ProcessPAIR users, against 262 min for other users, but with much room for improvement).

References

1. Hackystat Development Team, Hackystat {online}, available: http://code.google.com/p/hackystat/ (last release: January, 2010; last visited: February 2017).
2. Tuma Solutions LLC, Process Dashboard {online}, available: http://www.processdash.com (last release: December, 2016; last visited: February 2017).
3. H. Shin, H. Choi, and J. Baik, "Jasmine: A PSP supporting tool," Proc. of the Int. Conf. on Software Process (ICSP 2007), LNCS 4470, Springer-Verlag, 2007, pp. 73--83
4. S. Thisuk and S. Ramingwong, "WBPS: A new web based tool for Personal Software Process," 2014 11th Int. Conf. on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, IEEE, 2014.
5. M. Raza and J. Faria, "ProcessPAIR: A tool for automated performance analysis and improvement recommendation in software development," Proc of the 31st IEEE/ACM Int. Conf. on Automated Software Engineering (ASE 2016), ACM, 2016, pp. 798--803
6. W. Humphrey, PSP^SM: A Self-Improvement Process for Software Engineers, Addison-Wesley Professional, 2005.
7. G. Rong, H. Zhang, S. Qi, and D. Shao, "Can engineering students program defect-free? An educational approach," Proc. of the 38th Int. Conf. on Software Engineering Companion (ICSE '16), ACM, 2016, pp. 364--373
8. M. Raza and J. Faria, "A model for analyzing performance problems and root causes in the Personal Software Process," J. of Software: Evolution and Process, vol. 28, issue 4, April 2016, pp. 254--271.
9. M. Raza, J. Faria, J., and R. Salazar, "Empirical evaluation of the ProcessPAIR tool for automated performance analysis," 28th Int. Conf. on Software Engineering and Knowledge Engineering (SEKE 2016), July 2016