AuthorsG. R. Bergersen, J. E. Hannay, D. I. K. Sjøberg, T. Dybå and A. Karahasanovic
TitleInferring Skill From Tests of Programming Performance: Combining Time and Quality
AfilliationSoftware Engineering, Software Engineering
Publication TypeProceedings, refereed
Year of Publication2011
Conference NameProc. 5th Int'l Symp.Empirical Software Engineering and Measurement (ESEM)
PublisherIEEE Computer Society
ISBN Number978-1-4577-2203-5

The skills of software developers are crucial to the success of software projects. Also, controlling for individual differences is important when studying the general effect of a method or a tool. However, the way skill is determined in industry and research settings is often ad hoc or based on unvalidated methods. According to established test theory, validated tests of skill should infer skill levels from welldefined performance measures on multiple small, representative tasks. We show how time and quality can be meaningfully combined as task performance, and subsequently aggregated into an approximate measure of programming skill. Our results show significant and positive correlations between our proposed measures of skill and variables such as seniority, lines of code written, and self-evaluated expertise. The method for combining time and quality are a promising first step to measuring programming skill in industry and research settings.

Citation KeySimula.prepare.14