Skilled developers are important to the software industry. In empirical studies in software

engineering, knowing the skill level of the participants is also important for correct

interpretation of results. The current practice in industry and research for assessing

programming skills is mostly to use proxy variables of skill, such as education, experience,

and multiple-choice knowledge tests. There is as yet no valid and efficient way to measure

programming skill. Consequently, this thesis aimed to construct a valid instrument for

measuring programming skill, where skill is inferred from performance on programming

tasks.

The Rasch measurement model was used to construct the instrument. Sixty-five

professional developers from eight countries participated in validating the instrument,

solving 19 Java programming tasks over two days. The validity of the instrument was

theoretically investigated through commercial and research-based tests. Programming

skill, as measured by the instrument, was also investigated in terms of experience and

other background variables.

The instrument was found to have desirable psychometric properties, and the overall

results appear well aligned with theoretical expectations. This work has shown that

acceptable measures of programming skill may be obtained with less than one day of

testing. Further work should be directed at reducing the time needed to measure programming

skill without a↵ecting the validity of the instrument. The results of the research

have already been transferred to the industry through a commercial prototype.