Faculty Development

How are you going to measure whatever it is you need to measure? To some extent we have dealt with this throughout the module: when you know what it is you want to assess you need to find a valid and reliable tool to do the job. If you are measuring blood pressure or wound infections or various aspects of the immune system, this can be easy, but if you are considering opinions, feelings, interactions or communication, you may need to hunt for or even develop instruments. Typical instruments used in education research are interview or focus group schedules, psychometric tests, questionnaires, observation schedules and standardised rating sheets. Never design your own instrument when a perfectly good one already exists – the development of measures is a time-consuming and costly job if it is to be done properly with all the issues of reliability, validity and piloting taken care of. Doing a search on the topic usually reveals a number of good instruments that can be used with or without modification.

Instruments are reliable (replicable) to the extent to which using them on the same individual at different times or in different circumstances still produces a relatively similar result. This test–retest correlation should be at least 0.5 for groups and hopefully above 0.8 for individuals. Instruments are valid when they measure what they set out to measure. With face validity the measure should simply make sense to individuals as a reasonable way to assess what it is you want to measure. Criterion validity assesses the extent to which the measure correlates with other existing measures – for example, that a new measure of depression or quality of life has a high correlation with another measure for the same purpose. Construct validity measures whether an instrument relates in a theoretically predictable way to another established variable, for example that quality of teaching relates to years of experience, or that quality of life relates to adequacy of housing.