Setting the Pass Mark: Characteristics of Credible Standards
In this section, we will discuss the various methodologies associated with standard setting. Whichever method you feel is appropriate, make sure you determine that the method is defensible given the format of your exam, is credible, is supported by body of evidence in the testing literature, is feasible for your program’s characteristics, and finally, that it’s acceptable to your program’s stakeholders.
First, we’ll talk about absolute methods of standard setting studies. Examples of Absolute Standard Setting Methods are:
Advantages and Disadvantages of Absolute Methods
- They focus attention on item content
- They are relatively easy to use
- There is a considerable body of published work supporting their use
- They are used frequently in high stakes testing
- The concept of a "borderline group" is sometimes difficult to define
- Judges sometimes feel they are "pulling numbers out of the air"
- The methods can be tedious