- Easy to implement
- Educators are comfortable with the decisions
- The cut score may not be in the area defined by the judges’ estimates
- The method is not the first choice in a high stakes testing situation
Judges review the examination exam and determine the following information:
- Lowest acceptable percentage of failing candidates (minimum failure rate)
- Highest acceptable percentage of failing candidates (maximum failure rate)
- Lowest percent correct which would allow allows someone to pass (minimum passing score)
- Highest percent correct required for someone to pass (maximum passing score)
The expectations of the judges are averaged to determine the four values that will be are used. Those four values are plotted to create a rectangle (see image).
Image RemovedImage Added
After administration of the exam, the candidates’ performance scores are plotted on the X axis (number correct) and Y axis (number wrong) resulting in a curve of candidate percent correct scores or ability estimates.
A specific example of a compromise structure is the Contrasting Groups Method. The principal focus of judgment in the contrasting groups’ method is on the competence of examinees candidates rather than on the difficulty of a test an exam or its items.
It is appropriate for dichotomous and polytomous exam formats. Dichotomous means that the candidate gets the answer right or they do not. Polytomous means that the candidates can receive varying degrees of credit based on the response. An example is a MC Multiple-Choice item that instructs the candidate to select all options that apply.
The contrasting-groups method is based on the idea that the test takers can be divided into two contrasting groups, a qualified group and unqualified group, on the basis of the judgments of their knowledge and skills. An obvious choice for a passing score in this case would be the score at which there is just as many qualified test takers as unqualified test takers. So, the point at which the two distributions intersect is the location of the passing score.