In contrast to absolute methods, compromise methods draw on the information in the observed score distribution for a test to adjust the standard.
- Easy to implement
- Educators are comfortable with the decisions
- The cut score may not be in the area defined by the judges’ estimates
- The method is not the first choice in a high stakes testing situation
Judges review the exam and determine the following information:
- Lowest acceptable percentage of failing candidates (minimum failure rate)
- Highest acceptable percentage of failing candidates (maximum failure rate)
- Lowest percent correct which allows someone to pass (minimum passing score)
- Highest percent correct required for someone to pass (maximum passing score)
The expectations of the judges are averaged to determine the four values that are used. Those four values are plotted to create a rectangle (see image).
After administration of the exam, the candidates’ performance scores are plotted on the X axis (number correct) and Y axis (number wrong) resulting in a curve of candidate percent correct scores or ability estimates.
A diagonal line is drawn from the upper left to lower right of the rectangle. The point where the diagonal line intersects the plotted curve is the pass point.
A specific example of a compromise structure is the Contrasting Groups Method. The principal focus of judgment in the contrasting groups’ method is on the competence of candidates rather than on the difficulty of an exam or its items.
It is appropriate for dichotomous and polytomous exam formats. Dichotomous means that the candidate gets the answer right or they do not. Polytomous means that the candidates can receive varying degrees of credit based on the response. An example is a Multiple-Choice item that instructs the candidate to select all options that apply.
The contrasting-groups method is based on the idea that the test takers can be divided into two contrasting groups, a qualified group and unqualified group, on the basis of the judgments of their knowledge and skills. An obvious choice for a passing score in this case would be the score at which there is just as many qualified test takers as unqualified test takers. So, the point at which the two distributions intersect is the location of the passing score.