The Standard Setting component is ExamDeveloper's “best practice” method of setting a passing standard for an examination. The standard setting process is performed by a group of Subject Matter Experts (SME) and is best performed while everyone is on the system at the same time.
When an Organization sets a passing standard for an exam form, there are several factors they consider, including:
- What is the difficulty of the exam form? SMEs developing a high-stakes exam usually write items that span a wide range of difficulty, so it is advisable to document a quantitative evaluation of the difficulty of the exam form.
- What is the minimum level of performance that can be considered "competent"?
- What is the experience level of the target population with the material being tested?
- The process is performed using a representative committee of SMEs, typically ranging between eight and fifteen members.
Certification exams assess the required knowledge and skills of candidates in regards to professional performance, so it is critical that the passing standard be determined through considered analysis.
- There are established methods for determining passing standards: the Angoff Method, the Bookmark Method, Ebel's Method, and more.
- A methodical approach to the process ensures a reproducible, quantifiable examination passing point.
- ExamDeveloper uses the Angoff Method to establish a defensible passing standard.
ExamDeveloper allows you to set the passing standard for an exam form in the following three steps:
- Take the Exam: The SMEs take the examination exactly as a candidate would do.
- Rate the Difficulty: For each item, the SME sees the correct answer, what they chose as a response, and the percent of other SMEs who answered the item correctly. They then estimate the number of minimally competent people who would be able to answer it correctly.
- Review Ratings: The SME then evaluates his/her ratings in comparison to other SME ratings for each item. If the SME feels his/her rating for an item is overly harsh or lenient, he/she have the opportunity to adjust the ratings.
Set standards for an exam form
Select the Create Exam Form → Set Standard menu. The Summary tab opens on the Exam Listing page.
The Exams Listing page includes the following features:
Exam Form Name
The Name of the exam form.
Name of the person who created the exam form.
Indicates whether the exam is being drafted or has been published.
Click Set Standard to start the process of setting standards.
|Take Exam||Click Take Exam to start the Take Exam process.|
|Report||Run a report with the ratings of the users.|
Functionality of the Report button
Clicking the Report button downloads an Excel file with Set Standard data for exams, which have been rated by SME(s) via the Set Standard process.
The report consists of a set of similar columns for each SME:
- Question Number: This is the Question ID of items in the exam form.
- Response of SMEx: This is the Answer Option chosen for the item by the SME.
- Final Difficulty Rating by SMEx: This is the Difficulty Rating assigned to an item by the SME.
Comment of SMEx: This is the comment (if any) added by the SME to an item.