Navigation Menu

Click the "+" to see inside a chapter or use the search to the right.


Skip to end of metadata
Go to start of metadata

ExamDeveloper Tutorial: How to Review and Validate Items

Reviewing Items

First, log in using the User ID given and the password the user set up. After logging in, reviewers see the dashboard for the assigned projects. As shown below, the dashboard lists the projects for which you have been given access and how many items are in the current review pool.


On the dashboard is also where users can update contact information, email address, phone number, password, etc. Click the project name to open the assigned project.

After choosing the project, the Develop Questions and Observe tabs are displayed at the top of the screen.

In order to review items, click the Review link at the top of the page.The Project Review Summary lists the items assigned to the reviewer for the selected project.

Click the Edit  icon to open the assigned items in the Review Questions page.The Review Questions page includes a queue of the items available for review. From this screen the user can read the entire item, options, see any metadata attached, view the references used for each item and see any comments that previous users may have made on this item.

After reading the item, the user can choose one of three options. The item can be Advanced, Returned, or Modified.

Advancing an item: If you like the item and do not wish to change anything, click the Advance button. Advancing an item pulls the question out of the queue of questions for you to review. When an item has been advanced by the appropriate number of reviewers, it is moved on to the step or state in the process, which will be discussed later. After advancing the item, the next item in the queue is shown for you to review.

Users may see the same item multiple times


You may see the same item multiple times. If someone else makes a change to an item which you have reviewed previously, that item reappears in your queue of items to review. This is normal behavior, multiple people need to approve the latest version of the item before it is allowed to be completed, even if the change was small. If you like their change, just click Advance and move to the next item.

Modifying an item: If you decide that the basis of the item is fine but think that there may be some part of it that needs to be changed or corrected, you can choose to modify the item. A modification can be made for any reason, spelling or grammar, actual item content, or change of metadata, reference or blueprint tags. In order to modify the item, click the Modify button. This opens the Modify Question page shown below.

You can make necessary changes to the item. At the bottom of the screen, the grey bars expand when clicking on them. In this area, you can make edits to information depending on the needs of the project. In the screenshot above, the project requires that a reference for this item, indicated by a yellow warning sign. This means that the information can be changed if the reviewer deems it necessary; however it cannot be removed. After the desired changes are made, click the Advance button at the bottom of the page. After the changes are submitted, you are returned back to the Review Questions page with the next item in the queue.

Returnng an item: The third option you have is to return the item. Choosing this option returns the item to the previous review state if you are on a review state past the first review state, or the item is returned to the original author of the item for it to be completely redone. This option should only be used in rare instances, because if an item is returned to the original author, it has to go through the entire review process again, or another item should be written to replace it. If you decide to return an item, you must supply a reason for it.

After entering the comment and submitting, the item is returned back to the author to be changed and resubmitted, or it is returned to a previous review state.

You will continue this process until all of the items in your queue are reviewed. You can then go on to your validation assignment.

Revisit the site often as work required may change


It is important to keep in mind that you may need to revisit this part of the application several times over the course of the project, because other users will alter an item which will require your attention.

Validating Items

The next step in the process is to validate questions that have been approved in the Review stage. To access the validation assignment, click the Validate link at the top of the page.

Similar to the Project Review Summary review assignments page, the Project Review Summary Validate Questions tab shows all of the items within the user's assignment that are ready to be validated. Select the Edit   icon next to the pending assignment.

The Validate Questions page open. It displays the entire item just as it did when the user was reviewing the question. The bottom of the screen is where you will validate the item.

Validate each item by rating it on Importance, Criticality, and Frequency. You must think about the content of each item and think about the function of the job, and then rate how each of these three things apply to the job.

Also on this screen, the validator must ensure that the number of correct options is shown, that there is a reference listed and that the item is classified to the correct blueprint area.

The final you are asked to complete in validation is the Angoff rating. This is asking if there were 100 minimally competent people (just good enough to be considered competent on the job) doing this job and they were asked this question, how many would get this question correct? Typically, the ratings range from random chance (very hard) to 100 (very easy).

After those questions are answered, you can then Submit the item. Again, just as with the review stage, hitting submit brings up the next item in the queue that is ready to be validated. This completes the validation state and moves the item out of the queue.


  • No labels