Select the Manage → Setting → Review Workflow menu.
Allow users to review their own Questions
Select Yes to allow authors to review Questions they have written.
Number of reviewers needed to advance a Question
The number of Reviewers who must advance a Question in its current form before it is approved. Allowed values range from 0 to 15; the default is 3.
Setting this to 0 causes the review process to be entirely skipped.
Number of reviewers needed to return a Question
The number of Reviewers who must return a Question in its current form before it is returned to the author. Allowed values range from 1 to 15; the default is 3.
Require reviewers to classify unclassified Questions to level
Choose Yes if Reviewers should ensure the Question is correctly classified, and then select the level to which the Question should be classified from the drop-down list.
Require Comment on modifying Question
Select Yes if the Reviewer should record/explain changes made on the Create/Edit Question page.
|Allow Reviewers to Edit|
Allow or disallow users with Reviewer permission to edit the following areas: Item Text (Stem and Option text), Blueprint, Metadata, and Reference.
Each of these areas has check boxes next to them, which grant edit permissions to Reviewers if selected and deny them if not selected. All the check boxes are enabled by default, thus allowing Reviewers to edit all these areas unless manually deselected by the Manager.
|Show 'Retain Statistics' pop-up warning for modified items|
The pop-up to confirm retaining statistics while trying to save a Question with statistics after modifying for a Review State will not be shown, if this check box is not selected, and the statistics will be retained.
This check box is selected by default, meaning the pop-up to confirm retaining statistics appears while modifying Questions with statistics.
This check box is grayed out and the statistics are retained while saving a Question with statistics after modifying for a Review State, if the Show 'Retain Statistics' pop-up warning for modified items check box at the Project level is not selected.
|Allow reviewers to check-out items|
Allow or disallow users with Reviewer permission to manually check out a question for 1 day, 1 week, or 1 month.
This setting enables a Check-out button on the Review Question page. when the reviewer clicks the Check-out button, the question remains checked out for the number of days specified in the drop-down list. unless the user checks in the question. ExamDeveloper sends out an email reminder one day prior to automatically checking in the question. If the user does not check in the question before the expiration date, ExamDeveloper checks in the question at 3 am server time on the expiration date.
This manual check-out prevents other users from editing the question as long as it is checked out. When they get to the Review Questions page for a question that is checked out, a message notifies the reviewer which user has the question checked out.
|Outcomes||The Outcomes list what happens when you advance or return a question. There can be multiple advance and return states. As of the 1811 release, you are not able to specify if Comments are required, or specify whether Statistics are required. In a future release the Statistics and Comments may be modified.|
|Only Managers can access this page to modify these settings.|
Once Review States have been added to the Item Bank, you can add them to a Project. See Review State for details on adding a review state to an Item Bank.
When you add or move review states, a message notifies you how the review states have changed as a result of saving the changes. In this example, previously Review One had two possible outcomes: Return, which sent the Question back to the Original Author, and Advance, which sent the Question to the Validate state. After adding Review Two and Review Three there are still two possible outcomes: Return, which sends the Question back to the Original Author, and Advance, which now sends the Question on to Review Two. See Add Review Outcomes below for details on how to add more review outcomes.
If only one review state is configured, by default the Advance outcome moves the Question to the Validate state, and the Return outcome moves the Question back to the original author. If three review states are configured, by default each of the review states are configured as follows:
Outcomes can be added or deleted from each review state.
Prior to 1811, there was only a single linear path to forward a question. When reviewers Accepted or Advanced a Question, it went to the next review state. The final reviewer could determine who to reject or return the question to.
As of 1811 Project Managers can add multiple outcomes for users to forward or return questions. Only the final reviewer is given the option to forward or return the question and select the next review state or select who to return the question to. Each review state may have multiple Advance outcomes and multiple Return outcomes.
If there is only one Advance outcome for the final reviewer, the Question automatically advances to the next review state when they click Advance.
If there are multiple Advance outcomes for the final reviewer, they must select who to advance the Question to from the Outcome drop-down list on the Review or Modify Question page.
If there are no outcomes where the next state is a Return state, the Return button is hidden on the Review or Modify Question page.
You may have a need to rearrange the order of the review states. If Questions have already gone through review states, the historical information on the question is retained.
To save changes click Save, or to discard the changes click Cancel.
If you add, remove, or re-sequence review states, all outcomes for a review state are recalculated and any manual outcomes added to the review state are lost and must be recreated.