Navigation Menu

Click the "+" to see inside a chapter or use the search to the right.

.

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: clean up

...

Access the Review Workflow page

...

...

  1. Select the Manage → Setting → Review Workflow menu.

...

  1. Image Added

Image Removed


  1. Option

    Description

    Allow users to review their own

...

  1. Questions

    Select Yes to allow

...

  1. authors to review

...

  1. items they have written.

    Number of

...

  1. reviewers needed to

...

  1. advance a Question

    The number of

...

  1. Reviewers who must

...

  1. advance an item in its current form before it is approved. Allowed values range from 0 to 15; the default is 3.

Info

A reviewer who makes a modification to a question counts as its first approver.

  1. Setting this to 0

...

  1. causes the review process to be entirely skipped.

    Number of

...

  1. reviewers needed to

...

  1. return a Question

    The number of

...

  1. Reviewers who must

...

  1. return an item in its current form before it is returned to the author

...

  1. . Allowed values range from 1 to 15; the default is 3.

    Require reviewers to classify unclassified

...

  1. Questions to level

    Choose Yes if

...

  1. Reviewers should ensure the

...

  1. item is correctly classified, and then select the level to which the

...

Require Comment on modifying question

...

  1. item should be classified from the drop-down list.
    Lowest Level in the drop-down list indicates that in a blueprint of varying depth, the lowest level must be selected irrespective of the depth.

Info

This can not be set to None, with the Number of people needed to accept a question to approve it as 0.

  1. Warning

    When Yes is selected for Require reviewers to classify unclassified Questions to level setting on the Review Workflow page and an item with one of the Review States having Unclassified Blueprint or Blueprint level lower than the one specified in the drop-down setting on that page is opened on the Examine Questions page, then the following error message is displayed in red text. The State of such items cannot be changed to anything other than one of the available Review States or to the Obsolete state.

    Image Added

    Require Comment on modifying Question

    Select Yes if the Reviewer should record/explain changes made

...

  1. on the Create/Edit Question page.

    Allow Reviewers to Edit

    Allow or disallow users with Reviewer permission to edit the following areas:

...

  1. Item Text (Stem and Option

...

  1. text),

...

  1. Blueprint, Metadata, and Reference.

...

  1. Each of these areas has

...

  1. check boxes next to them, which grant edit permissions to Reviewers if

...

  1. selected and deny them if

...

  1. not selected. All the

...

  1. check boxes are enabled by default, thus allowing Reviewers to edit all these areas unless manually

...

  1. deselected by the Project Manager.

    Note
    A Project Manager

...

  1. is also

...

  1. bound by the permissions of a Reviewer on the pages affected by the above settings.

     

    Warning

...

  1. An item cannot be modified by a Reviewer if all the

...

  1. check boxes are

...

  1. not selected and an error message

...

  1. is displayed in red text.

    Image Added 

...

 

 

  1. Warning

    A Reviewer

...

  1. is able to partially modify

...

  1. an item if check boxes against some of the above items are

...

  1. not selected, thus allowing modifications to only certain areas of the

...

  1. item. A message in red text

...

  1. informs the Reviewer about

...

  1. areas

...

  1. that cannot be edited.

    If

...

  1. metadata and

...

  1. blueprint are set to

...

  1. not editable, then

...

  1. a disallowed icon Image Added (red circle icon with a diagonal slash across it)

...

  1. is displayed just after the configuration name. Moving the mouse cursor over these icons

...

  1. shows the message

...

  1. Manager has disabled editing this content within the current review state

...

  1. .

...

  1. Image Added

    Warning

    An error message in red

...

  1. is displayed if you try to disallow Reviewers to edit Metadata in Review Workflow when it is set to Mandatory under Manage → Settings → Metadata.

...

  1. If there are multiple review states, then at least one review state must allow the Reviewers to edit metadata.

    Image Added

    Warning

    When the

...

  1. check box next to Blueprint is removed in Review Workflow, meaning

...

  1. it cannot be edited by Reviewers, and

...

  1. you try to enforce a

...

  1. blueprint level against Require Reviewers to classify unclassified

...

  1. Questions to level, then you get an error message in red

...

  1. .

...

  1. Image Added

    Warning

    When you clear the

...

  1. check box for Reference in Review Workflow so that Reviewers should not be able to edit

...

  1. references in

...

  1. items, you will get an error in red

...

  1. if it has been set as mandatory (

...

  1. check box next to Questions must have references is enabled) in the  Questions tab of Manage → Settings → Configuration.

 Image Removed

Warning

If an area such as Metadata is set as mandatory, but has been set to be not editable by Reviewers, and the Metadata field of a Question is empty, then you will see an icon Image Removed (yellow rhombus with an exclamation mark within it) at the centre of the affected area. Moving the mouse cursor over it pops out the text "Some values in this section may be required", and this area can be edited.

 Image Removed

  1. Image Added

    Show 'Retain Statistics' pop-up warning for modified items

    The pop-up to confirm retaining statistics while trying to save an item with statistics after modifying for a review state is not shown, if this check box is not selected, and the statistics are retained.

    This check box is selected by default, meaning the pop-up to confirm retaining statistics appears while modifying items with statistics.

    This check box is grayed out and the statistics are retained while saving an item with statistics after modifying for a review state, if the Show 'Retain Statistics' pop-up warning for modified items check box at the Project level is not selected.

    Allow reviewers to check-out items

    Allow or disallow users with Reviewer permission to manually check out an item for 1 day, 1 week, or 1 month.

    Image Added

    This setting enables a Check-out button on the Review Question page. When the reviewer clicks the Check-out button, the item remains checked out for the number of days specified in the drop-down list, unless the user checks in the item. ExamDeveloper sends out an email reminder one day prior to automatically checking in the item. If the user does not check in the item before the expiration date, ExamDeveloper checks in the item at 3 am server time on the expiration date.

    Image Added

    This manual check-out prevents other users from editing the item as long as it is checked out. When they get to the Review Questions page for an item that is checked out, a message notifies the reviewer which user has the item checked out.

    Image Added

    OutcomesThe Outcomes list what happens when you advance or return an item. There can be multiple advance and return states.
    Note
    Only Project Managers can access this page to modify these settings.
  2. After selecting all information, click Update to save changes or Cancel to discard changes.

To select review states:

...

Add Review States to a project

Once Review States have been added to the Item Bank, you can add them to a project. See Review State for details on adding a review state to an Item Bank.

  1. Click the Select review states button. The page with a list of Available Review states in the current Item Bank and the Current Order

...

  1.  of the review states is displayed.
    Image Added
  2. Select the check boxes for the review states you wish to include in your project workflow.
  3. To save changes click Save, or to discard the changes click Cancel.
  4. When you add or move review states, a message notifies you how the review states have changed as a result of saving the changes. In this example, previously Review One had two possible outcomes: Return, which sent the item back to the Original Author, and Advance, which sent the item to the Validate state. After adding Review Two and Review Three there are still two possible outcomes: Return, which sends the item back to the Original Author, and Advance, which now sends the item on to Review Two. See Add Review Outcomes below for details on how to add more review outcomes.
    Image Added

    Info

    If only one review state is configured, by default the Advance outcome moves the item to the Validate state, and the Return outcome moves the item back to the original author. If three review states are configured, by default each of the review states are configured as follows:

    • Review One: Advance moves the item to Review Two, and Return moves the item to the Original Author
    • Review Two: Advance moves the item to Review Three, and there are two possible Return outcomes - the item can be returned to Review One, or to the Original Author
    • Review Three: Advance moves the item to the Validate state, and there are three possible Return outcomes - the item can be returned to Review Two, Review One, or to the Original Author

    Outcomes can be added or deleted from each review state.

Anchor
Add Review Outcomes
Add Review Outcomes

Prior to 1811, there was only a single linear path to forward an item. When reviewers Accepted or Advanced an item, it went to the next review state. The final reviewer could determine who to reject or return the item to.

Image Added

As of 1811 Project Managers can add multiple outcomes for users to forward or return items. Only the final reviewer is given the option to forward or return the item and select the next review state or select who to return the item to. Each review state may have multiple Advance outcomes and multiple Return outcomes.

Image Added

Info

If there is only one Advance outcome for the final reviewer, the item automatically advances to the next review state when they click Advance.

If there are multiple Advance outcomes for the final reviewer, they must select who to advance the item to from the Outcome drop-down list on the Review or Modify Question page.

Image Added

If there are no outcomes where the next state is a Return state, the Return button is hidden on the Review or Modify Question page.

Image Added

 

  1. Click the Add new outcome button for the desired review state.
    Image Added
  2. Click the Choose state drop-down list and select the desired outcome from the list.
    Image Added
    The new Outcome is added to the list.
    Image Added
  3. All Return Outcomes require reviewers to add a comment explaining why they are returning the item. You can specify if Comments are required for any of the other Outcome options. Click the Comments drop-down list for the newly added Outcome and select Required.
    Image Added
  4. For each of the Outcome states, you can specify whether to Always retain statistics or let the reviewer decide by selecting Always retain.
    Image Added
  5. Click Update. The Outcomes are updated for the review state. 
    Image Added

Delete Review Outcomes

  1. Click the Delete Image Added icon for the outcome you wish to remove from the review state. The outcome is removed from the review state.
  2. Click Update to save the changes.

Re-sequence Review States

You may have a need to rearrange the order of the review states. If items have already gone through review states, the historical information on the item is retained.

  1. Click the Select review states button. The page with a list of Available Review states in the current Item Bank and the Current Order of the Review States is displayed.
    Image Added
  2. You can change the order by

...

  1. clicking the Move Up and Move DownImage Added arrows.
    Image Added
  2. To save changes

...

  1. click Save, or to discard the changes

...

  1. click Cancel.

To delete review states:

...

  1. Note

    If you add, remove, or re-sequence review states, all outcomes for a review state are recalculated and any manual outcomes added to the review state are lost and must be recreated.

Disable Review States

  1. Select Select review states. The page with the list of Available Review states and the Current Order for that

...

  1. Review State is displayed.

...

  1. Deselect the Image Modified

...

  1.  check box beside the

...

  1. Review State you would like to

...

  1. disable for the current project and click Save. The

...

  1. Review State is disabled for the current project and is no longer visible on the Review Workflow page.

...


...

  1. Image Added
  2. Then click Update to configure the changes.
    If the

...

  1. Review State you want to delete has any assignments, it

...

  1. shows an error message: Please delete the assignments for the states to delete.

To delete these assignments: Go to ManageUsers Assign Question → Assign Work to Users page, and click the Image Removed Delete icon.

Image Removed

Now follow the steps 1 to 3.

It will show a Warning message for deleting review states as shown below.

Image Removed

Click Delete to update changes and click Cancel to discard the changes.

  1. Image Added