Navigation Menu

Click the "+" to see inside a chapter or use the search to the right.

.

Skip to end of metadata
Go to start of metadata

The Custom Report displays information related to the current version of items such as state, text, usage, metadata, validation data, statistics, and enemies. The report can display this data for multiple items when they are selected on the Search Questions page and you click the Generate Report button.

  1. Navigate to Manage → Project Bank to open the Search Questions page.
  2. Retrieve the items for which you wish to generate the Custom Report, or simply select the required items from the search result via corresponding check boxes.

    Icon
     How does the selection and pagination of search results work?

    Users can optimize the number of search items per page by selecting a value from the Items Per Page drop-down; this number ranges from 10 to 50 per page. The default number is 20.

    The selected number of search items is displayed in the grid below the drop-down.

    The search items per page can be changed and viewed at any time for the user's convenience.

    The search results are listed on multiple pages according to the selected number of Items Per Page in the drop-down, if the total number of search items resulting from the search criteria exceeds the selected number of items per page.

    The user can also directly jump to another page of the search results grid by clicking its corresponding link above the search results grid.

    The Selection drop-down in the left of the header row has four selection options: None, Some, Page, and All.

    This Selection drop-down is present on pages in which users need to select multiple search items for use such as viewing, modifying, etc.

    The Selection drop-down options perform the following functions:

    ColumnDescription
    NoneAll items in the list across all pages of the search results grid are deselected.
    Page

    All items on the currently visible page of the search results grid are selected.

    If you move to another page of the search results grid, you will see that none of the items on that page are selected, but the items selected on the earlier page are still retained.

    Some

    This selection appears when you manually select a few items from the list by selecting their corresponding check boxes. This selection also appears if you select Page from the drop-down and then increase the Items per page, or if you select All from the drop-down and then deselect any item on the search results page.

    It is also possible to select items across multiple pages of the search results grid by simply making the selection on one page and moving on to another - the item selection made on the earlier page is retained even if you move to another page to make additional selections.

    All

    All items in the list across all pages of the search results grid are selected.

    Even when moving to any other page of the search results grid, all the items on that new page are also selected.

    The selection drop-down remains as All if any of the items are not deselected, and it changes to Some if any of the items on any of the pages of the search results grid are deselected. Furthermore, if the deselection is made on the first page of the search results grid, then all the items on other pages are deselected. However, if you move to another page of the search results grid while the selection drop-down is All and deselect any item, the selection drop-down for that page changes to Some, while all items on the remaining pages remain selected with the selection drop-down selection showing as Page on those pages.



  3. Click the Generate Report button, you are taken to the Custom Reports page, where you can select Custom ReportBlueprint Validation Angoff Information by Item, or Question Longitudinal Analysis report from the Select Report drop-down. Select Custom Report here.

  4. Select one of the four formatting options available in the drop-down list for this report:
    Excel plain textText where the Rich Text Editor has been used is presented in the rendered form (plain text). This report is generated in the Excel format on the Queue page.
    Excel with HTML markupText where the Rich Text Editor has been used is presented in the form of HTML code. This report is generated in the Excel format on the Queue page.
    HTML formatted for the web browserText where the Rich Text Editor has been used is presented in the rendered form (plain text). This report is generated in the HTML format on the Queue page.
    JSONThis report is generated in the JSON format on the Queue page.
         See JSON Report Format topic for details on the JSON formatting option.
  5. Select the type of data to include in the report from the following:
    ■ Default: This is preselected and cannot be deselected. Data includes: Blueprint ID, Blueprint name, External ID, Language, Latest revision, Other blueprints, Question ID, State, Translation group, Type, Written by, Written on.
    ■ Content: Number of options, Option assets, Option content, Option label, Option shape label, Stem assets, Stem content.
      All but the following item type contents can be exported in a custom report: External item, Spreadsheet.
    ■ Key: Select the Key option to include the 'correct options' in the custom report.
    ■ Metadata: Comments, Metadata, References.
    ■ Associations: Case ID, Case name, Cloned from, Cloned to, Collections, Enemies, Exam forms, Exhibit reference, Parent question ID, Sections.
    ■ Statistics: Analysis set name, Candidates, Custom statistics, Difficulty, Discrimination, IRT, Option candidates, Option custom statistics, Option discrimination, Option score. This also requires selection of Analysis Group and type of Analysis Set (Default, Most Candidates, or Latest).
    ■ Validation: Average angoff, Average criticality, Average frequency, Average importance, Number of raters, Has one correct answer?, Has one verifiable reference?, Has correct exam blueprint?, Raters.
    ■ Include items referenced as exhibits: include contents of all the items referenced in the exhibits.

    See the Custom Report topic for details on the types of data to include in the report.

  6. Select Include items referenced as exhibits (split screen, pop up exhibits) check box if you want to include exhibits in your custom report.

    Seen below is an example of a Custom Report generated in the Excel plain text format after selecting All Data.

    The following table explains the various columns of this report.

    Column

    Description

    Question ID

    Question ID of the item.

    Blueprint IDBlueprint ID of the item.
    Blueprint nameName of the blueprint assigned to the item.
    External IdExternal ID if the item was imported from outside of ExamDeveloper.
    LanguageLanguage assigned to the item.
    Latest revisionThe most current revision number for the item.
    Other blueprintsThe other Blueprints of the item. This is populated when the item has Blueprint ID from another project.
    StateState of the item in the review workflow.
    Translation groupThe translation group number for the item.
    TypeType of the item.
    Written byAuthor of the item.
    Written onDate and time on which the item was written.
    Number of optionsNumber of Answer Options for the item.
    Stem assetsThe assets included in the stem for the item.
    Stem contentStem text of the item.
    Option A, B, C, ... assetsThe name of the assets included in the option.
    Option A, B, C, ... contentText for Answer Options of the item.
    Option A, B, C, ... labelThe answer option label.
    Option A, B, C, ... shape labelThe option shape's label.
    Option A, B, C, ... initial responseThe text in the Initial Response field of a Constructed Response or Oral item type.
    Option A, B, C, ... initial response assetsThe assets in the Initial Response field of a Constructed Response or Oral item type.
    Correct optionsDisplays the Correct Answer for the item.
    CommentsComments of the item.
    Metadata - Category: Name 1, 2, 3, ....Metadata contents.
    ReferencesNames of references of the item.
    Case IDThe system-generated numeric IDs of the cases of which the item is a part.
    Case nameNames of the cases of which the item is a part.
    Cloned fromThe item from which this item was cloned.
    Cloned toThe items cloned from this item.
    CollectionsNames of the collections of which the item is a part.
    EnemiesQuestion ID of items with enemy association with the item.
    Exam formsThe names of the exam forms that include the item.
    Exhibit referenceIdentifies the Exhibit item number and the type of exhibit.
    Parent question IDDisplays the parent item, for R-Type question stems.
    SectionsNames of the sections of which the item is a part.
    Analysis set name (Default)Name of the default analysis set associated with the item.
    Analysis set name (Most Candidates)Name of the most candidates analysis set associated with the item.
    Analysis set name (Latest)Name of the latest analysis set associated with the item.
    CandidatesNumber of candidates who answered the items.
    DifficultyDifficulty value for the item.
    DiscriminationDiscrimination value for the item.
    IRTa, b, cIRT values.
    Option A, B, C, ...- CandidatesNumber of candidates who selected Answer Option A, B, C, ....
    Option A, B, C, ...- DiscriminationDiscrimination value for Answer Option A, B, C, ....
    Option A, B, C, ...- ScoreScore for Answer Option A, B, C, ....
    Average angoffAverage Angoff Rating (Average Angoff Rating across all raters who rated the current version of this item).
    Average criticalityAverage Criticality Rating (using 0 -4 rating across all raters of current version of the item).
    Average frequencyAverage Frequency Rating (using 0 -4 rating across all raters of current version of item).
    Average importanceAverage Importance Rating (using 0 -4 rating across all raters of current version of item).
    Number of ratersNumber of raters who were used in calculating the averages listed above.
    Has one correct answer?Identifies (Yes or No) if the item has one correct answer.
    Has one verifiable reference?Identifies (Yes or No) if the item has one verifiable reference.
    Has one correct exam blueprint?Identifies (Yes or No) if the item has one correct exam blueprint.
    RatersAll the validators whose ratings were used in calculating the values above.

 

  • No labels