Navigation Menu

Click the "+" to see inside a chapter or use the search to the right.

.

Skip to end of metadata
Go to start of metadata

The Custom Report displays information related to the current version of items such as state, text, usage, metadata, validation data, statistics, and enemies. The report can display this data for multiple items when they are selected on the Search Questions page and you click the Generate Report button.

  1. Navigate to Manage → Project Bank to open the Search Questions page.
  2. Retrieve the items for which you wish to generate the Custom Report, or simply select the required items from the search result via corresponding check boxes.

    Icon
     How does the selection and pagination of search results work?

    Users can optimize the number of search items per page by selecting a value from the Items Per Page drop-down; this number ranges from 10 to 50 per page. The default number is 20.

    The selected number of search items is displayed in the grid below the drop-down.

    The search items per page can be changed and viewed at any time for the user's convenience.

    The search results are listed on multiple pages according to the selected number of Items Per Page in the drop-down, if the total number of search items resulting from the search criteria exceeds the selected number of items per page.

    The user can also directly jump to another page of the search results grid by clicking its corresponding link above the search results grid.

    The Selection drop-down in the left of the header row has four selection options: None, Some, Page, and All.

    This Selection drop-down is present on pages in which users need to select multiple search items for use such as viewing, modifying, etc.

    The Selection drop-down options perform the following functions:

    ColumnDescription
    NoneAll items in the list across all pages of the search results grid are deselected.
    Page

    All items on the currently visible page of the search results grid are selected.

    If you move to another page of the search results grid, you will see that none of the items on that page are selected, but the items selected on the earlier page are still retained.

    Some

    This selection appears when you manually select a few items from the list by selecting their corresponding check boxes. This selection also appears if you select Page from the drop-down and then increase the Items per page, or if you select All from the drop-down and then deselect any item on the search results page.

    It is also possible to select items across multiple pages of the search results grid by simply making the selection on one page and moving on to another - the item selection made on the earlier page is retained even if you move to another page to make additional selections.

    All

    All items in the list across all pages of the search results grid are selected.

    Even when moving to any other page of the search results grid, all the items on that new page are also selected.

    The selection drop-down remains as All if any of the items are not deselected, and it changes to Some if any of the items on any of the pages of the search results grid are deselected. Furthermore, if the deselection is made on the first page of the search results grid, then all the items on other pages are deselected. However, if you move to another page of the search results grid while the selection drop-down is All and deselect any item, the selection drop-down for that page changes to Some, while all items on the remaining pages remain selected with the selection drop-down selection showing as Page on those pages.



  3. Click the Generate Report button, you are taken to the Custom Reports page, where you can select Custom ReportBlueprint Validation Angoff Information by Item, or Question Longitudinal Analysis report from the Select Report drop-down. Select Custom Report here.

  4. Select one of the three available Data Group(s) for which you wish to generate the report by means of the drop-down.
    • Set my own configuration includes Question ID, Question Type, Written By, Name of the Case on which the Question exists, Stem and Options text and images, Option labels and shape labels, References, Metadata, Blueprint, Comments, Enemies, Parent Question ID (in case of R-Type Question Stems), Latest Revision ID, External ID, Question State, and Exhibits. (This is the default data group.)
    • Question Data (Excluding Question Text/Stem) is the default selection. This custom report includes the data that is currently part of the following data:
      ■ Question Attributes: Question State, Options, Correct Answer, Blueprint ID, Assets, Question Type, and Written By.
      ■ Question Usage: Exam Forms, Sections, Cases, Case Id, Collections, and Analysis Sets.
      ■ Question Metadata: Metadata details, and References.
      ■ Question Comments: Comments.
    • Question and Statistics Data includes all the data from the Question Data (Excluding Question Text/Stem) selection, in addition to the following data:
      ■ Statistics (Question Level): Analysis Set Name, Difficulty (p), Discrimination (rpb), IRT (A,B,C), and Candidates (N).
      ■ Statistics (Option Level): Candidates (N), Score, and Effectiveness (rbp).
      ■ Statistics (Custom): Custom columns uploaded with Analysis Sets (Question & Option Level).
    • All Data includes everything included in the above two data groups, in addition to the following:
      ■ Question Text: Stem text, Option text, and Case Lead-In text.
      ■ Validation Data: Question Validation : Avg. AngOff, Avg. Importance, Avg. Frequency, Standard Settings : Avg. AngOff, #Raters, and Raters.
      ■ Question Enemies: Enemies.


  5. If you choose Set my own configuration data group, there are four formatting options available in the drop-down list for this report and the resulting reports have all the above-mentioned information columns with some differences.
    Excel plain textText where the Rich Text Editor has been used is presented in the rendered form (plain text). This report is generated in the Excel format on the Queue page.
     Excel with HTML markupText where the Rich Text Editor has been used is presented in the form of HTML code. This report is generated in the Excel format on the Queue page.
    ■ HTML formatted for the web browserText where the Rich Text Editor has been used is presented in the rendered form (plain text). This report is generated in the HTML format on the Queue page.
    ■ JSON: Text that identifies information about the question. This report is generated in the JSON format on the Queue page.
       See JSON Report Format topic for details on the JSON formatting option.

    The user can also select the type of data from the following:
    ■ Default: This is preselected and cannot be deselected. Data includes: Blueprint ID, Blueprint name, External ID, Language, Latest revision, Other blueprints, Question ID, State, Translation group, Type, Written by, Written on.
    ■ Content: Number of options, Option assets, Option content, Option label, Option shape label, Stem assets, Stem content.
         All but the following item type contents can be exported in a custom report: External item, Spreadsheet.
    ■ Key: Select the Key option to include the 'correct options' in the custom report.
    ■ Metadata: Comments, Metadata, References.
    ■ Associations: Case ID, Case name, Cloned from, Cloned to, Collections, Enemies, Exam forms, Exhibit reference, Parent question ID, Sections.
    ■ Statistics: Analysis set name, Candidates, Custom statistics, Difficulty, Discrimination, IRT, Option candidates, Option custom statistics, Option discrimination, Option score. This also requires selection of Analysis Group and type of Analysis Set (Default, Most Candidates, or Latest).
    ■ Validation: Average angoff, Average criticality, Average frequency, Average importance, Number of raters, Has one correct answer?, Has one verifiable reference?, Has correct exam blueprint?, Raters.
    ■ Include items referenced as exhibits: include contents of all the items referenced in the exhibits.

    See the Custom Report topic for details on the types of data to include in the report.

  6. Select Include items referenced as exhibits (split screen, pop up exhibits) check box if you want to include exhibits in your custom report.
  7. If you select the data group Question and Statistics Data or All Data, drop-downs for Analysis Group and Analysis Set type appear.

    Select the required Analysis Group and the type of the Analysis Set (LatestDefault, or Most Candidates).

    Icon

    The report is not exported and an error message, "Please Select Analysis Group" is displayed if you try to export a Custom Report after selecting the data group Question and Statistics Data or All Data without selecting any Analysis Group from the drop-down.


    To make them portable, the reports are exportable in a variety of formats. Select the file format for your report from the Select Format drop-down. The available file formats are Excel, CSV, XML, MHTML, PDF, Word, and TIFF. Click the Export button. This sends you to the Queue page where the requested report is placed in a report generation queue along with any other already requested report(s). You can click the Cancel button if you do not wish to generate a report and return to the Search Questions page.
     

    Icon

    The report is not exported and an error message, "Please select report format to export" is displayed if you try to export a report without selecting the format.



    Seen below is an example of a Custom Report generated in the MHTML format after selecting All Data.


    The following table explains the various columns of this report.

    Column

    Description

    Question Number

    Question ID of the item.

    Parent Question

    Displays the parent item, if any.

    Question Revision

    Revision Code of the item.

    External Id

    External ID if the item was imported from outside of ExamDeveloper.

    Unique IdThis is the Unique ID of the item in the "Language Group.Language Code" format.

    State

    State of the item.

    # of Options

    Number of Answer Options.

    Correct Answer

    Correct Answer Options for the item.

    Blueprint

    Blueprint ID of the item.

    Assets

    Name of asset in the item, if any.

    Type

    Type of the item.

    WrittenBy

    Author of the item.

    Stem Text

    Stem text of the item.

    Option Text

    Text for Answer Options of the item.

    Case Lead-in Text

    If the item is part of a case, then the case lead-in text of the case.

    Exams

    Names of the exams of which the item is a part.

    Sections

    Names of the sections of which the item is a part.

    CaseId

    The system-generated numeric IDs of the cases of which the item is a part.

    CasesNames of the cases of which the item is a part.

    Collections

    Names of the collections of which the item is a part.

    AnalysisSet Name - (Latest)

    Name of the latest analysis set associated with the item.

    Reference

    Names of references of the item.

    Comment

    Comments of the item.

    Enemies

    Question ID of items with enemy association with the item.

    Avg Angoff

    Average Angoff Rating (Average Angoff Rating across all raters who rated the current version of this item).

    Avg Importance

    Average Importance Rating (using 0 -4 rating across all raters of current version of item).

    Avg Frequency

    Average Frequency Rating (using 0 -4 rating across all raters of current version of item).

    # of Raters

    Number of raters who were used in calculating the averages listed above.

    Raters

    All the validators whose ratings were used in calculating the values above.

    AnalysisSet Name - (Latest)

    Name of the latest analysis set associated with the item.

    Difficulty

    Difficulty value for the item.

    Discrimination

    Discrimination value for the item.

    IRTa, b, c

    IRT values.

    Candidates

    Number of candidates who answered the items.

    Metadata - Category: Name 1, 2, 3, ....Metadata contents.

    Option A, B, C, ...- Candidates

    Number of candidates who selected Answer Option A, B, C, ....

    Option A, B, C, ...- Score

    Score for Answer Option A, B, C, ....

    Option A, B, C, ...- Discrimination

    Discrimination value for Answer Option A, B, C, ....

    Question CustomName- custom1, 2, 3, …

    Custom name for

    Option A CustomName-

    Custom name for

    Option A CustomName- custom opt 1, 2, 3, ...

    Custom name for



  • No labels