Navigation Menu

Click the "+" to see inside a chapter or use the search to the right.

.

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Updated standard Custom report fields

...

  1. Navigate to Manage → Project Bank to open the Search Questions page.
  2. Retrieve the items for which you wish to generate the Custom Report, or simply select the required items from the search result via corresponding check boxes.

    Info
    Expand
    titleHow does the selection and pagination of search results work?

    Excerpt Include
    7.4.1.1 Search Results Pagination and Selection
    nopaneltrue



  3. Click the Generate Report button, you are taken to the Custom Reports page, where you can select Custom ReportBlueprint Validation Angoff Information by Item, or Question Longitudinal Analysis report from the Select Report drop-down. Select Custom Report here.

  4. Select one of the four formatting options available in the drop-down list for this report:
    Excel plain textText where the Rich Text Editor has been used is presented in the rendered form (plain text). This report is generated in the Excel format on the Queue page.
    Excel with HTML markupText where the Rich Text Editor has been used is presented in the form of HTML code. This report is generated in the Excel format on the Queue page.
    HTML formatted for the web browserText where the Rich Text Editor has been used is presented in the rendered form (plain text). This report is generated in the HTML format on the Queue page.
    JSONThis report is generated in the JSON format on the Queue page.
         See JSON Report Format topic for details on the JSON formatting option.
  5. Select the type of data to include in the report from the following:
    ■ Default: This is preselected and cannot be deselected. Data includes: Blueprint ID, Blueprint name, External ID, Language, Latest revision, Other blueprints, Question ID, State, Translation group, Type, Written by, Written on.
    ■ Content: Number of options, Option assets, Option content, Option label, Option shape label, Stem assets, Stem content.
      All but the following item type contents can be exported in a custom report: External item, Spreadsheet.
    ■ Key: Select the Key option to include the 'correct options' in the custom report.
    ■ Metadata: Comments, Metadata, References.
    ■ Associations: Case ID, Case name, Cloned from, Cloned to, Collections, Enemies, Exam forms, Exhibit reference, Parent question ID, Sections.
    ■ Statistics: Analysis set name, Candidates, Custom statistics, Difficulty, Discrimination, IRT, Option candidates, Option custom statistics, Option discrimination, Option score. This also requires selection of Analysis Group and type of Analysis Set (Default, Most Candidates, or Latest).
    ■ Validation: Average angoff, Average criticality, Average frequency, Average importance, Number of raters, Has one correct answer?, Has one verifiable reference?, Has correct exam blueprint?, Raters.
    ■ Include items referenced as exhibits: include contents of all the items referenced in the exhibits.

    See the Custom Report topic for details on the types of data to include in the report.

  6. Select Include items referenced as exhibits (split screen, pop up exhibits) check box if you want to include exhibits in your custom report.

    Seen below is an example of a Custom Report generated in the Excel plain text format after selecting All Data.

    The following table explains the various columns of this report.

    Column

    Description

    Question ID

    Question ID of the item.

    Blueprint IDBlueprint ID of the item.
    Blueprint nameName of the blueprint assigned to the item.
    External IdExternal ID if the item was imported from outside of ExamDeveloper.
    LanguageLanguage assigned to the item.
    Latest revisionThe most current revision number for the item.
    Other blueprintsThe other Blueprints of the item. This is populated when the item has Blueprint ID from another project.
    StateState of the item in the review workflow.
    Translation groupThe translation group number for the item.
    TypeType of the item.
    Written byAuthor of the item.
    Written onDate and time on which the item was written.
    Number of optionsNumber of Answer Options for the item.
    Stem assetsThe assets included in the stem for the item.
    Stem contentStem text of the item.
    Option A, B, C, ... assetsThe name of the assets included in the option.
    Option A, B, C, ... contentText for Answer Options of the item.
    Option A, B, C, ... labelThe answer option label.
    Option A, B, C, ... shape labelThe option shape's label.
    Option A, B, C, ... initial response The text in the Initial Response field of a Constructed Response or Oral item type.
    Option A, B, C, ... initial response assets The assets in the Initial Response field of a Constructed Response or Oral item type.
    Correct optionsDisplays the Correct Answer for the item.
    CommentsComments of the item.
    Metadata - Category: Name 1, 2, 3, ....Metadata contents.
    ReferencesNames of references of the item.
    Case IDThe system-generated numeric IDs of the cases of which the item is a part.
    Case nameNames of the cases of which the item is a part.
    Cloned fromThe item from which this item was cloned.
    Cloned toThe items cloned from this item.
    CollectionsNames of the collections of which the item is a part.
    EnemiesQuestion ID of items with enemy association with the item.
    Exam formsThe names of the exam forms that include the item.
    Exhibit referenceIdentifies the Exhibit item number and the type of exhibit.
    Parent question IDDisplays the parent item, for R-Type question stems.
    SectionsNames of the sections of which the item is a part.
    Analysis set name (Default)Name of the default analysis set associated with the item.
    Analysis set name (Most Candidates)Name of the most candidates analysis set associated with the item.
    Analysis set name (Latest)Name of the latest analysis set associated with the item.
    CandidatesNumber of candidates who answered the items.
    DifficultyDifficulty value for the item.
    DiscriminationDiscrimination value for the item.
    IRTa, b, cIRT values.
    Option A, B, C, ...- CandidatesNumber of candidates who selected Answer Option A, B, C, ....
    Option A, B, C, ...- DiscriminationDiscrimination value for Answer Option A, B, C, ....
    Option A, B, C, ...- ScoreScore for Answer Option A, B, C, ....
    Average angoffAverage Angoff Rating (Average Angoff Rating across all raters who rated the current version of this item).
    Average criticalityAverage Criticality Rating (using 0 -4 rating across all raters of current version of the item).
    Average frequencyAverage Frequency Rating (using 0 -4 rating across all raters of current version of item).
    Average importanceAverage Importance Rating (using 0 -4 rating across all raters of current version of item).
    Number of ratersNumber of raters who were used in calculating the averages listed above.
    Has one correct answer?Identifies (Yes or No) if the item has one correct answer.
    Has one verifiable reference?Identifies (Yes or No) if the item has one verifiable reference.
    Has one correct exam blueprint?Identifies (Yes or No) if the item has one correct exam blueprint.
    RatersAll the validators whose ratings were used in calculating the values above.

...