Navigation Menu

Click the "+" to see inside a chapter or use the search to the right.

.

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 59 Next »

Disclaimer: in order to ensure we make the right decisions for the future of the application, we may change or remove the information that appears here at any time

Coming soon to ExamDeveloper

Custom Report JSON Export

 

Available from

1807

Rollout pace

Gradual rollout - users will have visibility to limited features as a result of this upgrade

Last Updated 

 

Generate Reports and Custom Reports (Set my own configuration option) pages will be enhanced to include a new Formatting type option - JSON export. Exported data will be in JSON (JavaScript Object Notification) open standard file format. Generated output will be a zip file having multiple .json files, containing item data.

Following item contents can be exported in JSON format:

  • Default Data: Blueprint ID, Blueprint name, External ID, Language, Latest revision, Other blueprints, Question ID, State, Translation group, Type, Written by, Written on
  • Content: Case lead-in text, Number of options, Option assets, Option content, Option label, Option shape label, Stem assets, Stem content
  • Key: Correct options
  • Metadata: Comments, Metadata, References
  • Associations: Case ID, Case name, Cloned from, Cloned to, Collections, Enemies, Exam forms, Exhibit Reference, Parent question ID, Sections

Statistics and Validation data content will not be available for JSON export.

In near future, Exhibits will also be part of the JSON export. 

Removal of old reports from ExamDeveloper

 

 

Available from

1807 (approximately July 2018)

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

Last updated

 

 

In order to ensure reporting is more useful in ExamDeveloper in the future, we are looking to streamline the ‘additional reports’ available to Managers. A number of these were created to serve a specific purpose many years ago and are not believed to be used by anyone anymore.

The following reports are planned for removal from the Manage → Reports → Additional Reports tab in the application:

  • Blueprint Distribution Exam

  • Exam Item Statistics

  • Item Counts By Blueprint

  • Item Statistics Report

  • Outline Items

  • Pearson Exam Question Details

  • Validation Data Report

Please note that if you are using these reports, please reach out to examdeveloper@pearson.com in the first instance, so that we can set up some time to understand your needs and explore how we might best serve them.

Improvements to the way revision numbers are incremented

 

Available from

1807 (approximately July 2018)

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

Last updated 

 

In order to ensure we can support larger item banks in the application, we are changing the way revisions are stored behind-the-scenes. As part of this change, we have an opportunity to change the way that question revision numbers are incremented, as they are currently randomly set by the application whenever a new question revision is created. 

We will be moving to a more user-friendly system, which increments the question revision number by one each time a new revision is created, rather than assigning a random number.

  • New questions will have a logical revision number that indicates the iteration of the question, starting at one and increasing by one each time a new revision is created.

  • Existing questions will no longer be assigned a random revision number, but instead will start to increment by one each time a new revision is created (e.g., if the latest revision number of a question is 133905, the next revision number will be 133906, then 133907, etc. after this change.)

  • Existing revision numbers will not be altered.

Improvements to question history and user tracking

 

Available from

1807 (approximately July 2018)

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

Last updated14-June-2018 (changes/additions highlighted in blue)

 

In order to ensure we can support larger item banks in the application, we are changing the way revisions are stored behind-the-scenes. As part of this change, we have an opportunity to improve the way that we display question user tracking and history tabs to end users.

Changes to the question history tab:

  • When nothing is changed about a question and a user submits the question, a new revision will not be created (i.e. the system will no longer create duplicate revisions.)

  • A change to a question will cause a new question revision to be created, including changes to any of the following types of question data:

    • Content (stem, options, labels, etc.)

    • Type

    • State (so that a history of state/workflow history changes can be tracked.)

    • Language

    • Scoring information (weight, correct answers, etc.)

    • Blueprint

    • Metadata

    • References

    • Advanced Properties

  • Modifying any of the following associated question information will not create a new question revision:

    • Exhibits

    • Enemies

    • Statistics

  • Adding a comment will append it to the current revision, unless it’s being added as part of a change that creates a new revision (e.g. a comment added alongside a change to stem). In the latter scenario, the comment will be added to the new revision containing the change.

  • A new grid will be implemented, which displays:

    • the revision number.

    • the state of the question for the revision displayed.

    • the user who created the revision (i.e., the user who wrote/modified the question.)

    • the time the revision was created (i.e., the time the user wrote/modified the question.)

  • It will be possible to select a single question revision and view it without comparing changes against the previous revision.

  • Users will be able to compare changes between any two revisions (e.g. revision 2 vs. 5.)

  • The track changes feature will begin to work for the following additional data:
    • Metadata

    • Weight

    • Border/Background configuration (Matching questions)

    • Token/Target headers (Matching questions)

    • Cloned to / from

    • Translated languages

Changes to the user tracking tab:

  • Written by and Modified by will be removed, as they are now available on the history tab for every revision that is created.

  • The Reviewed by/Accepted by/Rejected by labels will be merged into a single layout that more clearly outlines the workflow history of the question.

Improvements to question batch edit processing

 

Available from

1807 (approximately July 2018)

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

Last Updated14-June-2018

 

The batch edit page is a powerful page for applying updates to many questions en-masse, but when one of those question fails to update, all questions fail and all changes are rolled back unnecessarily. We have taken steps to ensure that a batch edit processes changes where it can and gives users greater feedback about what and why certain questions couldn’t be updated, so they can focus on correcting the remainder left over, rather than having to sort through the entire batch.

Changes to the question batch edit screen in ExamDeveloper:

  • Only one change can be made for each batch edit that is run (i.e., I can change blueprint or stem text in separate batch edits, but not in the same batch edit), with the option of adding a comment that describes the change being made.

  • A progress bar will show how many items have been processed as the batch edit runs.

The following changes impact the way question batch edits are processed:

  • If changing a question does not cause an error, it will be processed and updated.

  • If changing a question does cause an error, it will be skipped and the process will continue with the next question in the batch (instead of causing the entire process to fail.)

  • At the end of the batch edit, if any questions failed, an Excel report will be automatically downloaded that details the questions that failed to update, giving reasons why the update failed.

Changes to import templates

 

Available from

1807 (approximately July 2018)

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

Last updated

 Added- 14-June-2018

 

The following templates will be updated to take advantage of modern Microsoft standards. Consequently, any existing templates held offline will no longer work and will need to be re-downloaded through ExamDeveloper:

  • Question blueprint import: Manage → Import  Question Blueprint  Upload [XLSX]

  • Question external ID import: Manage  Import  Question ExternalID  Upload [XLSX]

  • Question metadata import: Manage  Import  Question Metadata  Upload [XML]

Users may notice some slight look-and-feel changes to the new templates.

Furthermore, the above imports will process items similar to batch edits, meaning:

  • If changing a question does not cause an error, it will be processed and updated.

  • If changing a question does cause an error, it will be skipped and the process will continue with the next question in the batch (instead of causing the entire process to fail.)

  • At the end of the batch edit, if any questions failed, an Excel report will be automatically downloaded that details the questions that failed to update, giving reasons why the update failed.

  • A re-importable template containing just the failed questions will also be automatically downloaded, so that any remaining issues can be fixed and uploaded again.

Calculator on an individual item

 

Available from

1807 (approximately August 2018)

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

Last Updated 

 

The current functionality to add a calculator today is to add the calculator to a Section or Sections in the QTI Builder. 

  • Authors will now have the ability to set the calculator on an individual item in the Advanced Properties.

  • Visibility, Editability, and Calculator defaults can be set in the project configuration page.

  • In the QTI Builder, users will have the ability to set a Calculator at the Exam / Assessment level that can be referenced from each of the items along with any of the Sections.

  • Current calculator features will remain functional.

Removal of the Edit Exam Group page

 

Available from

1808 (approximately August 2018)

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

Last Updated 

In order to make ExamDeveloper more streamlined, we are retiring the Edit Exam Group page, which has very low usage. Most of the functionality (addition / modification and deletion) on this page is redundant and is available elsewhere in ExamDeveloper.

Word DOCX Export

 

Available from

1807 (approximately July 2018)

Rollout pace

Gradual rollout - users will have visibility over several releases

 


ExamDeveloper currently has 2 Microsoft Word export formats which use the Microsoft Word proprietary legacy format “.doc”. Starting in 2007 with the updated version of Word (and Microsoft Office), the default save format for Microsoft Word has changed to DOCX. (The extra X stands for the Office Open XML standard.) ExamDeveloper is being enhanced to allow export of Exams in the the .docx format. The .docx format will initially contain the Exam Key and Metadata information which then becomes optional with a future release. The export supports English items only and other languages will be added in future releases.

 

Ability to Search on Manage Item Bank - Add Content - Collections

 

 

Available from

1808 (approximately August 2018)

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

 

Add/Remove content feature, available through Manage Item Bank screen, does not allow users to filter collections fulfilling certain criteria thus restricting its use. As a result, we intend to bring search functionality for Collections similar to how it exists for Cases & Questions.

 
  • The change makes it possible to search for Collections on Add/Remove content screen.

  • It will not be possible to save a search with certain criteria.

 

Removal of the Edit Exam Group page


Available from

1808 (approximately August 2018)

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

Last updated

 

 

In order to make ExamDeveloper more streamlined, we are retiring Edit Exam Group page which has very low usage.  Most of the functionality (addition / modification and deletion) on this page is redundant and is available elsewhere in the ExamDeveloper.

The Edit Exam Group page will be replaced by the ability to allow users to change the Exam Group name, which can not be edited anywhere else in the application.

Cases: Separating the split screen exhibit from the case

 

Available from

1808 (approximately August 2018)

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

Last updated14-June-2018

 

Case lead-in text is intimately linked with the case, but this makes it difficult to define whether a case is a question type, a grouping of questions, an exam section or a supporting display exhibit, which make cases near-impossible to enhance and maintain without impacting other areas of the application.

We are taking steps to more robustly define and separate the individual components that make up a case, so that we can deliver additional case enhancements in future. This will result in the following initial changes:

  • New and existing case lead-in text and language will be treated as a display-only question type (referred to now as a case exhibit) with its own history and Question ID.

  • Users will have the ability to search for case exhibits and set metadata, enemies, blueprints, advanced properties, etc. against them, but configuring this data will not initially be available on the add/edit case page.

  • The case exhibit blueprint will be ‘Unassigned’ by default.

  • The case exhibit language and lead-in text will remain attached to the case, editable via the same pages, and deliverable to candidates in the same way.

  • Case exhibits will show up in the split screen exhibits menu when modifying a case question alongside stand-alone exhibits, and it will be possible to sequence the tab display order alongside stand-alone exhibits that don’t belong to the case, for that question.

  • When a case is deleted, it will be possible for the case exhibit to remain attached to the case questions in the project (as a stand-alone exhibit), but the ‘grouping’ of questions will no longer exist.

  • When a question is removed from a case, the case exhibit will be removed from that question automatically.

  • It will be possible for writers to submit the case exhibit for review (it will be reviewable independently like any other display-only question type), and they will not be able to edit the case exhibit while it’s not in a Draft or Rejected state.

  • Clicking ‘Submit All’ on the add/edit case page will also submit the case exhibit for review, alongside any case questions.

  • Case enemies and language will become part of the case exhibit. Consequently, exams and sections will check for exhibit enemies when saving.

  • A new case revision will be created whenever a user:

    • Changes the case name.

    • Adds or removes a question on the case.

    • Reorders questions on the case.

    • Changes the lock response configuration for questions on the case.

  • Changes to case exhibits and questions on the case will not create a new case revision, but they will create a new question revision.

  • A user will always see the latest revision of questions and case exhibits that are attached to the revision of the case they’re viewing.

  • When sharing case questions or case exhibits across projects, then the case (and other questions on the case) will be automatically shared in the same way.

  • When performing a question search by Case Name, if no case name is provided then the search will return questions that are on any case in the current project, instead of all questions in the project (regardless as to whether they are on a case or not.)

  • It will no longer be possible to search for assets by the ‘On Case’ criteria.

  • It will no longer be possible to search for questions by the ‘Order on Case’ criteria.

  • When performing a case search:

    • It will be possible to search by Case ID (which will no longer contain the case exhibit language), Case name, Case exhibit language, and Case exhibit text, and On exam form on the view cases, build section, build exam and case project sharing pages.

    • It will no longer be possible to search for cases using the following criteria: Editable On, Viewable On, Written By, Date Written, Date Modified, On Exam Section, Question ID, Blueprint ID, Type, Stem Text, Option Text, Question Metadata, Comments, External Id, Reviewed by, On Collection (Note that this does not affect searches that return questions instead of cases.)

  • The error that prevents cases from being saved if it contains two questions that are enemies of each other will be removed.

  • When importing an enemy relationship for a question or exhibit that is on a case, other questions on the case will no longer automatically be made enemies (e.g., Item A is made an enemy of Item B, which is on a case with Item C; Item A will no longer automatically be made an enemy of Item C, but it will be possible to set this relationship manually or by using the enemy import spreadsheet).

  • It will be possible to see how many questions are on the case on the view cases page.

  • The case ‘type’ column will be removed from the application.

  • The section and exam imports (in XML and XLS format) will allow users to add case questions using the <question> element instead of the <case> element.

  • On the enemy import spreadsheet, if a user sets a question as an enemy of a question belonging to a case, ExamDeveloper will no longer make other questions on the case enemies automatically. Users will still be able to define enemy relationships with other case questions manually via the spreadsheet.

  • Case lead-in text will be removed from the newest custom report.

  • The question import templates will be updated to create cases in a different way:

    • On the spreadsheet import, the cases tab will be removed and replaced with columns to indicate the case name on the questions tab.

    • On the XML import, <case> will be removed and instead case exhibits will be created using <question> (marked as a display type). It will be necessary to reference the case name from both the case exhibit (using a new <case> element), as well as from any questions that should be attached to the case (using the existing <exhibit> element).

Cases: adding a subset of case questions to a section or exam

 

Available from

1808 (approximately August 2018)

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

Last updated

 Added 14-Jun-2018

 

Today, if a case is added to an exam or section, then all questions on the case must be added. We intend to enhance the application to the point where it is possible to add individual case questions to an exam, without forcing the user to add all other questions on the case. This will result in the following behaviour changes to the add/edit exam and section pages:

  • The ‘Add cases’ tab will be removed.

  • It will be possible to search for case questions on the ‘Add questions’ tabs, and add them individually to the exam (like any standalone question works today.)

  • It will be possible to see which case a question belongs to on the build section / exam pages.

  • If case questions are on a section or exam, and more questions are added to the case, these will not automatically be added to the section/exam.

  • If case questions are on a section or exam, and those questions are removed from the case (or the case is deleted), these will not automatically be removed from the section/exam.

  • The section and exam imports (in XML and XLS format) will allow users to add a subset of case questions.

  • When adding case questions directly to an exam, if the project configuration setting ‘Export cases as sections’ is checked, then a section will automatically be created for each case being added. (If the setting is unchecked, then case questions will continue to be added to the ‘Unsectioned’ section that is automatically created.)

  • When an exam or section is published, the latest revision of standalone exhibits will now be ‘locked in’, just like case exhibits and questions are. (Note that assets will still not be a part of the locked in content.)

  • When saving a section or exam, there will be a check for exhibits that are enemies of each other, or of other questions, on the same section / exam.

  • The ‘Selected’ tab will be removed.

  • It will be possible to reorder questions and sections on a new tab named ‘Contents’, which will replace the existing ‘Questions’ tab.

  • It will no longer be possible to create more than one ‘Unsectioned’ section (e.g. in the instance where a user had a selected tab contains a question followed by a case or section, followed by another question.)

  • When new questions are added to an exam or section, ExamDeveloper will add them to the end of the existing section, starting with standalone questions (in Question ID order), then case questions (first by Case name, then by Case question order.)


Removal of ability to edit statistics on the statistics tab, when viewing a question

 

Available from

To be determined

Rollout pace

Full rollout - users will have immediate visibility as a result of upgrade

Last updated14-June-2018

 

From feedback we have received from statisticians/psychometricians, the current ability to import and export statistics in ExamDeveloper is the preferred process for statistics management, and they’d like to ensure that other users do not have the ability to alter statistics when viewing a question.

As a result of this feedback, whenever a user views a question in ExamDeveloper (such as on the Examine, Observe and Review Question pages, etc.) and they navigate to the statistics tab, they will continue to have the ability to select an analysis group / set to view statistics, however, it will no longer be possible to edit the statistics / default flag on these pages. 

Branching configurations in a review workflow

 

Available fromTo be determined
Rollout paceGradual rollout - users will have visibility over several releases
Last updated 

 

In ExamDeveloper it is currently possible to set up a review workflow that forces a reviewer to do one of two things:

  • Accept the question, which moves a question to the next state in the workflow
  • Reject the question, which moves a question to any previous state in the workflow.

We will enhance the configurability around this decision making process, to give managers greater control over where their reviewers can send questions in the workflow:

  • ‘Accept’ and ‘Reject’ will be renamed ‘Advance’ and ‘Return’ respectively, in order to indicate the direction that an item is travelling in through the workflow.
  • For each review state, project managers will be able to configure the ‘outcomes’ that are available to the reviewer when they decides to advance or return an item:
    • An outcome is given a custom name.
    • It will be possible to configure what the next state will be for a given outcome, including Returned (previously ‘Rejected’), other Review states, Validate, Completed and Obsolete.
    • Outcomes will make it possible to:
      • Allow reviewers to skip a state when advancing a question.
      • Restrict the reviewer to a specific state or states when returning a question; or offer no ability to return a question.
      • Require a reviewer to add a comment when advancing a question.
      • Configure whether the reviewer should decide to retain statistics or not (i.e. automatically retain them without offering the choice.)
    • If multiple reviewers are required to advance or return a question, then the last reviewer required to trigger the state change will continue to choose the outcome.
  • A reviewer will be able to modify a question during review and save their changes without that modification automatically counting as advancing the question.
  • Question history  / user tracking will reflect the outcomes and path that a question took through the review workflow at the time, even if the project’s review workflow configuration is modified later.
  • The user tracking tab will continue to reflect the review workflow in the order that it is sequenced, meaning:
    • If an outcome returns a question to a previous state in the workflow, then user tracking is reset back to the new state.
    • If an outcome advances a question through the workflow, and a state is skipped, then the latest question revision will show that skipped state as ‘Not reviewed’.
  • Existing review workflow configurations will continue to behave as they currently do, but can be updated to take advantage of the new functionality.

 



  • No labels