At the conclusion of the survey, the subject matter experts ratings are combined together to create an exam blueprint which specifies the number of questions needed for each content area of the JTA outline.
This blueprint serves as the roadmap for the item development and examination assembly process and ensures that the examination reflects the relative importance of the required knowledge and skills required to perform the role competently.
I have survey data...now what?
One must determine a Relevance Metric for the data
When the completed surveys are ready to be analyzed, one must determine a weighting of “Relevance Metric” for the contents in the JTA outline. There are a number of ways to create this metric. Perhaps the easiest method is to combine the average of Importance, Criticality and Frequency ratings for each statement in the JTA outline. The statements with the lowest ratings can be dropped from the blueprint as they don't warrant examination content creation. The resulting number for each statement determines the number of test questions to be written in that area. A task with a 3.0 average requires twice as many questions written than a task with a 1.5 average.
The final step of a JTA study is the documentation of the entire process in a technical report. This documentation is an important step in guarding against legal challenges to your exam. This report contains an overview of the stakeholder strategy discussion, the process used to develop the JTA outline, the validation data associated with the JTA outline, and the final exam blueprint.