Navigation Menu

Click the "+" to see inside a chapter or use the search to the right.


Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: clean up

After the domains are weighted and averaged by the SME panel, what's next? Validation!

This needs to must be done to make sure ensure that the domains, task statements, and knowledge and skill statements are acceptable by others in the profession. In order to confirm (or modify) a panel’s work, a validation study is the next necessary step and is highly recommended. You certainly hope that the work performed by your initial panel was perfect, but we all understand that's rarely the case. The validation study serves as the check and balance element of the JTA by the population at large.



Part of the JTA survey is a demographic section to allow for analysis of potential sub-group differences in the target population to see if there are differences in how people perceive the work. One variable which potentially could impact how a person views related duties and requirements is his or her amount of job experience. As people gain more job experience, their job roles can change to include more complex or higher-level tasks. Other common variables include education, salary, size of organization, and number of employees supervised. As such, it is important to compare the task ratings of individuals across each of the variables to make sure that everyone views the importance, criticality, and frequency similarly.


The field professionals will rate the domains and tasks that the SMEs created on the same requirements, Importance, Criticality, and Frequency.

After the survey results are returned back, the average of all the ratings, both the SMEs and the field professionals are compiled and averaged out. With the average, the blueprint for the exam will be created. These averages will show how many questions should be dedicated to each domain on the exam. The higher the rating, the more questions that should cover that topic.

After the blueprint is completed, the writing part of the exam can begin.



Validation Survey: Summary

  • Convert the JTA outline to a survey
  • Administer to sample of practitioners in the field -  
Need to
  • Must be representative of the field (demographics, experience, education, etc.)
  • Ask respondents to rate the importance, criticality and frequency of each statement in the JTA outline
  • Combine ratings into a “Relevance Metric”
  • Compare responses with that of the original panel


Here is a snapshot of typical validation scales and other examples that one can use in a validation survey:


Typical Validation Scales




How essential is this activity during competent performance of this profession?

0        1        2        3        4

(not important to most important)


How much harm can be caused by an inability to perform this activity during competent performance of this profession? (e.g. legal, financial, psychological, physical, or other forms of harm)

0        1        2        3        4
(no harm to extreme harm)


How frequently does a competent professional spend time executing this activity during competent performance of this profession?

0        1        2        3        4
(never performed to very frequent)




These validation scales can also be appropriate when validating training content.





Other Validation Scales



Need for Instruction:

How necessary is it that practitioners receive training about this duty or procedure?

0        1        2        3        4

(not necessary to essential)

Difficulty of Learning:

How difficult is it for practitioners to gain competence with this duty or procedure? 

0        1        2        3        4
(not difficult to very difficult)

Point of Acquisition:

At what point in their careers do practitioners acquire this duty or procedure? 

0        1        2        3        4
(before 1st day on job to late in career)